1 Tutorial: Image Gradient
2 ========================
8 This comprehensive (and long) tutorial will walk you through an example of
9 using GIL to compute the image gradients.
11 We will start with some very simple and non-generic code and make it more
12 generic as we go along. Let us start with a horizontal gradient and use the
13 simplest possible approximation to a gradient - central difference.
15 The gradient at pixel x can be approximated with the half-difference of its
16 two neighboring pixels::
18 D[x] = (I[x-1] - I[x+1]) / 2
20 For simplicity, we will also ignore the boundary cases - the pixels along the
21 edges of the image for which one of the neighbors is not defined. The focus of
22 this document is how to use GIL, not how to create a good gradient generation
25 Interface and Glue Code
26 -----------------------
28 Let us first start with 8-bit unsigned grayscale image as the input and 8-bit
29 signed grayscale image as the output.
31 Here is how the interface to our algorithm looks like:
35 #include <boost/gil.hpp>
36 using namespace boost::gil;
38 void x_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
40 assert(src.dimensions() == dst.dimensions());
41 ... // compute the gradient
44 ``gray8c_view_t`` is the type of the source image view - an 8-bit grayscale
45 view, whose pixels are read-only (denoted by the "c").
47 The output is a grayscale view with a 8-bit signed (denoted by the "s")
48 integer channel type. See Appendix 1 for the complete convention GIL uses to
51 GIL makes a distinction between an image and an image view.
52 A GIL **image view**, is a shallow, lightweight view of a rectangular grid of
53 pixels. It provides access to the pixels but does not own the pixels.
54 Copy-constructing a view does not deep-copy the pixels. Image views do not
55 propagate their constness to the pixels and should always be taken by a const
56 reference. Whether a view is mutable or read-only (immutable) is a property of
59 A GIL `image`, on the other hand, is a view with associated ownership.
60 It is a container of pixels; its constructor/destructor allocates/deallocates
61 the pixels, its copy-constructor performs deep-copy of the pixels and its
62 ``operator==`` performs deep-compare of the pixels. Images also propagate
63 their constness to their pixels - a constant reference to an image will not
64 allow for modifying its pixels.
66 Most GIL algorithms operate on image views; images are rarely
67 needed. GIL's design is very similar to that of the STL. The STL
68 equivalent of GIL's image is a container, like ``std::vector``,
69 whereas GIL's image view corresponds to STL range, which is often
70 represented with a pair of iterators. STL algorithms operate on
71 ranges, just like GIL algorithms operate on image views.
73 GIL's image views can be constructed from raw data - the dimensions,
74 the number of bytes per row and the pixels, which for chunky views are
75 represented with one pointer. Here is how to provide the glue between
80 void ComputeXGradientGray8(
81 unsigned char const* src_pixels, ptrdiff_t src_row_bytes,
83 signed char* dst_pixels, ptrdiff_t dst_row_bytes)
85 gray8c_view_t src = interleaved_view(w, h, (gray8_pixel_t const*)src_pixels, src_row_bytes);
86 gray8s_view_t dst = interleaved_view(w, h, (gray8s_pixel_t*)dst_pixels, dst_row_bytes);
90 This glue code is very fast and views are lightweight - in the above example
91 the views have a size of 16 bytes. They consist of a pointer to the top left
92 pixel and three integers - the width, height, and number of bytes per row.
97 Focusing on simplicity at the expense of speed, we can compute the horizontal
102 void x_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
104 for (int y = 0; y < src.height(); ++y)
105 for (int x = 1; x < src.width() - 1; ++x)
106 dst(x, y) = (src(x-1, y) - src(x+1, y)) / 2;
109 We use image view's ``operator(x,y)`` to get a reference to the pixel at a
110 given location and we set it to the half-difference of its left and right
111 neighbors. ``operator()`` returns a reference to a grayscale pixel.
112 A grayscale pixel is convertible to its channel type (``unsigned char`` for
113 ``src``) and it can be copy-constructed from a channel.
114 (This is only true for grayscale pixels).
116 While the above code is easy to read, it is not very fast, because the binary
117 ``operator()`` computes the location of the pixel in a 2D grid, which involves
118 addition and multiplication. Here is a faster version of the above:
122 void x_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
124 for (int y = 0; y < src.height(); ++y)
126 gray8c_view_t::x_iterator src_it = src.row_begin(y);
127 gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
129 for (int x=1; x < src.width() - 1; ++x)
130 dst_it[x] = (src_it[x-1] - src_it[x+1]) / 2;
134 We use pixel iterators initialized at the beginning of each row. GIL's
135 iterators are Random Access Traversal iterators. If you are not
136 familiar with random access iterators, think of them as if they were
137 pointers. In fact, in the above example the two iterator types are raw
138 C pointers and their ``operator[]`` is a fast pointer indexing
141 The code to compute gradient in the vertical direction is very
146 void y_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
148 for (int x = 0; x < src.width(); ++x)
150 gray8c_view_t::y_iterator src_it = src.col_begin(x);
151 gray8s_view_t::y_iterator dst_it = dst.col_begin(x);
153 for (int y = 1; y < src.height() - 1; ++y)
154 dst_it[y] = (src_it[y-1] - src_it[y+1]) / 2;
158 Instead of looping over the rows, we loop over each column and create a
159 ``y_iterator``, an iterator moving vertically. In this case a simple pointer
160 cannot be used because the distance between two adjacent pixels equals the
161 number of bytes in each row of the image. GIL uses here a special step
162 iterator class whose size is 8 bytes - it contains a raw C pointer and a step.
163 Its ``operator[]`` multiplies the index by its step.
165 The above version of ``y_gradient``, however, is much slower (easily an order
166 of magnitude slower) than ``x_gradient`` because of the memory access pattern;
167 traversing an image vertically results in lots of cache misses. A much more
168 efficient and cache-friendly version will iterate over the columns in the inner
173 void y_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
175 for (int y = 1; y < src.height() - 1; ++y)
177 gray8c_view_t::x_iterator src1_it = src.row_begin(y-1);
178 gray8c_view_t::x_iterator src2_it = src.row_begin(y+1);
179 gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
181 for (int x = 0; x < src.width(); ++x)
183 *dst_it = ((*src1_it) - (*src2_it)) / 2;
191 This sample code also shows an alternative way of using pixel iterators -
192 instead of ``operator[]`` one could use increments and dereferences.
197 Unfortunately this cache-friendly version requires the extra hassle of
198 maintaining two separate iterators in the source view. For every pixel, we
199 want to access its neighbors above and below it. Such relative access can be
200 done with GIL locators:
204 void y_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
206 gray8c_view_t::xy_locator src_loc = src.xy_at(0,1);
207 for (int y = 1; y < src.height() - 1; ++y)
209 gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
211 for (int x = 0; x < src.width(); ++x)
213 (*dst_it) = (src_loc(0,-1) - src_loc(0,1)) / 2;
215 ++src_loc.x(); // each dimension can be advanced separately
217 src_loc+=point<std::ptrdiff_t>(-src.width(), 1); // carriage return
221 The first line creates a locator pointing to the first pixel of the
222 second row of the source view. A GIL pixel locator is very similar to
223 an iterator, except that it can move both horizontally and
224 vertically. ``src_loc.x()`` and ``src_loc.y()`` return references to a
225 horizontal and a vertical iterator respectively, which can be used to
226 move the locator along the desired dimension, as shown
227 above. Additionally, the locator can be advanced in both dimensions
228 simultaneously using its ``operator+=`` and ``operator-=``. Similar to
229 image views, locators provide binary ``operator()`` which returns a
230 reference to a pixel with a relative offset to the current locator
231 position. For example, ``src_loc(0,1)`` returns a reference to the
232 neighbor below the current pixel. Locators are very lightweight
233 objects - in the above example the locator has a size of 8 bytes - it
234 consists of a raw pointer to the current pixel and an int indicating
235 the number of bytes from one row to the next (which is the step when
236 moving vertically). The call to ``++src_loc.x()`` corresponds to a
237 single C pointer increment. However, the example above performs more
238 computations than necessary. The code ``src_loc(0,1)`` has to compute
239 the offset of the pixel in two dimensions, which is slow. Notice
240 though that the offset of the two neighbors is the same, regardless of
241 the pixel location. To improve the performance, GIL can cache and
244 void y_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
246 gray8c_view_t::xy_locator src_loc = src.xy_at(0,1);
247 gray8c_view_t::xy_locator::cached_location_t above = src_loc.cache_location(0,-1);
248 gray8c_view_t::xy_locator::cached_location_t below = src_loc.cache_location(0, 1);
250 for (int y = 1; y < src.height() - 1; ++y)
252 gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
254 for (int x = 0; x < src.width(); ++x)
256 (*dst_it) = (src_loc[above] - src_loc[below]) / 2;
260 src_loc+=point<std::ptrdiff_t>(-src.width(), 1);
264 In this example ``src_loc[above]`` corresponds to a fast pointer indexing
265 operation and the code is efficient.
267 Creating a Generic Version of GIL Algorithms
268 --------------------------------------------
270 Let us make our ``x_gradient`` more generic. It should work with any image
271 views, as long as they have the same number of channels. The gradient
272 operation is to be computed for each channel independently.
274 Here is how the new interface looks like:
278 template <typename SrcView, typename DstView>
279 void x_gradient(const SrcView& src, const DstView& dst)
281 gil_function_requires<ImageViewConcept<SrcView> >();
282 gil_function_requires<MutableImageViewConcept<DstView> >();
283 gil_function_requires
285 ColorSpacesCompatibleConcept
287 typename color_space_type<SrcView>::type,
288 typename color_space_type<DstView>::type
292 ... // compute the gradient
295 The new algorithm now takes the types of the input and output image
296 views as template parameters. That allows using both built-in GIL
297 image views, as well as any user-defined image view classes. The
298 first three lines are optional; they use ``boost::concept_check`` to
299 ensure that the two arguments are valid GIL image views, that the
300 second one is mutable and that their color spaces are compatible
301 (i.e. have the same set of channels).
303 GIL does not require using its own built-in constructs. You are free
304 to use your own channels, color spaces, iterators, locators, views and
305 images. However, to work with the rest of GIL they have to satisfy a
306 set of requirements; in other words, they have to \e model the
307 corresponding GIL _concept_. GIL's concepts are defined in the user
310 One of the biggest drawbacks of using templates and generic
311 programming in C++ is that compile errors can be very difficult to
312 comprehend. This is a side-effect of the lack of early type
313 checking - a generic argument may not satisfy the requirements of a
314 function, but the incompatibility may be triggered deep into a nested
315 call, in code unfamiliar and hardly related to the problem. GIL uses
316 ``boost::concept_check`` to mitigate this problem. The above three
317 lines of code check whether the template parameters are valid models
318 of their corresponding concepts. If a model is incorrect, the compile
319 error will be inside ``gil_function_requires``, which is much closer
320 to the problem and easier to track. Furthermore, such checks get
321 compiled out and have zero performance overhead. The disadvantage of
322 using concept checks is the sometimes severe impact they have on
323 compile time. This is why GIL performs concept checks only in debug
324 mode, and only if ``BOOST_GIL_USE_CONCEPT_CHECK`` is defined (off by
327 The body of the generic function is very similar to that of the
328 concrete one. The biggest difference is that we need to loop over the
329 channels of the pixel and compute the gradient for each channel:
333 template <typename SrcView, typename DstView>
334 void x_gradient(const SrcView& src, const DstView& dst)
336 for (int y=0; y < src.height(); ++y)
338 typename SrcView::x_iterator src_it = src.row_begin(y);
339 typename DstView::x_iterator dst_it = dst.row_begin(y);
341 for (int x = 1; x < src.width() - 1; ++x)
342 for (int c = 0; c < num_channels<SrcView>::value; ++c)
343 dst_it[x][c] = (src_it[x-1][c]- src_it[x+1][c]) / 2;
347 Having an explicit loop for each channel could be a performance problem.
348 GIL allows us to abstract out such per-channel operations:
352 template <typename Out>
353 struct halfdiff_cast_channels
355 template <typename T> Out operator()(T const& in1, T const& in2) const
357 return Out((in1 - in2) / 2);
361 template <typename SrcView, typename DstView>
362 void x_gradient(const SrcView& src, const DstView& dst)
364 typedef typename channel_type<DstView>::type dst_channel_t;
366 for (int y=0; y < src.height(); ++y)
368 typename SrcView::x_iterator src_it = src.row_begin(y);
369 typename DstView::x_iterator dst_it = dst.row_begin(y);
371 for (int x=1; x < src.width() - 1; ++x)
373 static_transform(src_it[x-1], src_it[x+1], dst_it[x],
374 halfdiff_cast_channels<dst_channel_t>());
379 The ``static_transform`` is an example of a channel-level GIL algorithm.
380 Other such algorithms are ``static_generate``, ``static_fill`` and
381 ``static_for_each``. They are the channel-level equivalents of STL
382 ``generate``, ``transform``, ``fill`` and ``for_each`` respectively.
383 GIL channel algorithms use static recursion to unroll the loops; they never
384 loop over the channels explicitly.
386 Note that sometimes modern compilers (at least Visual Studio 8) already unroll
387 channel-level loops, such as the one above. However, another advantage of
388 using GIL's channel-level algorithms is that they pair the channels
389 semantically, not based on their order in memory. For example, the above
390 example will properly match an RGB source with a BGR destination.
392 Here is how we can use our generic version with images of different types:
396 // Calling with 16-bit grayscale data
397 void XGradientGray16_Gray32(
398 unsigned short const* src_pixels, ptrdiff_t src_row_bytes,
400 signed int* dst_pixels, ptrdiff_t dst_row_bytes)
402 gray16c_view_t src=interleaved_view(w, h, (gray16_pixel_t const*)src_pixels, src_row_bytes);
403 gray32s_view_t dst=interleaved_view(w, h, (gray32s_pixel_t*)dst_pixels, dst_row_bytes);
407 // Calling with 8-bit RGB data into 16-bit BGR
408 void XGradientRGB8_BGR16(
409 unsigned char const* src_pixels, ptrdiff_t src_row_bytes,
411 signed short* dst_pixels, ptrdiff_t dst_row_bytes)
413 rgb8c_view_t src = interleaved_view(w, h, (rgb8_pixel_t const*)src_pixels, src_row_bytes);
414 rgb16s_view_t dst = interleaved_view(w, h, (rgb16s_pixel_t*)dst_pixels, dst_row_bytes);
415 x_gradient(src, dst);
418 // Either or both the source and the destination could be planar - the gradient code does not change
419 void XGradientPlanarRGB8_RGB32(
420 unsigned short const* src_r, unsigned short const* src_g, unsigned short const* src_b,
421 ptrdiff_t src_row_bytes, int w, int h,
422 signed int* dst_pixels, ptrdiff_t dst_row_bytes)
424 rgb16c_planar_view_t src = planar_rgb_view (w, h, src_r, src_g, src_b, src_row_bytes);
425 rgb32s_view_t dst = interleaved_view(w, h,(rgb32s_pixel_t*)dst_pixels, dst_row_bytes);
429 As these examples illustrate, both the source and the destination can be
430 interleaved or planar, of any channel depth (assuming the destination channel
431 is assignable to the source), and of any compatible color spaces.
433 GIL 2.1 can also natively represent images whose channels are not
434 byte-aligned, such as 6-bit RGB222 image or a 1-bit Gray1 image.
435 GIL algorithms apply to these images natively. See the design guide or sample
436 files for more on using such images.
438 Image View Transformations
439 --------------------------
441 One way to compute the y-gradient is to rotate the image by 90 degrees,
442 compute the x-gradient and rotate the result back.
443 Here is how to do this in GIL:
447 template <typename SrcView, typename DstView>
448 void y_gradient(const SrcView& src, const DstView& dst)
450 x_gradient(rotated90ccw_view(src), rotated90ccw_view(dst));
453 ``rotated90ccw_view`` takes an image view and returns an image view
454 representing 90-degrees counter-clockwise rotation of its input. It is
455 an example of a GIL view transformation function. GIL provides a
456 variety of transformation functions that can perform any axis-aligned
457 rotation, transpose the view, flip it vertically or horizontally,
458 extract a rectangular subimage, perform color conversion, subsample
459 view, etc. The view transformation functions are fast and shallow -
460 they don't copy the pixels, they just change the "coordinate system"
461 of accessing the pixels. ``rotated90cw_view``, for example, returns a
462 view whose horizontal iterators are the vertical iterators of the
463 original view. The above code to compute ``y_gradient`` is slow
464 because of the memory access pattern; using ``rotated90cw_view`` does
465 not make it any slower.
467 Another example: suppose we want to compute the gradient of the N-th
468 channel of a color image. Here is how to do that:
472 template <typename SrcView, typename DstView>
473 void nth_channel_x_gradient(const SrcView& src, int n, const DstView& dst)
475 x_gradient(nth_channel_view(src, n), dst);
478 ``nth_channel_view`` is a view transformation function that takes any
479 view and returns a single-channel (grayscale) view of its N-th
480 channel. For interleaved RGB view, for example, the returned view is
481 a step view - a view whose horizontal iterator skips over two channels
482 when incremented. If applied on a planar RGB view, the returned type
483 is a simple grayscale view whose horizontal iterator is a C pointer.
484 Image view transformation functions can be piped together. For
485 example, to compute the y gradient of the second channel of the even
486 pixels in the view, use:
490 y_gradient(subsampled_view(nth_channel_view(src, 1), 2,2), dst);
492 GIL can sometimes simplify piped views. For example, two nested
493 subsampled views (views that skip over pixels in X and in Y) can be
494 represented as a single subsampled view whose step is the product of
495 the steps of the two views.
500 Let's go back to ``x_gradient`` one more time. Many image view
501 algorithms apply the same operation for each pixel and GIL provides an
502 abstraction to handle them. However, our algorithm has an unusual
503 access pattern, as it skips the first and the last column. It would be
504 nice and instructional to see how we can rewrite it in canonical
505 form. The way to do that in GIL is to write a version that works for
506 every pixel, but apply it only on the subimage that excludes the first
511 void x_gradient_unguarded(gray8c_view_t const& src, gray8s_view_t const& dst)
513 for (int y=0; y < src.height(); ++y)
515 gray8c_view_t::x_iterator src_it = src.row_begin(y);
516 gray8s_view_t::x_iterator dst_it = dst.row_begin(y);
518 for (int x = 0; x < src.width(); ++x)
519 dst_it[x] = (src_it[x-1] - src_it[x+1]) / 2;
523 void x_gradient(gray8c_view_t const& src, gray8s_view_t const& dst)
525 assert(src.width()>=2);
526 x_gradient_unguarded(subimage_view(src, 1, 0, src.width()-2, src.height()),
527 subimage_view(dst, 1, 0, src.width()-2, src.height()));
530 ``subimage_view`` is another example of a GIL view transformation
531 function. It takes a source view and a rectangular region (in this
532 case, defined as x_min,y_min,width,height) and returns a view
533 operating on that region of the source view. The above implementation
534 has no measurable performance degradation from the version that
535 operates on the original views.
537 Now that ``x_gradient_unguarded`` operates on every pixel, we can
538 rewrite it more compactly:
542 void x_gradient_unguarded(gray8c_view_t const& src, gray8s_view_t const& dst)
544 gray8c_view_t::iterator src_it = src.begin();
545 for (gray8s_view_t::iterator dst_it = dst.begin(); dst_it!=dst.end(); ++dst_it, ++src_it)
546 *dst_it = (src_it.x()[-1] - src_it.x()[1]) / 2;
549 GIL image views provide ``begin()`` and ``end()`` methods that return
550 one dimensional pixel iterators which iterate over each pixel in the
551 view, left to right and top to bottom. They do a proper "carriage
552 return" - they skip any unused bytes at the end of a row. As such,
553 they are slightly suboptimal, because they need to keep track of their
554 current position with respect to the end of the row. Their increment
555 operator performs one extra check (are we at the end of the row?), a
556 check that is avoided if two nested loops are used instead. These
557 iterators have a method ``x()`` which returns the more lightweight
558 horizontal iterator that we used previously. Horizontal iterators have
559 no notion of the end of rows. In this case, the horizontal iterators
560 are raw C pointers. In our example, we must use the horizontal
561 iterators to access the two neighbors properly, since they could
562 reside outside the image view.
564 STL Equivalent Algorithms
565 -------------------------
567 GIL provides STL equivalents of many algorithms. For example,
568 ``std::transform`` is an STL algorithm that sets each element in a
569 destination range the result of a generic function taking the
570 corresponding element of the source range. In our example, we want to
571 assign to each destination pixel the value of the half-difference of
572 the horizontal neighbors of the corresponding source pixel. If we
573 abstract that operation in a function object, we can use GIL's
574 ``transform_pixel_positions`` to do that:
578 struct half_x_difference
580 int operator()(const gray8c_loc_t& src_loc) const
582 return (src_loc.x()[-1] - src_loc.x()[1]) / 2;
586 void x_gradient_unguarded(gray8c_view_t const& src, gray8s_view_t const& dst)
588 transform_pixel_positions(src, dst, half_x_difference());
591 GIL provides the algorithms ``for_each_pixel`` and
592 ``transform_pixels`` which are image view equivalents of STL
593 ``std::for_each`` and ``std::transform``. It also provides
594 ``for_each_pixel_position`` and ``transform_pixel_positions``, which
595 instead of references to pixels, pass to the generic function pixel
596 locators. This allows for more powerful functions that can use the
597 pixel neighbors through the passed locators. GIL algorithms iterate
598 through the pixels using the more efficient two nested loops (as
599 opposed to the single loop using 1-D iterators)
604 Instead of computing the gradient of each color plane of an image, we
605 often want to compute the gradient of the luminosity. In other words,
606 we want to convert the color image to grayscale and compute the
607 gradient of the result. Here how to compute the luminosity gradient of
608 a 32-bit float RGB image:
612 void x_gradient_rgb_luminosity(rgb32fc_view_t const& src, gray8s_view_t const& dst)
614 x_gradient(color_converted_view<gray8_pixel_t>(src), dst);
617 ``color_converted_view`` is a GIL view transformation function that
618 takes any image view and returns a view in a target color space and
619 channel depth (specified as template parameters). In our example, it
620 constructs an 8-bit integer grayscale view over 32-bit float RGB
621 pixels. Like all other view transformation functions,
622 ``color_converted_view`` is very fast and shallow. It doesn't copy the
623 data or perform any color conversion. Instead it returns a view that
624 performs color conversion every time its pixels are accessed.
626 In the generic version of this algorithm we might like to convert the
627 color space to grayscale, but keep the channel depth the same. We do
628 that by constructing the type of a GIL grayscale pixel with the same
629 channel as the source, and color convert to that pixel type:
633 template <typename SrcView, typename DstView>
634 void x_luminosity_gradient(SrcView const& src, DstView const& dst)
636 using gray_pixel_t = pixel<typename channel_type<SrcView>::type, gray_layout_t>;
637 x_gradient(color_converted_view<gray_pixel_t>(src), dst);
640 When the destination color space and channel type happens to be the
641 same as the source one, color conversion is unnecessary. GIL detects
642 this case and avoids calling the color conversion code at all -
643 i.e. ``color_converted_view`` returns back the source view unchanged.
648 The above example has a performance problem - ``x_gradient``
649 dereferences most source pixels twice, which will cause the above code
650 to perform color conversion twice. Sometimes it may be more efficient
651 to copy the color converted image into a temporary buffer and use it
652 to compute the gradient - that way color conversion is invoked once
653 per pixel. Using our non-generic version we can do it like this:
657 void x_luminosity_gradient(rgb32fc_view_t const& src, gray8s_view_t const& dst)
659 gray8_image_t ccv_image(src.dimensions());
660 copy_pixels(color_converted_view<gray8_pixel_t>(src), view(ccv_image));
662 x_gradient(const_view(ccv_image), dst);
665 First we construct an 8-bit grayscale image with the same dimensions
666 as our source. Then we copy a color-converted view of the source into
667 the temporary image. Finally we use a read-only view of the temporary
668 image in our ``x_gradient algorithm``. As the example shows, GIL
669 provides global functions ``view`` and ``const_view`` that take an
670 image and return a mutable or an immutable view of its pixels.
672 Creating a generic version of the above is a bit trickier:
676 template <typename SrcView, typename DstView>
677 void x_luminosity_gradient(const SrcView& src, const DstView& dst)
679 using d_channel_t = typename channel_type<DstView>::type;
680 using channel_t = typename channel_convert_to_unsigned<d_channel_t>::type;
681 using gray_pixel_t = pixel<channel_t, gray_layout_t>;
682 using gray_image_t = image<gray_pixel_t, false>;
684 gray_image_t ccv_image(src.dimensions());
685 copy_pixels(color_converted_view<gray_pixel_t>(src), view(ccv_image));
686 x_gradient(const_view(ccv_image), dst);
689 First we use the ``channel_type`` metafunction to get the channel type
690 of the destination view. A metafunction is a function operating on
691 types. In GIL metafunctions are class templates (declared with
692 ``struct`` type specifier) which take their parameters as template
693 parameters and return their result in a nested typedef called
694 ``type``. In this case, ``channel_type`` is a unary metafunction which
695 in this example is called with the type of an image view and returns
696 the type of the channel associated with that image view.
698 GIL constructs that have an associated pixel type, such as pixels,
699 pixel iterators, locators, views and images, all model
700 ``PixelBasedConcept``, which means that they provide a set of
701 metafunctions to query the pixel properties, such as ``channel_type``,
702 ``color_space_type``, ``channel_mapping_type``, and ``num_channels``.
704 After we get the channel type of the destination view, we use another
705 metafunction to remove its sign (if it is a signed integral type) and
706 then use it to generate the type of a grayscale pixel. From the pixel
707 type we create the image type. GIL's image class is specialized over
708 the pixel type and a boolean indicating whether the image should be
709 planar or interleaved. Single-channel (grayscale) images in GIL must
710 always be interleaved. There are multiple ways of constructing types
711 in GIL. Instead of instantiating the classes directly we could have
712 used type factory metafunctions. The following code is equivalent:
716 template <typename SrcView, typename DstView>
717 void x_luminosity_gradient(SrcView const& src, DstView const& dst)
719 typedef typename channel_type<DstView>::type d_channel_t;
720 typedef typename channel_convert_to_unsigned<d_channel_t>::type channel_t;
721 typedef typename image_type<channel_t, gray_layout_t>::type gray_image_t;
722 typedef typename gray_image_t::value_type gray_pixel_t;
724 gray_image_t ccv_image(src.dimensions());
725 copy_and_convert_pixels(src, view(ccv_image));
726 x_gradient(const_view(ccv_image), dst);
729 GIL provides a set of metafunctions that generate GIL types -
730 ``image_type`` is one such meta-function that constructs the type of
731 an image from a given channel type, color layout, and
732 planar/interleaved option (the default is interleaved). There are also
733 similar meta-functions to construct the types of pixel references,
734 iterators, locators and image views. GIL also has metafunctions
735 ``derived_pixel_reference_type``, ``derived_iterator_type``,
736 ``derived_view_type`` and ``derived_image_type`` that construct the
737 type of a GIL construct from a given source one by changing one or
738 more properties of the type and keeping the rest.
740 From the image type we can use the nested typedef ``value_type`` to
741 obtain the type of a pixel. GIL images, image views and locators have
742 nested typedefs ``value_type`` and ``reference`` to obtain the type of
743 the pixel and a reference to the pixel. If you have a pixel iterator,
744 you can get these types from its ``iterator_traits``. Note also the
745 algorithm ``copy_and_convert_pixels``, which is an abbreviated version
746 of ``copy_pixels`` with a color converted source view.
751 So far we have been dealing with images that have pixels stored in
752 memory. GIL allows you to create an image view of an arbitrary image,
753 including a synthetic function. To demonstrate this, let us create a
754 view of the Mandelbrot set. First, we need to create a function
755 object that computes the value of the Mandelbrot set at a given
756 location (x,y) in the image:
760 // models PixelDereferenceAdaptorConcept
763 typedef point<ptrdiff_t> point_t;
765 typedef mandelbrot_fn const_t;
766 typedef gray8_pixel_t value_type;
767 typedef value_type reference;
768 typedef value_type const_reference;
769 typedef point_t argument_type;
770 typedef reference result_type;
771 static bool constexpr is_mutable = false;
774 mandelbrot_fn(const point_t& sz) : _img_size(sz) {}
776 result_type operator()(const point_t& p) const
778 // normalize the coords to (-2..1, -1.5..1.5)
779 double t=get_num_iter(point<double>(p.x/(double)_img_size.x*3-2, p.y/(double)_img_size.y*3-1.5f));
780 return value_type((bits8)(pow(t,0.2)*255)); // raise to power suitable for viewing
785 double get_num_iter(const point<double>& p) const
787 point<double> Z(0,0);
788 for (int i=0; i<100; ++i) // 100 iterations
790 Z = point<double>(Z.x*Z.x - Z.y*Z.y + p.x, 2*Z.x*Z.y + p.y);
791 if (Z.x*Z.x + Z.y*Z.y > 4)
792 return i/(double)100;
798 We can now use GIL's ``virtual_2d_locator`` with this function object
799 to construct a Mandelbrot view of size 200x200 pixels:
803 typedef mandelbrot_fn::point_t point_t;
804 typedef virtual_2d_locator<mandelbrot_fn,false> locator_t;
805 typedef image_view<locator_t> my_virt_view_t;
807 point_t dims(200,200);
809 // Construct a Mandelbrot view with a locator, taking top-left corner (0,0) and step (1,1)
810 my_virt_view_t mandel(dims, locator_t(point_t(0,0), point_t(1,1), mandelbrot_fn(dims)));
812 We can treat the synthetic view just like a real one. For example,
813 let's invoke our ``x_gradient`` algorithm to compute the gradient of
814 the 90-degree rotated view of the Mandelbrot set and save the original
819 gray8s_image_t img(dims);
820 x_gradient(rotated90cw_view(mandel), view(img));
822 // Save the Mandelbrot set and its 90-degree rotated gradient (jpeg cannot save signed char; must convert to unsigned char)
823 jpeg_write_view("mandel.jpg",mandel);
824 jpeg_write_view("mandel_grad.jpg",color_converted_view<gray8_pixel_t>(const_view(img)));
826 Here is what the two files look like:
828 .. image:: ../images/mandel.jpg
830 Run-Time Specified Images and Image Views
831 -----------------------------------------
833 So far we have created a generic function that computes the image
834 gradient of an image view template specialization. Sometimes,
835 however, the properties of an image view, such as its color space and
836 channel depth, may not be available at compile time. GIL's
837 ``dynamic_image`` extension allows for working with GIL constructs
838 that are specified at run time, also called _variants_. GIL provides
839 models of a run-time instantiated image, ``any_image``, and a run-time
840 instantiated image view, ``any_image_view``. The mechanisms are in
841 place to create other variants, such as ``any_pixel``,
842 ``any_pixel_iterator``, etc. Most of GIL's algorithms and all of the
843 view transformation functions also work with run-time instantiated
844 image views and binary algorithms, such as ``copy_pixels`` can have
845 either or both arguments be variants.
847 Lets make our ``x_luminosity_gradient`` algorithm take a variant image
848 view. For simplicity, let's assume that only the source view can be a
849 variant. (As an example of using multiple variants, see GIL's image
850 view algorithm overloads taking multiple variants.)
852 First, we need to make a function object that contains the templated
853 destination view and has an application operator taking a templated
858 #include <boost/gil/extension/dynamic_image/dynamic_image_all.hpp>
860 template <typename DstView>
861 struct x_gradient_obj
863 typedef void result_type; // required typedef
866 x_gradient_obj(const DstView& dst) : _dst(dst) {}
868 template <typename SrcView>
869 void operator()(const SrcView& src) const { x_luminosity_gradient(src, _dst); }
872 The second step is to provide an overload of ``x_luminosity_gradient`` that
873 takes image view variant and calls GIL's ``apply_operation`` passing it the
878 template <typename SrcViews, typename DstView>
879 void x_luminosity_gradient(const any_image_view<SrcViews>& src, const DstView& dst)
881 apply_operation(src, x_gradient_obj<DstView>(dst));
884 ``any_image_view<SrcViews>`` is the image view variant. It is
885 templated over ``SrcViews``, an enumeration of all possible view types
886 the variant can take. ``src`` contains inside an index of the
887 currently instantiated type, as well as a block of memory containing
888 the instance. ``apply_operation`` goes through a switch statement
889 over the index, each case of which casts the memory to the correct
890 view type and invokes the function object with it. Invoking an
891 algorithm on a variant has the overhead of one switch
892 statement. Algorithms that perform an operation for each pixel in an
893 image view have practically no performance degradation when used with
896 Here is how we can construct a variant and invoke the algorithm:
900 #include <boost/mpl/vector.hpp>
901 #include <boost/gil/extension/io/jpeg_dynamic_io.hpp>
903 typedef mpl::vector<gray8_image_t, gray16_image_t, rgb8_image_t, rgb16_image_t> my_img_types;
904 any_image<my_img_types> runtime_image;
905 jpeg_read_image("input.jpg", runtime_image);
907 gray8s_image_t gradient(runtime_image.dimensions());
908 x_luminosity_gradient(const_view(runtime_image), view(gradient));
909 jpeg_write_view("x_gradient.jpg", color_converted_view<gray8_pixel_t>(const_view(gradient)));
911 In this example, we create an image variant that could be 8-bit or
912 16-bit RGB or grayscale image. We then use GIL's I/O extension to load
913 the image from file in its native color space and channel depth. If
914 none of the allowed image types matches the image on disk, an
915 exception will be thrown. We then construct a 8 bit signed
916 (i.e. ``char``) image to store the gradient and invoke ``x_gradient``
917 on it. Finally we save the result into another file. We save the view
918 converted to 8-bit unsigned, because JPEG I/O does not support signed
921 Note how free functions and methods such as ``jpeg_read_image``,
922 ``dimensions``, ``view`` and ``const_view`` work on both templated and
923 variant types. For templated images ``view(img)`` returns a templated
924 view, whereas for image variants it returns a view variant. For
925 example, the return type of ``view(runtime_image)`` is
926 ``any_image_view<Views>`` where ``Views`` enumerates four views
927 corresponding to the four image types. ``const_view(runtime_image)``
928 returns a ``any_image_view`` of the four read-only view types, etc.
930 A warning about using variants: instantiating an algorithm with a
931 variant effectively instantiates it with every possible type the
932 variant can take. For binary algorithms, the algorithm is
933 instantiated with every possible combination of the two input types!
934 This can take a toll on both the compile time and the executable size.
939 This tutorial provides a glimpse at the challenges associated with
940 writing generic and efficient image processing algorithms in GIL. We
941 have taken a simple algorithm and shown how to make it work with image
942 representations that vary in bit depth, color space, ordering of the
943 channels, and planar/interleaved structure. We have demonstrated that
944 the algorithm can work with fully abstracted virtual images, and even
945 images whose type is specified at run time. The associated video
946 presentation also demonstrates that even for complex scenarios the
947 generated assembly is comparable to that of a C version of the
948 algorithm, hand-written for the specific image types.
950 Yet, even for such a simple algorithm, we are far from making a fully
951 generic and optimized code. In particular, the presented algorithms
952 work on homogeneous images, i.e. images whose pixels have channels
953 that are all of the same type. There are examples of images, such as a
954 packed 565 RGB format, which contain channels of different
955 types. While GIL provides concepts and algorithms operating on
956 heterogeneous pixels, we leave the task of extending x_gradient as an
957 exercise for the reader. Second, after computing the value of the
958 gradient we are simply casting it to the destination channel
959 type. This may not always be the desired operation. For example, if
960 the source channel is a float with range [0..1] and the destination is
961 unsigned char, casting the half-difference to unsigned char will
962 result in either 0 or 1. Instead, what we might want to do is scale
963 the result into the range of the destination channel. GIL's
964 channel-level algorithms might be useful in such cases. For example,
965 \p channel_convert converts between channels by linearly scaling the
966 source channel value into the range of the destination channel.
968 There is a lot to be done in improving the performance as
969 well. Channel-level operations, such as the half-difference, could be
970 abstracted out into atomic channel-level algorithms and performance
971 overloads could be provided for concrete channel
972 types. Processor-specific operations could be used, for example, to
973 perform the operation over an entire row of pixels simultaneously, or
974 the data could be pre-fetched. All of these optimizations can be
975 realized as performance specializations of the generic
976 algorithm. Finally, compilers, while getting better over time, are
977 still failing to fully optimize generic code in some cases, such as
978 failing to inline some functions or put some variables into
979 registers. If performance is an issue, it might be worth trying your
980 code with different compilers.