Color histogram

In image processing and photography, a color histogram is a representation of the distribution of colors in an image. For digital images, a color histogram represents the number of pixels that have colors in each of a fixed list of color ranges, that span the image's color space, the set of all possible colors.

The color histogram can be built for any kind of color space, although the term is more often used for three-dimensional spaces such as RGB or HSV. For monochromatic images, the term intensity histogram may be used instead. For multi-spectral images, where each pixel is represented by an arbitrary number of measurements (for example, beyond the three measurements in RGB), the color histogram is N-dimensional, with N being the number of measurements taken. Each measurement has its own wavelength range of the light spectrum, some of which may be outside the visible spectrum.

If the set of possible color values is sufficiently small, each of those colors may be placed on a range by itself; then the histogram is merely the count of pixels that have each possible color. Most often, the space is divided into an appropriate number of ranges, often arranged as a regular grid, each containing many similar color values. The color histogram may also be represented and displayed as a smooth function defined over the color space that approximates the pixel counts.

Like other kinds of histograms, the color histogram is a statistic that can be viewed as an approximation of an underlying continuous distribution of color values.

Overview

Color histograms are flexible constructs that can be built from images in various color spaces, whether RGB, rg chromaticity or any other color space of any dimension. A histogram of an image is produced first by discretization of the colors in the image into a number of bins, and counting the number of image pixels in each bin. For example, a Red–Blue chromaticity histogram can be formed by first normalizing color pixel values by dividing RGB values by R G B, then quantizing the normalized R and B coordinates into N bins each. A two-dimensional histogram of Red-Blue chromaticity divided into four bins (N=4) might yield a histogram that looks like this table:

		red
		0-63	64-127	128-191	192-255
blue	0-63	43	78	18	0
	64-127	45	67	33	2
	128-191	127	58	25	8
	192-255	140	47	47	13

A histogram can be N-dimensional. Although harder to display, a three-dimensional color histogram for the above example could be thought of as four separate Red-Blue histograms, where each of the four histograms contains the Red-Blue values for a bin of green (0-63, 64-127, 128-191, and 192-255).

The histogram provides a compact summarization of the distribution of data in an image. The color histogram of an image is relatively invariant with translation and rotation about the viewing axis, and varies only slowly with the angle of view.^[1] By comparing histogram signatures of two images and matching the color content of one image with the other, the color histogram is particularly well suited for the problem of recognizing an object of unknown position and rotation within a scene. Importantly, translation of an RGB image into the illumination invariant rg-chromaticity space allows the histogram to operate well in varying light levels.

1. What is a histogram?

A histogram is a graphical representation of the number of pixels in an image. In a more simple way to explain, a histogram is a bar graph, whose X-axis represents the tonal scale(black at the left and white at the right), and Y-axis represents the number of pixels in an image in a certain area of the tonal scale. For example, the graph of a luminance histogram shows the number of pixels for each brightness level(from black to white), and when there are more pixels, the peak at the certain luminance level is higher.

2. What is a color histogram?

A color histogram of an image represents the distribution of the composition of colors in the image. It shows different types of colors appeared and the number of pixels in each type of the colors appeared. The relation between a color histogram and a luminance histogram is that a color histogram can be also expressed as “Three Luminance Histograms”, each of which shows the brightness distribution of each individual Red/Green/Blue color channel.

Characteristics of a color histogram

A color histogram focuses only on the proportion of the number of different types of colors, regardless of the spatial location of the colors. The values of a color histogram are from statistics. They show the statistical distribution of colors and the essential tone of an image.

In general, as the color distributions of the foreground and background in an image are different, there might be a bimodal distribution in the histogram.

For the luminance histogram alone, there is no perfect histogram and in general, the histogram can tell whether it is over-exposure or not, but there are times when you might think the image is over exposed by viewing the histogram; however, in reality it is not.

Principles of the formation of a color histogram

The formation of a color histogram is rather simple. From the definition above, we can simply count the number of pixels for each 256 scales in each of the 3 RGB channel, and plot them on 3 individual bar graphs.

In general, a color histogram is based on a certain color space, such as RGB or HSV. When we compute the pixels of different colors in an image, if the color space is large, then we can first divide the color space into certain numbers of small intervals. Each of the intervals is called a bin. This process is called color quantization. Then, by counting the number of pixels in each of the bins, we get the color histogram of the image.

The concrete steps of the principles can be viewed in Example 1.

Examples

Example 1

Given the following image of a cat (an original version and a version that has been reduced to 256 colors for easy histogram purposes), the following data represents a color histogram in the RGB color space, using four bins.

Bin 0 corresponds to intensities 0-63

Bin 1 is 64-127

Bin 2 is 128-191 and Bin 3 is 192-255.

Red	Green	Blue	Pixel Count
Bin 0	Bin 0	Bin 0	7414
Bin 0	Bin 0	Bin 1	230
Bin 0	Bin 0	Bin 2	0
Bin 0	Bin 0	Bin 3	0
Bin 0	Bin 1	Bin 0	8
Bin 0	Bin 1	Bin 1	372
Bin 0	Bin 1	Bin 2	88
Bin 0	Bin 1	Bin 3	0
Bin 0	Bin 2	Bin 0	0
Bin 0	Bin 2	Bin 1	0
Bin 0	Bin 2	Bin 2	10
Bin 0	Bin 2	Bin 3	1
Bin 0	Bin 3	Bin 0	0
Bin 0	Bin 3	Bin 1	0
Bin 0	Bin 3	Bin 2	0
Bin 0	Bin 3	Bin 3	0
Bin 1	Bin 0	Bin 0	891
Bin 1	Bin 0	Bin 1	13
Bin 1	Bin 0	Bin 2	0
Bin 1	Bin 0	Bin 3	0
Bin 1	Bin 1	Bin 0	592
Bin 1	Bin 1	Bin 1	3462
Bin 1	Bin 1	Bin 2	355
Bin 1	Bin 1	Bin 3	0
Bin 1	Bin 2	Bin 0	0
Bin 1	Bin 2	Bin 1	101
Bin 1	Bin 2	Bin 2	882
Bin 1	Bin 2	Bin 3	16
Bin 1	Bin 3	Bin 0	0
Bin 1	Bin 3	Bin 1	0
Bin 1	Bin 3	Bin 2	0
Bin 1	Bin 3	Bin 3	0
Bin 2	Bin 0	Bin 0	1146
Bin 2	Bin 0	Bin 1	0
Bin 2	Bin 0	Bin 2	0
Bin 2	Bin 0	Bin 3	0
Bin 2	Bin 1	Bin 0	2552
Bin 2	Bin 1	Bin 1	9040
Bin 2	Bin 1	Bin 2	47
Bin 2	Bin 1	Bin 3	0
Bin 2	Bin 2	Bin 0	0
Bin 2	Bin 2	Bin 1	8808
Bin 2	Bin 2	Bin 2	53110
Bin 2	Bin 2	Bin 3	11053
Bin 2	Bin 3	Bin 0	0
Bin 2	Bin 3	Bin 1	0
Bin 2	Bin 3	Bin 2	170
Bin 2	Bin 3	Bin 3	17533
Bin 3	Bin 0	Bin 0	11
Bin 3	Bin 0	Bin 1	0
Bin 3	Bin 0	Bin 2	0
Bin 3	Bin 0	Bin 3	0
Bin 3	Bin 1	Bin 0	856
Bin 3	Bin 1	Bin 1	1376
Bin 3	Bin 1	Bin 2	0
Bin 3	Bin 1	Bin 3	0
Bin 3	Bin 2	Bin 0	0
Bin 3	Bin 2	Bin 1	3650
Bin 3	Bin 2	Bin 2	6260
Bin 3	Bin 2	Bin 3	109
Bin 3	Bin 3	Bin 0	0
Bin 3	Bin 3	Bin 1	0
Bin 3	Bin 3	Bin 2	3415
Bin 3	Bin 3	Bin 3	53929

Example 2

Application in camera:

Nowadays, some cameras have the ability to show the 3 color histograms when we take photos.

We can examine clips (spikes on either the black or white side of the scale) in each of the 3 RGB color histograms. If we find one or more clipping on a channel of the 3 RGB channels, then this would result in a loss of detail for that color.

To illustrate this, consider this example:

1. We know that each of the three R, G, B channels has a range of values from 0-255 (8 bit). So consider a photo that has a luminance range of 0-255.

2. Assume the photo we take is made of 4 blocks that are adjacent to each other and we set the luminance scale for each of the 4 blocks of original photo to be 10, 100, 205, 245. Thus, the image looks like the topmost figure on the right.

3. Then, we overexpose the photo a little, say, the luminance scale of each block is increased by 10. Thus, the luminance scale for each of the 4 blocks of new photo is 20, 110, 215, 255. Then, the image looks like the second figure on the right.

There is not much difference between both figures, all we can see is that the whole image becomes brighter (the contrast for each of the blocks remain the same).

4. Now, we overexpose the original photo again, this time the luminance scale of each block is increased by 50. Thus, the luminance scale for each of the 4 blocks of the new photo is 60, 150, 255, 255. The new image now looks like the third figure on the right.

Note that the scale for the last block is 255 instead of 295, for 255 is the top scale and thus the last block has clipped! When this happens, we lose the contrast of the last 2 blocks, and thus, we cannot recover the image no matter how we adjust it.

To conclude, when taking photos with a camera that displays histograms, always keep the brightest tone in the image below the largest scale 255 on the histogram in order to avoid losing details.

Drawbacks and other approaches

The main drawback of histograms for classification is that the representation is dependent on the color of the object being studied, ignoring its shape and texture. Color histograms can potentially be identical for two images with different object content which happens to share color information. Conversely, without spatial or shape information, similar objects of different color may be indistinguishable based solely on color histogram comparisons. There is no way to distinguish a red and white cup from a red and white plate. Put it another way: histogram-based algorithms have no concept of a generic 'cup', and a model of a red and white cup is no use when given an otherwise identical blue and white cup. Another problem is that color histograms have high sensitivity to noisy interference such as lighting intensity changes and quantization errors. High dimensionality (bins) color histograms are also another issue. Some color histogram feature spaces often occupy more than one hundred dimensions.^[2]

Some of the proposed solutions have been color histogram intersection, color constant indexing, cumulative color histogram, quadratic distance, and color correlograms. Although there are drawbacks of using histograms for indexing and classification, using color in a real-time system has several advantages. One is that color information is faster to compute compared to other invariants. It has been shown in some cases that color can be an efficient method for identifying objects of known location and appearance.

Further research into the relationship between color histogram data to the physical properties of the objects in an image has shown they can represent not only object color and illumination but relate to surface roughness and image geometry and provide an improved estimate of illumination and object color.^[3]

Usually, Euclidean distance, histogram intersection, or cosine or quadratic distances are used for the calculation of image similarity ratings.^[4] Any of these values do not reflect the similarity rate of two images in itself; it is useful only when used in comparison to other similar values. This is the reason that all the practical implementations of content-based image retrieval must complete computation of all images from the database, and is the main disadvantage of these implementations.

Another approach to representative color image content is two-dimensional color histogram. A two-dimensional color histogram considers the relation between the pixel pair colors (not only the lighting component).^[5] A two-dimensional color histogram is a two-dimensional array. The size of each dimension is the number of colors that were used in the phase of color quantization. These arrays are treated as matrices, each element of which stores a normalized count of pixel pairs, with each color corresponding to the index of an element in each pixel neighborhood. For comparison of two-dimensional color histograms it is suggested calculating their correlation, because constructed as described above, is a random vector (in other words, a multi-dimensional random value). While creating a set of final images, the images should be arranged in decreasing order of the correlation coefficient.

The correlation coefficient may also be used for color histogram comparison. Retrieval results with correlation coefficient are better than with other metrics.^[6]

Intensity histogram of continuous data

The idea of an intensity histogram can be generalized to continuous data, say audio signals represented by real functions or images represented by functions with two-dimensional domain.

Let $f\in L^{1}(\mathbb {R} ^{n})$ (see Lebesgue space), then the cumulative histogram operator $H$ can be defined by:

H(f)(y)=\mu \{x:f(x)\leq y\}

.

$\mu$ is the Lebesgue measure of sets. $H(f)$ in turn is a real function. The (non-cumulative) histogram is defined as its derivative.

h(f)=H(f)'

.

References

^ Shapiro, Linda G. and Stockman, George C. "Computer Vision" Prentice Hall, 2003 ISBN 0-13-030796-3
^ Xiang-Yang Wang, Jun-Feng Wu, and Hong-Ying Yang "Robust image retrieval based on color histogram of local feature regions" Springer Netherlands, 2009 ISSN 1573-7721
^ Anatomy of a color histogram; Novak, C.L.; Shafer, S.A.; Computer Vision and Pattern Recognition, 1992. Proceedings CVPR '92., 1992 IEEE Computer Society Conference on 15–18 June 1992 Page(s):599 - 605 doi:10.1109/CVPR.1992.223129
^ Integrated Spatial and Feature Image Systems: Retrieval, Analysis and Compression; Smith, J.R.; Graduate School of Arts and Sciences, Columbia University, 1997
^ Effectiveness estimation of image retrieval by 2D color histogram; Bashkov, E.A.; Kostyukova, N.S.; Journal of Automation and Information Sciences, 2006 (6) Page(s): 84-89
^ Content-Based Image Retrieval Using Color Histogram Correlation; Bashkov, E.A.; Shozda, N.S.; Graphicon proceedings, 2002 Page(s): [1] Archived 2012-07-07 at the Wayback Machine

External links

3D Color Inspector/Color Histogram, by Kai Uwe Barthel. (Free Java applet.)
Stanford Student Project on Image Based Retrieval - more in depth look at equations/application
MATLAB/Octave code for plotting Color Histograms and Color Clouds - The source code can be ported to other languages

[1] Shapiro, Linda G. and Stockman, George C. "Computer Vision" Prentice Hall, 2003 ISBN 0-13-030796-3

[2] Xiang-Yang Wang, Jun-Feng Wu, and Hong-Ying Yang "Robust image retrieval based on color histogram of local feature regions" Springer Netherlands, 2009 ISSN 1573-7721

[3] Anatomy of a color histogram; Novak, C.L.; Shafer, S.A.; Computer Vision and Pattern Recognition, 1992. Proceedings CVPR '92., 1992 IEEE Computer Society Conference on 15–18 June 1992 Page(s):599 - 605 doi:10.1109/CVPR.1992.223129

[4] Integrated Spatial and Feature Image Systems: Retrieval, Analysis and Compression; Smith, J.R.; Graduate School of Arts and Sciences, Columbia University, 1997

[5] Effectiveness estimation of image retrieval by 2D color histogram; Bashkov, E.A.; Kostyukova, N.S.; Journal of Automation and Information Sciences, 2006 (6) Page(s): 84-89

[6] Content-Based Image Retrieval Using Color Histogram Correlation; Bashkov, E.A.; Shozda, N.S.; Graphicon proceedings, 2002 Page(s): [1] Archived 2012-07-07 at the Wayback Machine

[1]

[2]

[3]

[4]

[5]

[6]