Each pixel value in a greyscale image represents what is loosely called brightness. However, brightness is defined formally as the attribute of avisual sensation according to which an area appears to emit more or less light. This definition is obviously subjective: Brightness can’t be measured, so is an inappropriate metric for image data.
Also, according to colour appearance theory, brightness has no top end: Brightness is not relative to anything.
See Appendix B, Introduction to radiometry and photometry, on page 601. Sound intensity is conceptually very different from light intensity.
Intensity is radiant power in a particular direction, that is, power per unit solid angle [W·sr-2]; radiance is intensity per unit projected area. These terms disregard wavelength composition, but in colour imaging, wave-length is important! Neither of these quantities is a suitable metric for colour image data.
Luminance is radiance weighted by the spectral sensi-tivity associated with the lightness sensation of vision.
Luminance is proportional to intensity; in the SI system, it carries units of candelas per meter squared [cd·m-2], commonly called nits [nt]. Imaging systems rarely use pixel values proportional to luminance; values nonlin-early related to luminance are usually used.
Illuminance describes light falling on an object; tech-nically, it is luminance integrated over a half-sphere.
Lightness is defined by the CIE as the brightness of an area judged relative to the brightness of asimilarly illu-minated area that appears to be white or highly transmit-ting. A purist may claim this definition to be subjective;
however, an objective quantity L* is defined as the stan-dard estimate of the perceptual response to relative luminance. It is computed by subjecting relative lumi-nance to a nonlinear transfer function that mimics the
Old texts use “brightness,” symbol-ized B, where today we would use (absolute) luminance, symbolized L.
In image reproduction, we are usually concerned not with (abso-lute) luminance, but with relative luminance, symbolized Y, as I will describe on page 206. The term luminance is often carelessly and incorrectly used by video engineers to refer to luma, as I will describe.
response of vision at a certain state of adaptation.
A few greyscale imaging systems have pixel values proportional to L*.
Value refers to measures of lightness roughly equiva-lent to CIE L* (but typically scaled 0 to 10, not 100). In image science, value is rarely– if ever – used in any sense consistent with accurate colour. (Several different value scales are graphed in Figure 20.2 on page 208.)
Regrettably, many practitioners of digital image processing and computer graphics have a cavalier atti-tude toward the terms described here. In the HSB, HSI, HSL, and HSV systems, B allegedly stands for brightness, I for intensity, L for lightness, and V for value; however, none of these systems is associated with brightness, intensity, luminance, or value according to any defini-tion that is recognized in colour science!
Colour images are sensed and reproduced based upon tristimulus values (tristimuli), whose amplitudes are proportional to intensity but whose spectral compo-sitions are carefully chosen according to the principles of colour science. Relative luminance can be regarded as a distinguished tristimulus value useful on its own;
apart from that, tristimulus values come in sets of three.
The image sensor of a digital camera produces values, proportional to radiance, that approximate red, green, and blue (RGB) tristimulus values. I call these values linear-light. However, in most imaging systems, RGB tristimulus values are coded using a nonlinear transfer function– gamma correction– that mimics the perceptual response of human vision. Most image coding systems use R’G’B’ values that are not propor-tional to intensity; the primes in the notation denote imposition of a perceptually motivated nonlinearity.
Luma is comparable to lightness; it is often carelessly and incorrectly called luminance by video engineers.
See Appendix A, YUV and luminance considered harmful, on page 567.
Luma (Y’) is formed as a suitably weighted sum of R’G’B’; it is the basis of luma/colour difference coding in video, MPEG, JPEG, and similar image coding systems.
In video, the nonlinearity implicit in gamma correction that forms R’G’B’ components is subsequently incorpo-rated into the luma and chroma (Y’CBCR) components.
Contrast
The term contrast, as used in imaging, is heavily over-loaded. “Contrast” can relate to physical properties of light in a scene, physical properties of light in an image,
properties of an imaging system’s mapping between the two, physical properties of a display system (indepen-dent of any image), or various attributes of perception.
The histogram computed and displayed by a D-SLR camera typi-cally has luma as the independent axis, not luminance or log luminance.
Photographers speak of scenes having “high contrast” or “low contrast.” A scene’s contrast can be characterized by the coefficient of variation of its nance – that is, by the standard deviation of its lumi-nance divided by its mean lumilumi-nance. Scene contrast is closely related to the width of the scene’s luminance histogram with respect to its “centre of gravity.” A dark scene and a light scene may have identical scene contrast. Scene contrast is unaffected by noise.
The term key used by photographers is unrelated to key used in video to refer to matting.
Photographers also speak of “high key” and “low key” scenes. Loosely speaking, these are predominantly light or dark (respectively). From a cumulative histo-gram of log luminance, key can be quantified as the fraction of the interval along the x-axis between the 10th and 90th percentile where the 50th percentile falls.
Contrast ratio
A basic property of a display is its contrast ratio, the ratio between specified light and dark luminances – typically the luminance associated with reference (or peak) white and the luminance associated with refer-ence black. Inter-image (or on-off, or sequential) contrast ratio is measured between fully-white and fully-black images presented individually. Intra-image contrast ratio is measured from white and black regions of a single test image such as that specified by ANSI IT7.228-1997 (now withdrawn, for projectors) or ITU-R BT.815. Visual performance of a display system is best characterized by intra-image contrast ratio.
BT.815 specifies a test image, having 16:9 aspect ratio, useful for measuring contrast ratio ;see Figure 3.1.
Contrast ratio is the luminance of the white square divided by the average luminance of the black squares.
In practical imaging systems, many factors conspire to increase the luminance of black, thereby lessening contrast ratio and impairing picture quality. On an elec-tronic display or in a projected image, intra-image contrast ratio is typically less than 800:1 owing to flare in the display system and/or spill light (stray light) in the ambient environment. Typical contrast ratios are shown in Table 3.1. Contrast ratio is a major
determi-Simultaneous contrast ratio is some-times shortened to simultaneous contrast, which unfortunately has a second (unrelated) meaning. See Surround effect, on page 116.
Owing to this ambiguity, I avoid the term simultaneous contrast ratio and use the term intra-image contrast ratio instead. Contrast ratio without qualification should be taken as intra-image.
nant of subjective image quality, so much so that an image reproduced with a high simultaneous contrast ratio may be judged sharper than another image that has higher measured spatial frequency content.
Because contrast ratio involves measurement of just two quantities – reference white and reference black – it is independent of transfer function. Contrast ratio (and its first cousin, dynamic range) say nothing about bit depth: A bilevel image contains both reference white and reference black, and suffices to produce the optical stimulus necessary to measure contrast ratio.
Perceptual uniformity
I introduced perceptual uniformity on page 8. Coding of a perceptual quantity is perceptually uniform if a small perturbation to the coded value is approximately equally perceptible across the range of that value.
Vision cannot distinguish two luminance levels if the ratio between them is less than about 1.01– in other words, the visual threshold for luminance difference is about 1%. This contrast sensitivity threshold is estab-lished by experiments using the test pattern such as the Figure 3.1 The ITU-R BT.815
pattern is used to characterize intra-image contrast ratio. The grey background is at 8-bit inter-face code 126. A small square of reference white lies at the centre of the image; it occupies 2/15 of the image height. Four identical small squares of reference black are disposed around the centre, offset by 1/3 of the image height, at 12 o’clock, 3 o’clock,
6 o’clock, and 9 o’clock.
Application
Max. (Ref.) luminance [cd·m– 2]
Typ. PDR luminance [cd·m– 2]
Typical inter-image contrast ratio
Typical BT.815 intra-image
contrast ratio Minimum L*
Cinema (projection) 480 (48) 24 1 000:1 100:1 9
HD, studio mastering 120 (100) 90 10 000:1 1000:1 0.9
HD, living room (typ.) 200) 200 1 000:1 400:1 2.3
Office (sRGB, typ.) 320) 320 100:1 100:1 16
Table 3.1 Typical contrast ratios are summarized. PDR refers to perfect diffuse reflector.
one sketched in Figure 3.2 in the margin; details will be presented in Contrast sensitivity, on page 249. Ideally, pixel values are placed at these just noticeable differ-ence (JND) increments along the scale from referdiffer-ence black to reference white.
The “code 100” problem and nonlinear image coding Consider 8-bit pixel values proportional to luminance, where code zero represents black, and the maximum code value of 255 represents white, as in Figure 3.3 below. Code 100 lies at the point on the scale where the ratio between adjacent luminance values is 1%:
Owing to the approximate 1% contrast threshold of vision, the boundary between a region of code-100 samples and a region of code-101 samples is liable to be visible.
As pixel value decreases below 100, the difference in luminance between adjacent codes becomes increas-ingly perceptible: At code 20, the ratio between adja-cent luminance values is 5%. In a large area of smoothly
L L+∆L LB
Figure 3.2 A contrast sensi-tivity test pattern reveals that a difference in lumi-nance will be observed in certain conditions when ∆L exceeds about 1% of L. This threshold is called a just-noticeable difference (JND).
0
Figure 3.3 The “code 100”
problem with linear-light coding is that for code levels below 100 the steps between code values have ratios larger than the visual threshold: With just 256 steps, some steps are liable to be visible as banding.
0
Figure 3.4 The “code 100”
problem is mitigated by using more than 8 bits to represent luminance. Here, 12 bits are used, placing the top end of the scale at 4095. However, the majority of these 4096 codes cannot be distinguished visually.
varying shades of grey, these luminance differences are likely to be visible or even objectionable. Visible jumps in luminance produce contouring or banding artifacts.
Linear-light codes above 100 suffer no banding arti-facts. However, as code value increases toward white, the codes have decreasing perceptual utility: At code 200, the luminance ratio between adjacent codes is just 0.5%, below the threshold of visibility. Codes 200 and 201 are visually indistinguishable; code 201 could be discarded without its absence being noticed.
High-quality image display requires a contrast ratio of at least 30 to 1 between the luminance of white and the luminance of black. In 8-bit linear-light coding, the ratio between the brightest luminance (code 255) and the darkest luminance that can be reproduced without banding (code 100) is only 2.55:1. Linear-light coding in 8 bits is therefore unsuitable for high-quality images.
One way to mitigate the “code 100” problem is to place the top end of the scale at a code value much higher than 100, as sketched in Figure 3.4. If luminance is represented in 12 bits, with white at code 4095, the luminance ratio between code 100 and white reaches 40.95:1. However, the vast majority of those 4096 code values cannot be distinguished visually; for example, codes 4000 through 4039 are visually indistinguish-able. Rather than coding luminance linearly with a large number of bits, we can use many fewer code values, and assign them on a perceptual scale that is nonlinear with respect to light power. Such nonlinear-light – or perceptually uniform– coding is pervasive in digital imaging; it works so beautifully that many practitioners don’t even know it’s happening.
In video (including motion-JPEG, MPEG, and H.264/AVC), and in digital photography (including JPEG/JFIF/Exif), R’G’B’ components are coded in a per-ceptually uniform manner. A video display has a non-linear transfer function from code value to luminance.
That function is comparable to the function graphed in Figure 1.7 on page 8, and approximates the inverse of the lightness sensitivity of human vision. Perceptual uniformity is effected by applying a nonlinear transfer function– gamma correction– to each tristimulus esti-mate sensed from the scene. Gamma correction param-eters are chosen – by default, manually, or
automatically – such that the intended image appear-ance is obtained on the reference display in the refer-ence viewing environment. Gamma correction thereby incorporates both considerations of perceptual unifor-mity and considerations of picture rendering.
An example of 8-bit quantization as commonly used in computing is shown in the right-hand sketch of Figure 4.1, on page 37.
A truecolour image in computing is usually repre-sented in R’G’B’ components of 8 bits each, as I will explain further on page 36. Each component ranges from 0 (black) through 255 (white). Greyscale and true-colour data in computing is usually coded so as to exhibit approximate perceptual uniformity: The steps are not proportional to intensity, but are instead uniformly spaced perceptually. The number of steps required depends upon properties of the display system, of the viewing environment, and of perception.
Reference display and viewing parameters for studio video have historically been either poorly standardized or not standardized at all. In Reference display and viewing conditions, on page 427, I summarize a set of realistic parameters.
Video standards such as BT.709 and SMPTE ST 274 have historically established standard opto-electronic conversion functions (OECFs), as if video had scene-referred image state and as if cameras had no
adjustments! In fact, video is effectively output (display) referred, and what matters most is not the OECF but the electro-optical conversion function, EOCF! Nonlinear coding is the central topic of Chapter 27, Gamma, on page 315; that chapter discusses OECF and EOCF.
If the threshold of vision behaved strictly according to the 1% relationship across the whole tone scale, then luminance could be coded logarithmically. For a contrast ratio of 100:1, about 462 code values would be required, corresponding to about 9 bits.
Conversely, display R’G’B’ values are proportional to displayed luminance raised to approximately the 0.42-power.-See Gamma, on page 315.
For reasons to be explained in Luminance and light-ness, on page 255, instead of modelling the lightness sensitivity of vision as a logarithmic function, in most digital imaging systems we model it as a power func-tion. The luminance of the red, green, or blue primary light produced by a display is made proportional to code value raised to approximately the 2.4-power. The equation in the margin computes the ratio of linear-light value between the highest 8-bit code value (255) and the next-lowest value (254): The 1.01-requirement log
of human vision is met. As code values decrease, the ratio gets larger; however, as luminance decreases the luminance ratio required to maintain the “JND” of vision gets larger. The 2.4-power function turns out to be a very good match to the perceptual requirement.
A related claim is that 8-bit imaging has an optical density range of about 2.4, where 2.4 is the base-10 log of
1/255. This claim similarly rests upon linear-light coding.
Eight-bit imaging systems are often claimed to have
“dynamic range” of 255:1 or 256:1. Such claims arise from the assumption that image data codes are linearly related to light. However, most 8-bit image data is coded perceptually, like sRGB, assuming a 2.2- or 2.4-power function at the display: For an ideal display, the dynamic range associated with code 1 would be close to a million to one. In practice, physical parameters of the display and its environment limit the dynamic range.
The cathode ray tube (CRT) was, for many decades, the dominant display device for television receivers and for desktop computers. Amazingly, a CRT exhibits an EOCF that is very nearly the inverse of vision’s lightness sensitivity! The nonlinear lightness response of vision and the power function intrinsic to a CRT combine to cause the display code value (historically, voltage) to exhibit perceptual uniformity, as demonstrated in Figures 3.5 and 3.6 (opposite). The CRT’s characteris-tics were the basis for perceptual uniformity for the first half century of electronic imaging. Most modern display devices – such as LCDs, PDPs, and DLPs – do not have a native, physical power function like that of a CRT;
however, signal processing circuits impose whatever transfer function is necessary to make the device behave as if it had a 2.2- or 2.4-power function from signal value to displayed tristimulus value.
In video, this perceptually uniform relationship is exploited by gamma correction circuitry incorporated into every video camera. The R’G’B’ values that result from gamma correction– the values that are processed, recorded, and distributed in video– are roughly propor-tional to the 0.42-power of the intended display lumi-nance: R’G’B’ values are nearly perceptually uniform.
Perceptual uniformity allows as few as 8 bits to be used for each R’G’B’ component. Without perceptual unifor-mity, each component would need 11 bits or more.
Image coding standards for digital still cameras – for example, JPEG/Exif – adopt a similar approach.
1
See Bit depth requirements, on page 326.
0 50 100 150 200 250 Pixel value, 8-bit scale
Figure 3.5 A greyscale ramp on a CRT display is generated by writing successive integer values 0 through 255 into the columns of a framebuffer. When processed by a digital-to-analog converter (DAC), and presented to a CRT display, a perceptually uniform sweep of lightness results. A naive experimenter might conclude – mistakenly! – that code values are proportional to intensity.
Figure 3.6 A greyscale ramp, augmented with CIE relative luminance (Y, proportional to inten-sity, on the middle scale), and CIE lightness (L*, on the bottom scale). The point midway across the screen has lightness value midway between black and white. There is a near-linear relation-ship between code value and lightness. However, luminance at the midway point is only about 18% of white! Luminance produced by a CRT is approximately proportional to the 2.4-power of code value. Lightness is approximately the 0.42-power of luminance. Amazingly, these rela-tionships are near inverses. Their near-perfect cancellation has led many workers in video, computer graphics, and digital image processing to misinterpret the term intensity, and to underestimate – or remain ignorant of – the importance of nonlinear transfer functions.
0 50 100 150 200 250
Pixel value, 8-bit scale
Luminance, Y, relative 0 0.02 0.05 0.1 0.2 0.4 0.6 0.8 1
CIE Lightness, L* 010 20 40 60 80 100
Linear and nonlinear
Modern image sensor devices such as CCD and CMOS sensors effectively convert photons to electrons: They produce signals whose amplitude is proportional to physical intensity; they are said to be linear.
Video signals were historically processed through analog circuits that had linear response to voltage.
Today’s digital video systems are often linear with respect to the arithmetic performed on the pixel values.
Video systems are said to be linear.
Linear is an adjective, not a noun! However, linearity in one domain cannot be carried across to another domain if a nonlinear function sepa-rates the two. In video, scene luminance is in a linear optical domain, and the video signal is subject to linear operations in the electrical signal domain. However, OECF and EOCF functions are imposed between the domains. Luminance and pixel value are therefore not linearly related. When you ask an optical engineer if her system is linear, she will say, “Of course!”– referring to
Linear is an adjective, not a noun! However, linearity in one domain cannot be carried across to another domain if a nonlinear function sepa-rates the two. In video, scene luminance is in a linear optical domain, and the video signal is subject to linear operations in the electrical signal domain. However, OECF and EOCF functions are imposed between the domains. Luminance and pixel value are therefore not linearly related. When you ask an optical engineer if her system is linear, she will say, “Of course!”– referring to