The volume-rendering equation in its differential form (Equation 1.6) can be solved for radiance by integrating along the direction of light flow from the starting point s =s0 to the endpoint s= D, leading to the volume-rendering integral
1.3 Volume-Rendering Integral 9 The termI0represents the light entering the volume from the background at the position s=s0; I(D) is the radiance leaving the volume at s=D and finally reaching the camera. The first term in Equation 1.7 describes the light from the background attenuated by the volume. The second term represents the integral contribution of the source terms attenuated by the participating medium along the remaining distances to the camera.
The term
τ(s1, s2) =
s2
s1
κ(t) dt (1.8)
is defined as the optical depth between positions s1 and s2. The optical depth has a physical interpretation in the form of a measure for how long light may travel before it is absorbed; i.e., optical depth indicates the typical length of light propagation before scattering occurs. Small values for the optical depth mean that the medium is rather transparent, and high values for the optical depth are associated with a more opaque material. The corresponding transparency (for a material betweens1ands2) is
T(s1, s2) =e−τ(s1,s2)=e
−sÊ2
s1
κ(t) dt
. (1.9)
With this definition of transparency, we obtain a slightly different version of the volume-rendering integral
I(D) =I0T(s0, D) + D s0
q(s)T(s, D) ds .
The volume-rendering integral is the most common description of volume rendering. Large portions of this book are devoted to efficient algorithms for computing this integral.
1.3.1 Local Illumination for Volume Rendering
The volume-rendering integral in its classic form, Equation 1.7, accurately represents the emission-absorption model but no scattering effects. Single scattering of external light can be included to introduce greater realism into this optical model. In a simple volume-shading model, the external illumination is assumed to unimpededly reach a point in the volume from an outside light source, neglecting any absorption or scattering along its way.
Single scattering is often approximated by a local illumination model that imitates local surface rendering, such as the Phong or the Blinn-Phong
i i
i i
i i
i i
10 Theoretical Background and Basic Approaches
illumination models. In volume shading, the gradient of the scalar field serves as the normal vector for these local illumination models because the gradient is identical to the normal vector on an isosurface through the respective point in space. In this way, volume shading produces an effect similar to an illuminated isosurface. Local illumination is included in the volume-rendering integral (see Equations 1.5 and 1.7) by extending the source term to
qextended(x, ω) =qemission(x, ω) +qillum(x, ω).
The emissivity qemission(x, ω) is identical to the source term in the pure emission-absorption model. The additional scattering termqillum(x, ω) de-scribes the additional light coming from local reflection (i.e., single scatter-ing). We refer to Chapter 5 for a detailed discussion of local illumination in volume rendering. The advantage of local illumination is that it does not increase the complexity of the computations for the volume-rendering integral but improves the perception of volume models.
1.3.2 Density Model
Sometimes, optical properties are derived from a density model of the par-ticipating medium. In this description,ρrepresents the density of the ma-terial, and all optical properties are weighted by this density. For example, the total absorption coefficientχis replaced byχ according to
χ=χρ .
There are similar substitutions for the true absorption coefficient,κ=κρ, the scattering coefficient,σ=σρ, and the true emission term,q=qρ.
This density model builds upon a description of a density of particles that are responsible for emission, absorption, and scattering. Originally, such a particle model was proposed by Sabella [230] for his density-emitter model of volume rendering. Williams and Max [300] provide a detailed discussion of a volume-density model.
1.4 Discretization
The main goal of volume rendering is to compute the volume-rendering integral, Equation 1.7. Typically, the integral cannot be evaluated analyti-cally. Instead, numerical methods are applied to find an approximation as close to the solution as possible.
1.4 Discretization 11
s0 s1 s2 s3 sn-1 sn s
f(s) segment 3
Figure 1.4. Partitioning of the integration domain into several intervals. The inter-vals are described by locationss0 < s1 < · · · < sn−1 < sn. Theith interval or segment is[si−1, si]. The hatched box indicates the integration result for the third segment.
1.4.1 Splitting into Several Integration Intervals
A common approach splits the integration domain intonsubsequent inter-vals. The intervals are described by locations s0< s1<· · ·< sn−1 < sn, where s0 is the starting point of the integration domain and sn = D is the endpoint. Please note that the intervals do not necessarily have equal lengths. Figure 1.4 illustrates the partitioning of the integration domain into several intervals or segments.
Considering the light transport within theith interval [si−1, si] (with 0< i≤n), we can obtain the radiance at locationsi according to
I(si) =I(si−1)T(si−1, si) +
si
si−1
q(s)T(s, si) ds .
We introduce a new notation for the transparency and color contribution (i.e., radiance contribution) of theith interval:
Ti=T(si−1, si), ci=
si
si−1
q(s)T(s, si) ds . (1.10)
The hatched box in Figure 1.4 illustrates the result of integration over one interval. The radiance at the exit point of the volume is then given by
I(D) =I(sn) =I(sn−1)Tn+cn= (I(sn−2)Tn−1+cn−1)Tn+cn=. . . , which can be written as
I(D) = n i=0
ci n j=i+1
Tj, withc0=I(s0). (1.11)
i i
i i
i i
i i
12 Theoretical Background and Basic Approaches
s0 s1 s2 s3 sn-1 sn s f(s)
Figure 1.5. Approximation of an integral by a Riemann sum.
In general, the notationl
i=kai means that all terms ai (for k ≤i ≤l) are summed. Similarly,l
i=kai is a notation for the multiplication of all terms ai (for k ≤ i ≤ l). Often, transparency Ti is replaced by opacity αi= 1−Ti.
At this point, we have halfway solved the problem of computing volume rendering: the integration domain is segmented into n discrete intervals, and the summations and multiplications in Equation 1.11 can be com-puted. The missing point is the evaluation of the transparency and color contributions of the intervals.
A most common approach approximates the volume-rendering integral by a Riemann sum overnequidistant segments of length ∆x= (D−s0)/n.
Here, the function to be integrated is approximated by a piecewise-constant function, as illustrated in Figure 1.5. The integral over a single interval corresponds to the area of the rectangle defined by the function value at a sampling point and by the sampling width (see the hatched boxes in Figure 1.5).
In this approximation, the transparency of theith segment is
Ti ≈e−κ(si)∆x (1.12)
and the color contribution for theith segment is
ci≈q(si)∆x . (1.13)
Many volume-rendering methods employ the above approximation with a piecewise-constant function over equidistant sampling points. A prominent example is the widely used approach with texture-based volume rendering, as explained in Chapter 3.
Despite the widespread use of a piecewise-constant approximation with equidistant sampling, other alternatives are sometimes employed to ob-tain a better solution for the segments’ transparency and color contribu-tions. One approach is to apply alternative quadrature rules (i.e., inte-gration rules) for evaluating the volume-rendering integral, for example,
1.4 Discretization 13 the trapezoidal rule, Simpson’s rule, or other higher-order rules accord-ing to Newton-Cotes formulas [217]. Unfortunately, the volume-renderaccord-ing integral typically contains discontinuities, for example, at the boundary between different materials. Therefore, higher-order integration methods often do not provide appropriate accuracy. Instead, adaptive sampling is a much more common approach for improved volume rendering because it modifies the sampling rate based on the properties of the integrand (see Section 9.1 for details). Another alternative is a Monte Carlo approach that introduces a jittering in sampling positions to avoid artifacts from uniform sampling (also see Section 9.1 for details).
Finally, pre-integration partially computes the volume-rendering inte-gral in a preprocessing step, which can be done with high accuracy. The essential idea is to pre-compute all possible results for the volume-rendering integral along a single segment, as described by Equation 1.10. During run-time, only the discrete summations and multiplications according to Equa-tion 1.11 are evaluated. Therefore, pre-integraEqua-tion leads to high rendering quality at high speed, making it one of the most popular approaches for volume rendering. Details of pre-integration are given in Sections 4.5 and 9.3.
1.4.2 Compositing Schemes
Compositing is the basis for the iterative computation of the discretized volume-rendering integral (Equation 1.11). The idea is to split the summa-tions and multiplicasumma-tions that are contained in Equation 1.11 into several, yet simpler operations that are executed sequentially. Two different basic compositing schemes are common: front-to-back compositing and back-to-front compositing.
The front-to-back compositing scheme is applied when the viewing rays are traversed from the eye point into the volume. Here, we use slightly different variable names: C represents a color, typically given as a three-channel RGB (red, green, blue) color value. Both the newly contributed radiancec and the accumulated radianceI from the previous sections are now associated with such a color description.
Then, the front-to-back iteration equations are Cˆi= ˆCi+1+ ˆTi+1Ci, Tˆi= ˆTi+1(1−αi), with the initialization
Cˆn=Cn, Tˆn= 1−αn.
i i
i i
i i
i i
14 Theoretical Background and Basic Approaches
The results of the current iteration step are ˆCi and ˆTi; ˆCi+1 and ˆTi+1 are the accumulated results of the previous computations. The source termCi
and the opacityαiare given by the transfer function or they originate from a physical model of a gas (see Section 1.1). The iteration starts at the first sampling positioni=n(closest to the camera) and ends at i= 0 (at the backside of the volume).
By renaming the variables according toCdst = ˆCj (withj =i, i+ 1), Csrc =Ci, αdst = 1−Tˆj (withj =i, i+ 1), andαsrc =αi, front-to-back compositing can be written in its most common way:
Cdst←Cdst+ (1−αdst)Csrc,
αdst←αdst+ (1−αdst)αsrc. (1.14)
This set of assignment equations explicitly shows the iterative nature of compositing. Variables with subscriptsrc(as for “source”) describe quan-tities introduced as inputs from the optical properties of the data set (e.g., through a transfer function or from a physical model of a gas), whereas vari-ables with subscript dst (as for “destination”) describe output quantities that hold accumulated colors and opacities. Equation 1.14 is repeatedly applied while marching along a ray, updating color Cdst and opacity αdst along its way.
By reversing the traversal direction, we obtain the back-to-front com-positing scheme:
Cˆi= ˆCi−1(1−αi) +Ci, Tˆi= ˆTi−1(1−αi), with the initialization
Cˆ0=C0, Tˆ0= 1−α0.
The iteration starts ati= 0 and ends ati=n. Note that the accumulated transparency ˆTi is not needed to compute the color contribution ˆCi and can thus be omitted.
Analogously to the front-to-back scheme, we can also rewrite the back-to-front compositing in an explicitly iterative fashion:
Cdst←(1−αsrc)Cdst+Csrc. (1.15)
1.4 Discretization 15 Note that there is no iterative update of opacity needed because accu-mulated opacity (or transparency) is not required to determine the color contribution.
In addition to the above compositing schemes, alternative approaches are sometimes used. For example, maximum intensity projection (MIP) and x-ray or weighted sum projections are often applied in medical imaging applications. MIP is computed according to the compositing equation
Cdst←max(Cdst, Csrc). (1.16) The final result is the maximum color contribution along a ray. This com-positing scheme is independent of the traversal order; i.e., it may be applied in a back-to-front, a front-to-back, or any other order. The main applica-tion for MIP is virtual angiography—the display of vessel structures in medical scans (see the survey article [218]). More details on MIP and its implementation are provided in Section 3.2.3, along with an example image in Figure 3.4. X-ray or weighted sum projections result in a weighted sum of color contributions along a ray. All of these alternative schemes have an order-independent compositing process in common [187].
In the remainder of this book, unless otherwise noted, we use a com-positing scheme that implements the emission-absorption model—either according to back-to-front or front-to-back traversal.
We would like to point out that colorsC should be interpreted as ra-diances values (per wavelength band) in order to obtain a consistent and physically correct description of light transport. From the radiance that arrives at the virtual camera, the corresponding detector response on the image plane can be derived. For fixed camera parameters and a linear response of the camera, the strength of the recorded color values is pro-portional to the incident radiance. Therefore, the final radiance C can be interpreted as a measure for RGB colors. Moreover, it is common prac-tice to already specify intermediate colorsCas RGB values—and we often adopt this sloppy way of dealing with radiance throughout this book.
1.4.3 Opacity Correction
Volume rendering is typically described in terms of the discrete sum (Equa-tion 1.11) or the corresponding compositing schemes. This discrete ap-proach often assumes an equidistant sampling, for example, in conjunction with the approximation through a Riemann sum. A problem arises when the sampling rate needs to be changed: the discretized opacity and color contributions need to be modified accordingly because their values depend on the sampling distance (see Equations 1.10, 1.12, and 1.13). There are several reasons for changing the sampling rate, such as adaptive sampling
i
16 Theoretical Background and Basic Approaches
for accelerated volume rendering and improved image quality (see Sec-tion 9.1) or purely geometric reasons in slice-based volume rendering (see the discussion of 2D texture slicing in Section 3.2).
According to Equations 1.9 and 1.10, the transparency of a segment of length ∆x(ranging from positionsi to si+ ∆x) is
T =e
−si+∆xÊ
si κdt
=e−κ∆x,
if we assume a constant absorption coefficientκwithin the segment. Analo-gously, a segment of different length ∆˜xhas transparency ˜T = exp(−κ∆˜x).
Therefore, the two transparency values are related to each other according to
T˜=T(∆˜∆xx).
In terms of opacity, the two intervals of different lengths are related by
˜
α= 1−(1−α)(∆˜∆xx). (1.17) This equation describes opacity correction and allows for varying sample rates even if the optical properties are only given with respect to a discrete version of the volume-rendering integral, i.e., even if the optical properties are described by opacities and not by a continuous absorption coefficient κ(s).
Analogously, the color contribution, which is approximately c = q∆x for the original length of a segment, is transformed according to the color correction equation
So far, we have assumed associated colors, as introduced by Blinn [14].
Associated colors consist of color components that are already weighted by their corresponding opacity. An alternative description uses color compo-nents that have not been premultiplied with opacity. The previous equa-tions have to be modified to allow for nonassociated colors: original color terms have to be replaced by color terms that are explicitly weighted by opacity.
For example, Csrc needs to be substituted by αsrcCsrc in the iterative compositing equations. With nonassociated colors, the front-to-back com-positing Equation 1.14 is replaced by
Cdst←Cdst+ (1−αdst)αsrcCsrc αdst←αdst+ (1−αdst)αsrc.