Fragment Processing

The fragment processing stage consists of the rasterization unit and the fragment program. Rasterization denotes the process of converting screen-space primitives into fragments. Each fragment corresponds to a single raster position in the resulting image, and many fragments may contribute to the final color of a pixel. The rasterization unit calculates a set of attributes for each fragment it generates by interpolating the vertex at-tributes given at the primitive’s vertices. These primitive interpolants are the input of the fragment program. Based on this set of attributes, the fragment program computes the final color of the fragment.

2.3.1 Fragment Programs

Fragment programs are user-written microprograms that substitute major parts of the traditional fixed-function computation of the rasterization unit.

They are used to compute the final color and optionally the depth value

1Texel: short for texture element.

2.3 Fragment Processing 39 of each fragment. The fragment program is executed once per fragment:

Every time that primitive rasterization produces a fragment, the fragment processor receives a set of attributes, such as colors and texture coordinates, executes the fragment program once, and writes the final color and z-value of the fragment to the output registers.

The diagram for the programmable fragment processor is shown in Fig-ure 2.3. The instruction cycle of the fragment processor is similar to the

Emit

Figure 2.3. For each fragment, the programmable fragment processor executes a micro-program. In addition to reading the input and temporary registers, the fragment processor is able to generate filtered texture samples from the texture images stored in video memory. (Image inspired by Mark Kilgard’s original diagram inThe Cg Tutorial[71].)

i

vertex processor, with a separate path for texture fetch instructions. At first an instruction is fetched and decoded. The operands for the instruc-tion are read from either the input registers, which contain the fragment’s attributes, or from temporary registers, which are used to store interme-diate results. The mapping step again computes the component swizzling and negation.

If the current instruction is a texture fetch instruction, the fragment processor computes the texture address with respect to texture coordinates and level of detail. Afterwards, the texture unit fetches all the texels that are required to interpolate a texture sample at the given coordinates. These texels are finally filtered to interpolate the texture color value.

If the current instruction is not a texture fetch instruction, it is executed with the specified operands and the result is written to the respective target registers. At the end of the loop, the fragment processor checks whether or not there are more instructions to be executed and decides to reenter the loop or terminate the program by emitting the output registers to the fragment processing stage.

Textures are (one-, two-, or three-dimensional) raster images that are mapped onto the polygon according to texture coordinates specified at the vertices. For each fragment, these texture coordinates are interpolated, and a texture look-up is performed at the resulting position. This process generates a texture sample, which refers to an interpolated color value sam-pled from the texture map. For maximum efficiency, it is also important to take into account that most hardware implementations maintain a tex-ture cache. We will have a detailed look at textex-ture caching and memory management strategies in Chapters 8 and 17.

In Cg, texture images are declared as uniform parameters of type sampler1D, sampler2D, or sampler3D, with respect to the dimension of the texture. The sampler typessamplerRECTor samplerCUBEare used for

recti-// A simple fragment shader

float4 main ( half4 primary : COLOR half2 uv : TEXCOORD0,

uniform sampler2D texture) : COLOR {

float4 texel = tex2D(texture,uv);

return texel * primary;

}

Listing 2.2. A simple example of a fragment program in Cg. The final color is computed as the component-wise product of a texture sample and the primary color of the fragment.

2.3 Fragment Processing 41 linear textures and cube maps, respectively. The commands for sampling a given texture image aretex1D,tex2D, and so forth. As an example, a simple fragment program is shown in Listing 2.2. It performs a 2D texturing oper-ation, which modulates the primary color with the texture color, equivalent to the fixed function pipeline in a standardOpenGLenvironment. We will see a variety of different fragment programs throughout this book.

texture 0

polygon texture 1

(s ,t )0 0

RGBA s

t

G final color

(s ,t )1 1 R (s ,t )2 2

(s,t )

Figure 2.4. Dependent texture look-up: the texture coordinates(s, t)are interpo-lated as usual from the values given at the polygon vertices. An RGBA quadruplet is obtained from the first texture. The red (R) and the green (G) components of this quadruplet are used as texture coordinates for the second texture look-up. The resulting final texel value is used to color the fragment.

2.3.2 Texture Indirection

One important feature of the programmable fragment processor is its capa-bility to performdependenttexture look-ups. This means that the texture coordinates used to access a texture image are not directly computed from the values given at the vertices. Instead, the texture coordinates are ob-tained by sampling another texture image. This concept is illustrated in Figure 2.4, and the corresponding fragment program is given in Listing 2.3.

// fragment program using texture indirection half4 main( half2 texUV : TEXCOORD0,

uniform sampler2D first texture,

uniform sampler2D second texture) : COLOR {

half4 texUV2 = tex2D(first texture, texUV);

half4 result = tex2D(second texture, texUV2.xy);

return result;

}

Listing 2.3. A simple Cg fragment program using texture indirection.

i i

i i

i i

i i

42 GPU Programming

Dependent textures do not need to be interpreted as image data. In many real-time applications, dependent textures are used to implement abstract one-, two-, or three-dimensional functions as look-up tables. If you ever find that the analytical evaluation of a complex function is too expensive for real-time performance, it can probably be pre-computed as a large table and accessed via dependent texture look-up at runtime.