Embedded Linux – GSE5
- 1 -
4 - Linux framebuffer
Objectives
• Develop a userspace program displaying a YUV video file based on Linux Framebuffer.
Lab preparation
• Read section 1.1.
• A sample project is available at http://users.polytech.unice.fr/~bilavarn/ (Example 2 - Video display / Linux Framebuffer). Explain broadly what this program does and the role of function mmap.
Description
This lab addresses the use of the mmap method and an introduction to video and image display through the use of the Linux framebuffer and development of a YUV to RGB conversion program.
A copy of the sample project is present on the SD MMC of the Exynos boards in directory
Development/LABS/TP_FRAMEBUFFER.
1.1 Introduction
1.1.1 YUV 4 :2 :0
Different file formats can be used in digital video processing. The YUV model defines a color space based on 3 components: Y is the luminance (grey level) and U and V are
the chrominance (color). YUV 4:2:0 is a class of the YUV format which defines the proportion of YUV samples: 1 pixel U and 1 pixel V for 4 pixels Y. Less chrominance information is present because the eye is less sensitive to color information, and more sensitive to contrast. Frame data in a YUV 4:2:0 video file is thus organized as follows:
Figure 1. Frame data in a YUV 4:2:0 video file.
A YUV sample video (GTAV_1280x720_482f.yuv) of a 1280x720 (CIF) sequence of 482 frames is provided to help developing and test your code. In accordance with figure 1, data is stored in the following order in this file:
Embedded Linux – GSE5
- 2 -
- 1280x720 pixels Y
- 1280x720/4 pixels U
- 1280x720/4 pixels V 1.1.2 YUV to RGB conversion The conversion scheme is given by:
The corresponding expression is:
R = (Y – 16) + 1,13983 * (V – 128)
G = (Y – 16) – 0,39465 * (U – 128) – 0,58060 * (V – 128) B = (Y – 16) + 2,03211 * (U – 128)
1.1.3 Framebuffer
The Linux framebuffer is a portion of RAM memory used for graphic display. Previous equation transforms the YUV triplet of a pixel into a RGB triplet. The framebuffer used has a 32-bit pixel configuration (RGB888), which means that the three RGB components of a same pixel are coded using a single 32-bit data (unsigned int). From the float R, G and B values resulting from previous expression, the 32-bit unsigned int RGB pixel value to be written in the framebuffer can be constructed using the following bit masks (transparency T can be set to 255):
(((unsigned int)(B))&0x000000FF | ((unsigned int)(G)<<8)&0x0000FF00 | ((unsigned int)(R)<<16)&0x00FF0000 | ((unsigned int)(T)<<24)&0xFF000000)
When this RGB pixel value is written to an address of the framebuffer (which beginning is given by fb_ptr), the corresponding pixel is displayed. Pixels of an image are disposed at consecutive addresses from fb_ptr, and line by line (fb_ptr[0] → 1st pixel, fb_ptr[1] →
2nd pixel, fb_ptr[2] → 3rd pixel, etc).
1.2 Floating point YUV / RGB conversion
All developments are made using Exynos boards under Linux Ubuntu (see Lab3).
The sample project is present on the SD MMC of the Exynos board in directory
Development/LABS/TP_FRAMEBUFFER.
This project let you read a YUV 4:2:0 video file and opens the Linux Framebuffer.
• Compile and run the project on the Exynos board.
• Explain how the program makes use of the framebuffer using mmap. R
G B
1 0 1,13983
1 -0,39465 -0,58060
1 2,03211 0
Y U V
=
16 128 128
-
Embedded Linux – GSE5
- 3 -
• You can have a look to function blueRectangle to understand how to fill a rectangle of size VIDEO_WIDTH*VIDEO_HEIGHT with blue pixels.
• Complete function YUVtoRGB and test your code. In this first version, computations are made using floating point arithmetic. Make a copy of this version and give the execution time.
Note: you have to switch to full command line mode (CTRL – ALT – F1) on the Exynos board to display video frames properly. You can come back to Ubuntu desktop with CTRL – ALT – F7.
1.3 Fixed point YUV / RGB conversion
Floating point arithmetic is relatively slow, especially when used in embedded processors without Floating Point Unit (FPU). A typical optimization is thus to use fixed point arithmetic.
The principle of fixed point computation is to replace floating point values in the program (float, double) by integer values (short, int, long int). To achieve this, we multiply both sides of an expression by a same integer value. This value is chosen to be a power of two that will correspond to the decimal part of the representation.
In our example, we will use 14 bits for the decimal part. The fixed point transformation consists in multiplying both sides of equation by 16384, which is equivalent to consider a 32- bit representation where 18 most significant bits are for the integer part and the other 14 bits are for the decimal part (18.14):
16384*R = 16384* (Y – 16) + 16384*1,13983 * (V – 128)
16384*G = 16384* (Y – 16) – 16384*0,39465 * (U – 128) – 16384*0,58060 * (V – 128)
16384*B = 16384* (Y – 16) + 16384*2,03211 * (U – 128)
Which is equivalent to:
R = ((Y – 16)<<14 + 18675 * (V – 128)) >>14
G = ((Y – 16)<<14 – 6466 * (U – 128) – 9513 * (V – 128)) >>14 B = ((Y – 16)<<14 + 33294 * (U – 128)) >>14
• Transform previous code in order to realize the YUV to RGB conversion with fixed point computation, and check that video display is still correct.
• Compare with previous execution time ? How much is the difference and why ?
Lab report : a pdf file giving answers to all the questions raised and a copy of your code with explanations.
