In-Band MCTF (IB-MCTF)

Figure7.11represents the block diagram for IB-MCTF. In this scheme, the motion estimation [13] is performed in-band or in the sub-bands. In case of IB-MCTF, the motion estimation is performed in-band, and the performance of the transcoder suf-fers from the problem of shift variance. Because DWT is shift variant, it is not possible to use the critically sampled DWT sub-bands for motion estimation. So for effective motion compensation in the wavelet domain, we need to use over-complete DWT.

Apart from the sequence of temporal and spatial decomposition and performing ODWT instead of the critically sampled DWT, the basic framework of IB-MCTF [9]

is similar to that of SD-MCTF. IB-MCTF also has the same structure for MCTF except for Motion Vector (MV) information and Inverse Motion Vector (IMV) infor-mation, in which the phase information has to be computed. IB-MCTF is also called 2D+t transform.

Temporal Split Predict Update

Fig. 7.11 Block diagram for IB-MCTF with predicting and update blocks

Candidate

Fig. 7.12 Step 1: Spatial decomposition to obtain the sub-bands; critically sampled DWT for candidate frame and ODWT for reference video frame

The algorithm of IB-MCTF consists of a number of steps as follows:

Step 1 The first step of IB-MCTF involves splitting of the video frame into the sub-bands. Because of the problem of shift variance, ODWT is performed to obtain the phase information. The even frames are the reference frames (Fig.7.12).

Step 2 Motion estimation is performed on the sub-band LL of the candidate frame and the reference frame. For motion estimation, the information that is sent to the ME block is the candidate sub-band, and the reference sub-bands for

100 7 Introduction to Scalable Image and Video Coding

Motion Estimation using Block

Matching

00 01

10 11

Reference Sub-bands Candidate Sub-band

MVx MVy Phase

dx dy 11

Best matched macro-block for the candidate macro-block

Fig. 7.13 Step 2: Motion estimation process in case of IB-MCTF, the phase information also considered for ME

00 01

10 11

Reference Sub-bands

predicted Sub-band

MVx MVy Phase

dx dy 11

Best matched macro-block Motion vector with Phase

Information ^dx

dy Sub-band of the

particular phase selected

Best matched macro-block copied in to the candidate

macro-block position

Fig. 7.14 Step 3: Predicted sub-band obtained using information from the reference sub-band and the motion vector data along with false information

Pixel-by-pixel subtraction

Candidate sub-band Predicted sub-band Error sub-band

Fig. 7.15 Step 4: Error sub-band by subtracting predicted sub-band from the candidate sub-band

all the phases (00, 01, 10 and 11). ME is performed using the block matching algorithm, and the output of the ME block is the motion vector. In addition to the motion vector, phase information is obtained (Fig.7.13).

Step 3 Using the phase information and the motion vector, a predicted sub-band is obtained. The best matched macro-block from the reference sub-band of the particular phase is copied into the position of the candidate MB in the predicted sub-band (Fig.7.14).

Step 4 The error sub-band is a pixel-by-pixel subtraction of the predicted sub-band from candidate sub band (Fig.7.15).

Step 5 The next step is the formation of the update sub-band. For this process, the inverse motion vector is obtained. The process for IMV is not as straight-forward as in case of SD-MCTF. The algorithm to obtain the inverse mo-tion vector has been menmo-tioned in [9, 12]. If the phase of the best match is 00, then the inverse motion vector is obtained by the same procedure as in SD-MCTF. However, for non-zero phase, the inverse motion vector is mod-ified depending on the phase information. The problem of multi-connected and non-connected macro-blocks is still present in the case of IB-MCTF.

Unlike the predict stage where the different phases of the reference sub-band are present, the different phases of the error sub-band are unavailable at this stage. Here, CODWT is performed. The phase information and the inverse motion vector are used to form the updated sub-band (Figs.7.16and7.18).

Step 6 This is the averaging step. In this step, the reference sub-band is added to the updated sub-band/2. Each pixel value in the average sub-band is formed by

00 01

10 11

Phase information using CODWT

updated Sub-band

IMVx IMVy Phase

dx’ dy’ 11

Best matched macro-block Inverse Motion vector

with Phase Information dx’

dy’

Sub-band of the particular phase

selected

Best matched macro-block copied in to the error macro-block position Error sub-band

CODWT

Fig. 7.16 Step 5: Formation of the update frame from the error frame using IMV and phase

102 7 Introduction to Scalable Image and Video Coding

Pixel-by-pixel addition

Reference sub-band Update sub-band/2 Average sub-band

Fig. 7.17 Step 6: Formation of the average sub-band by adding the reference sub-band to the update sub-band/2

MVx MVy Phase

dx dy 11

Motion vector with Phase

For a macro-block at pos. ((x+dx),

(y+dy)) IMVx information is (-dx,-dy).

MVx MVy

dx’ dy’

MVx MVy Phase

dx dy 11

x y

Inverse motion vector without

phase compensation

Phase information along with x and y

Inverse motion vector with phase

Fig. 7.18 Inverse motion vector with phase information from motion vector using IB-MCTF algo-rithm mentioned in [9]

the sum of the pixel value of reference sub-band and 1/2 value of the pixel of the updated sub-band (Fig.7.17).

Step 7 The previous steps of IB-MCTF algorithm are performed for decomposing the video stream for one spatial and one temporal level. For further decom-position, step 1 should be repeated for further spatial decomposition levels and then Step 2 to Step 6 should be repeated for subsequent temporal levels.