5.12 High-level syntax and slice structure
5.12.6 SEI messages
JCTVC-I0218 Changes to the hash SEI calculation for improved usability [T. Hellman, W.
Wan (Broadcom)]
Reviewed in HLS BoG (chaired by J. Boyce).
This contribution claims that the two hash SEI messages, as presently defined, are difficult for a real-time encoder or decoder to use. It states that three independent post-decode memory passes are necessary to compute the hashes. It recommends two changes: computing separate MD5 hashes for luma and chroma, rather than just one, and replacing the second hash type (CRC) with a checksum. It claims that the first change allows real-time decoders to compute the MD5 at display time, and the second allows real-time computation of the hash at both encode time and decode time.
It was suggested that the size of the MD5sum could be reduced, and that 16 bytes is stronger than needed.
Replacing CRC with a checksum allows order independent processing, but offers less protection. It was noted that Rec. ITU-T H.271 uses CRC32.
It was noted that checksum or CRC may be useful for error detection of parameter sets.
A suggestion was made to use a checksum method as a 3rd method, not removing the CRC.
A suggestion was made to clarify operation if calculation overflows 32 bits.
The BoG recommended to add support for the additional hash type described as solution 2 of I0218 (an order-independent scheme) for the checksum method, in addition to the CRC and MD5sum schemes.
Also always send 3 checksums/hashes – one per color component. Decision: Adopt (both aspects described within this paragraph).
JCTVC-I0245 LCU based input order scanning for hash SEI Message [R. R. Srinivasan, M. Mody, C. Ghone (TI)]
Reviewed in HLS BoG (chaired by J. Boyce).
Hash SEI messages are defined to provide checksum of the decoded picture in the current access unit.
With the current definition, computation of these checksum can be done only as an end of frame operation, as there is a sequential dependency between different color components and within in a color component, lines taken in raster scan order are input to checksum engine. This proposal suggests breaking the dependency between different color components and also proposes a way to do this checksum
calculation during LCU decoding process immediately after in-loop filtering process – thereby making the checksum calculation as a LCU based operation, instead of a line-based operation. This reduces frame-end computation complexity and reduces memory bandwidth.
It was proposed to send 3 separate MD5sum values, one for each color component.
The processing is not aligned with LCU grid because of post-filtering.
The interaction with slices and tiles was questioned. The coding order of the SEI message w.r.t. the coded picture should be considered. Perhaps placing a POC in the SEI message would help. There is a
restriction now that SEI messages must precede the VCL NAL units of an access unit, and perhaps this should be relaxed.
First issue: 1 vs. 2 vs. 3 MD5sums. It was remarked that having 3 could help identify which color component had a problem.
It was recommended to have 3 MD5sums, one per color component. If we keep the CRC, it should also have 3.
It was remarked that if the checksum were used, individual LCU operation and offset alignment is unnecessary.
See notes relating to I0218.
JCTVC-I0044 Modification of recovery point SEI message [K. Kazui, J. Koyama, S.
Shimada, A. Nakagawa (Fujitsu)]
This contribution proposes to modify the specification of the recovery point SEI message in HM6.0 text.
Mainly, recovery_frame_cnt syntax element is replaced with recovery_poc_cnt syntax element for specifying a recovery point since frame_num syntax element is no more used in HM6.0.
Decision (BF): Adopt, and remove changing_slice_group_idc, and set prevPicOrderCntMsb = 0 at CRA.
JCTVC-I0057 Frame packing arrangement SEI extension for HEVC [O. Nakagami, T.
Suzuki (Sony)]
Reviewed in HLS BoG (chaired by J. Boyce).
In the current draft, frame packing arrangement SEI message is defined the same as the SEI message defined in AVC (Rec. ITU-T H.264 | ISO/IEC 14496-10). This contribution proposes to modify the frame packing arrangement SEI for HEVC to support coding tools in HEVC.
Two extensions to the frame packing arrangement SEI were proposed. The proposed extension links the view information and coding structure. One is an extension for frame sequential coding using
temporal_id. The other is an extension for frame packing coding using tile.
The proposed extensions were suggested to enable represent view information with fewer bits.
Furthermore, it was asserted that decoders can output only the base view without entire decoding process since the additional view is independent of the base view in the coding structure point of view.
There was a suggested use case using temporal_id.
Another use case was described using tiles. A change was proposed to the SEI message syntax, such that the frame packing SEI message would use tile parameters from the PPS. It was noted that the proposed syntax has a parsing issue, as the proposed SEI message uses PPS parameters that may not be activated until the SH is parsed.
The semantics for the temporal_id aspects were not clearly described.
The BoG recommended no action.
JCTVC-I0058 On stereo 3D coding using frame packing arrangement SEI [O. Nakagami, T. Suzuki (Sony)]
Reviewed in HLS BoG (chaired by J. Boyce).
Frame packing stereo 3D coding using the frame packing arrangement SEI message is supported in the current draft. However, one asserted issue is compatibility among decoders since decoding SEI messages
is not a mandatory process. This contribution proposes three alternative methods to realize the 2D compatibility among the decoders.
The contribution had 3 proposal options. It proposed adding a 1-bit flag in VUI to indicate the existence of the SEI message.
It was remarked that the contributions uses “should” wording which isn't done in the spec.
If some conditions in SEI message are met, the normative decoding cropping process is proposed to be changed.
It was mentioned that DVB uses this method for AVC frame packing.
There was an IT NB comment to fix such schemes in HEVC.
The BoG recommended further review.
This contribution proposes a VUI flag combined with the FPA SEI.
A participant remarked that the flag might not actually be necessary.
The concept of the proposal seemed generally understood.
Further study was recommended, e.g. in relation to other 3D video considerations. Some coordination seems needed with other activities.
See also the notes relating to I0072.
JCTVC-I0072 Usage of cropping rectangle and sample aspect ratio to support
2D-compatible services in “Frame 2D-compatible Plano-Stereoscopic 3DTV” within the HEVC specification [M. Arena, P. Sunna (RAI)]
This contribution proposes a signalling scheme that is asserted to enable distribution of 3D stereoscopic frame compatible video in a "2D compatible" manner.
There is a desire for "2d service compatibility". The contribution proposes to use the frame cropping rectangle and SAR for this purpose.
The concept of the "legality" of using the region outside the cropping rectangle was discussed.
Specific syntax was not proposed in the contribution – it was a concept-level contribution.
It was remarked that there is no legacy of deployed HEVC decoders, so decoders will be created with awareness of whatever is written in version 1 of the spec.
It was suggested to be feasible to use the cropping window for 2D compatibility which would enable 2d receivers to not need to pay attention to the SEI message.
Another suggestion was to have two cropping windows specified.
See also notes relating to I0058.
The contribution was reviewed. No action was taken; further study is needed: In the case of stereo, it may be inappropriate to copy SEI messages from AVC "as is"; this should be investigated in the broader range of upcoming definitions on 3D and also considered by the upcoming JCT on 3D video extensions.
JCTVC-I0062 Comments on field indication SEI message [M. Li, P. Wu (ZTE), C. Fogg (Harmonic)]
In this document, some modifications of the field indication SEI message are proposed. Different from the flag array structure in the field indication SEI message in JCTVC-H1003-v22, the flags for field
applications are classified into two groups according to the sequence type information, which copes with features of field sequence and frame sequence, respectively.
The contribution has figures to illustrate several use cases.
This was closely related to I0393 and the NB comments identified as US66, US67, US68.
See notes in section on I0393.
JCTVC-I0393 How to use the field sequence SEI [C. Fogg (Harmonic), A. Wells (Ambarella)]
An alternative but functionally identical conditional tree syntax for the field indication SEI is proposed.
Examples of how to use the new field sequence SEI are also provided. In particular, the use of systems time stamps such as PTS can identify how to re-interleave progressive fields back into whole progressive frames when progressive content has been passed between field sequence equipment (PFS). It was proposed that the JCT provide liaison statements to relevant organizations such as SMPTE, SCTE, DVB, informing them of the field indication SEI proper use.
Some syntax elements can indicate progressive movies that have been converted for "pulldown" mode.
The proposed syntax indicates progressive/interlaced, top/bottom first, duplicate fields. A goal is simplify deinterlacing. It was asked what is the intended application and why the deinterlacing would not be done ahead of the HEVC encoding. It was commented that this could be used for transcoding from MPEG-2.
This was closely related to I0062 and US66, US67, US68.
Decision: Adopted. (G. Sullivan and C. Fogg agreed to assist in the editing.) The current VUI has just one presence flag.
Decision: Instead of presence flag in VUI, put field_sequence_flag in VUI and always allow SEI message to be sent, but require the SEI field sequence_flag to be equal to the VUI flag value. (And, an editorial topic, not use exactly the same syntax element name). When the flag is 1, field sequence SEI must be present in all pictures. (Optional when the flag is 0.) Note that this affects our response to US66, US67, US68.
JCTVC-I0502 On Chroma considerations for Interlaced material coding [J. Vieron, J.-M.
Thiesse (Ateme)] [late]
This contribution proposes a modification relating to the chroma phase issue when encoding 4:2:0 interlaced material in the current HEVC design. Specifically, the proposed method deals both with 1) the misalignment of chroma sample locations of top field with regard to bottom field and 2) the misalignment of chroma sample locations with regard to the so-called "nominal" sample locations of 4:2:0 content in progressive mode.
It is proposed to align both top and bottom fields chroma samples before encoding, to signal the chroma phase parameters and realign the chroma samples in a proper way at the decoding side.
This proposal reports further experiments based on the HM6.0. The evaluation has been made on different interlaced 4:2:0 sequences. Using the proposed method, it is reported that average gains of −6.3% and
−6.5% on respectively U and V chroma components were reported (with −0.1% impact on luma BD BR).
There is still a gain (at the tested QP values) if the decoder does not shift back the chroma position: about 1% in chroma.
It was asserted that the benefit was visible.
It was noted that JCTVC-G196 had proposed an AVC-like adjustment to the chroma motion compensation process.
It was proposed to indicate that a phase shift had been applied by the encoder.
It was questioned whether the decoder could be expected to actually try to shift the data back to
compensate for the shift. A participant indicated that this seemed unlikely, and that there doesn't seem to be value in sending the SEI indication if the decoder won't use it for anything.
In practice, it seems common for decoders to get the chroma handling wrong.
If the decoder ignores the message, the chroma could be jiggling a little bit.
It was remarked that spending extra bits on the chroma can help without doing this and without introducing the jiggling.
No action was taken on this.