5.6 Functionalities
5.6.1 Scalable coding hooks
5.6.1.1 NAL Unit Header and high layer parameter set related
A summary of contributions in this area was provided as an attachment in a revision of the AHG12 report:
Doc NAL ref flag NAL unit header Higher Layer Parameter Sets
I0132 From perspective of v1 decoder,
temporal_id is in a different place in the NUH.
8 bits in NUH allocated as: 2-3 bits for scalability extension type (SET), 5-6 for layer_id with bit allocation defined by SET.
In extended future specification, the temporal_id bits might be used for somehting else. Also use SEI message to provide mapping of layer _id to view_id, temporal_id, dependency_id, etc.
VPS not needed
I0217 Remove temporal_id in NUH. 8
bits in NUH allocated: 2 bits scenario identifier, 6 bits to provide 1-3 scalable identifiers in NUH, according to pre-defined table mapping scenario identifier to bit allocations
In VPS: signal scalability dimensions, profile&level, optionally send profile&
level for all operation points.
I0230 Use reserved 5 bits for layer_id SPS points to VPS.
In VPS: mapping of layer_id to
dependency_id, view_id, etc., temporal scalability params, and inter-layer dependency info.
I0571 Similar to I0230.
In VPS: max temporal layers, profile &
level per temporal layer, and inter-layer dependencies, image resolution
parameters per layer, optional additional sub-bitstream profiles & levels.
I0524 Use reserved 5 bits for layer_id Similar to I0230.
vps_id in slice header of IDR and CRA slices. In VPS: mapping of layer_id to dependency_id, view_id, etc., profile &
level for bitstream subsets, max temporal layers, dependencies of component sequences.
Base spec decoder may not need to use the VPS.
Use reserved 5 bits for priority_id
I0252 Proposes a "basic parameter set" (like
VPS) which SPS refers to, contains profile & level, image resolution, bit depth, temporal scalability params.
Signal bps_id with AUD or SEI message.
I0253 Assumes I0252 BPS definition. In
extension, also in BPS: view_id_flag, dependency_id_flag, etc., and inter-layer dependency info.
I0262 Use one of reserved bits as a flag for extensions, and conditionally use a second reserved bit.
Optionally extend length of NUH by additional 3-4 bytes to send view_id, dependency_id,
Use reserved 5 bits for layer_id.
Add NAL unit types for depth (IDR, CRA, TLA, other) sends own profile & level, inherits all other params from reference SPS which is referenced using view_ref_layer.
Inheritance is resolved at activation time.
I0536 (no base
Use reserved 5 bits for layer_id Assumes SPS prediction. Each view or layer has own SPS. In extension in SPS:
mapping of layer_id to dependency_id,
spec impact)
view_id, etc., inter-layer dependency.
Discussion of VPS vs. SPS prediction tradeoffs.
In the discussion, it was noted that start code emulation possibilities need to be considered.
The potential parsing dependency between the proposed VPS and SPS was mentioned as a possible issue.
It was remarked that, with regard to VPS, we should consider the need for random access at locations other than CRA and IDR.
It was remarked that it is generally better to try to avoid using profile_idc values in a way that makes the parsing or decoding process directly dependent on them.
It was agreed that, at this time, we will not make changes to the NUH syntax.
Immediate questions:
Modification to NUH
VPS w.r.t. "base" specification
Note that the base specification needs temporal scalability, and this should be designed in anticipation of how it would fit into an extended scheme.
It was remarked that an SEI message (or something like it) can be used to carry descriptive metadata.
Currently the SPS has a loop on temporal layers for max_dec_pic_buffering, num_reorder_pics, max_latency_increase.
Currently we don't have a way to indicate a different profile, level or different HRD parameters, for different temporal layers.
A VPS would contain some of that.
In principle, the VPS could be metadata-only, from the perspective of the base specification.
Regarding where to put a VPS ID – these possibilities were discussed:
SPS
Redundantly, in the SH for IDR & CRA (which could have a recovery point SEI message issue), as a way to avoid the need to trace indirect references to determine the VPS ID
Redundantly, in AUD (currently optional, and loss-fragile) or SEI Note that JCTVC-I0338 is relevant, and includes an additional "GPS" concept.
The potential need for being able to provide an enhancement layer for an already-encoded base layer was requested to be considered. SVC did not support this. "SPS inheritance" is an alternative scheme that could "build up" the information from the bottom up. However, it was suggested that it is probably possible to solve this issue within the VPS concept and that this scenario should not be considered an obstacle to the VPS concept consideration. A suggestion was to put a version number in the VPS.
Suggested minimum content for the VPS:
max temporal layers (and/or max layers of some sort)
temporal scalability information (temporal_id_nesting_flag, max_dec_pic_buffering, max_latency_increase, max_num_reorder_pics)
profile & level per temporal layer (there was some disagreement over this)
extension payload(s)
One issue to consider is whether the VPS is to be an indication of what is actually in the bitstream or is a description of the maximum of what could be in the bitstream.
The ability to adapt the bitstream without needing to send an IDR was suggested to be needed. The use of the VPS as a description of maximum possible bitstream content was suggested to be friendly to this scenario.
In SVC and MVC we have SEI messages defined to indicate when some information is not present in the bitstream. Something like this can be a way to enable a description of the actual content of the bitstream.
A suggestion was to put ue(v) into the SPS after sps_id called something like likely_to_become_vps_id.
Another suggestion: put 16 bit FLC reserved there.
Another suggestion: put extra_data_length as ue(v) followed by that number of bits of data.
This was further discussed in plenary on Sunday 6 May.
Decision: Adopt VPS syntax per I0230-v3 section 2.1. Do not remove syntax elements from SPS that are duplicated in the VPS syntax. Do not use the same syntax element names in both places. Constrain them to match. (The base spec decoder can ignore the VPS.) Put video_parameter_set_id in the SPS, coded as ue(v), after seq_parameter_set_id. Range of vps_id is 0 to 15, inclusive. The encoder shall send it.
Activation language specified. Details per (base spec part of) I0230-v3.