Extension Formatting for the Opus Codec

Extension Format The Opus padding mechanism provides a safe way to extend the Opus codec while preserving interoperability and without having to transmit any extra packets. specifies that all padding bytes "MUST be set to zero" by the encoder, while the decoder "MUST accept any value for the padding bytes". In that way, any non-zero padding will indicate to an extended decoder that an extension is present and can be processed. On the other hand, for any all-zero padding, the decoder will just discard the padding like any non-extended decoder. A non-extended decoder receiving a packet with an extension will simply discard the extension and proceed as if none was present. An extension starts with a byte that signals a 7-bit ID, as well as a binary flag L for length signaling. For extension IDs 1 and 3 through 31, L=0 means that no data follows the extension, whereas L=1 means that exactly one byte of extension data follows. These are collectively called "short extensions". For IDs 32 to 127, L=0 signals that the extension data takes up the rest of the padding, and L=1 signals that a length indicator follows. These are collectively called "long extensions". For ID 0, L=0 has the same meaning as for long extensions, but L=1 signals a length of zero (no length indicator follows). In any given packet containing padding, the "rest of the padding" cannot appear more than once. For ID 2, the extension itself has no payload (for either L=0 or L=1), but it is used to signal that previously coded extensions are repeated in subsequent Opus frames. Their payloads follow immediately after (see ). When a long extension signals that it requires a length indicator, the following byte contains a length value from 0 to 254. If the length byte is 255, then the length is 255 plus the length signaled from the next byte, with the 255 case being allowed to repeat as long as the size of the padding is not exceeded. Any extension signaled with a length that would cause the decoder to read beyond the bounds of the packet MUST be ignored by the decoder.

Extension framing A decoder MUST ignore any extension it does not know, decoding the rest of the packet as if the extension was not present. Additionally, a decoder MAY ignore any other extension even if it technically supports it. An encoder MUST NOT alter the way it encodes the non-extension part of an Opus packet in such a way as to noticeably reduce its quality when decoded with a non-extended decoder. A given extension ID may appear multiple times and extension ID ordering within each Opus frame is significant (see, e.g., ). A particular extension ID definition MAY place further restrictions on count and ordering of these extensions (see ). Reordering of extensions between Opus frames caused by the repeat mechanism is not significant and an extended decoder MUST treat repeated extensions as equivalent to the same extensions coded individually (see ).

ID 0: Original Padding For compatibility reasons, an ID of 0 means that the content of the extension is actual padding, as originally defined in . As in its original definition, the padding bytes MUST be set to zero by the encoder, while the decoder MUST ignore any non-zero padding. In the case where the L flag is set, the 0x01 header byte is simply skipped and extension decoding continues from the next byte. This can be useful as a way to insert padding one byte at a time, since appending zeros at the end may cause an increase in size from having to signal a multi-byte length indicator for the last extension.

ID 1: Separator In the case where multiple Opus frames are packed inside the same packet, there may be a need to specify which extension(s) apply to which frame. By default, all extensions apply to the first Opus frame in the packet. An extension with ID=1 acts as a separator between extensions from different Opus frames. When parsing extensions sequentially, any time a separator with L=0 is encountered, the associated frame is increased by one. If L=1 is used, the following data byte indicates the increment applied for the new associated frame. The associated frame value MUST NOT exceed the bound equal to the number of frames in the packet, minus one (indexing starts at zero). Similarly, L=0 separators MUST NOT cause the associated frame to exceed the above bound. The decoder MUST ignore all extensions associated with an out-of-bound frame index.

ID 2: Repeat These Extensions (RTE) In the case where multiple Opus frames are packed inside the same packet, the overhead of coding the extension IDs and frame separators can be reduced by using the Repeat These Extensions (RTE) extension. An extension with ID=2 acts as a signal that all of the non-padding extensions seen so far in the current frame, starting after the last RTE extension (if any), are to be repeated in all subsequent frames. The extension itself has no payload, but it is immediately followed by the extension data for the repeated extensions. The payloads for all of the repeated extensions for the next frame come first, followed by those of the frame after, etc. The extension ID byte for each payload is not coded: only the length (if needed) and the extension data. An RTE extension MAY appear multiple times in the same frame. Only the extensions which follow the most recent RTE (if any) are repeated by it. An RTE extension MAY appear in the last frame. In this case, no extensions are repeated. For short extensions, the repeated extension payloads use the same L flag as the extension being repeated. If the length of a short extension needs to change between frames, this repeat mechanism cannot be used to signal that. All repeated long extension payloads except the final instance of the last repeated long extension in the last frame are coded as if with L=1 (using an explicit length indicator). The final repeated long extension payload is coded with the L flag specified by the RTE extension. In the case that the RTE extension specifies L=0, and the last repeated long extension is followed by one or more repeated short extensions with a payload, then the final long extension does not consume the rest of the padding as normal, but leaves enough room for the payloads of the repeated short extensions that follow. If there is not enough room for the repeated short extensions that follow, even if the length of the final long extension were set to zero, then that extension and all remaining extensions MUST be ignored by the decoder. If the RTE extension uses L=1, then extension coding continues afterwards for the same frame as the RTE extension appeared in. This allows a frame to contain both repeated and non-repeated extensions. This also means that the complete list of extensions for a given frame might not all be contiguous in the packet. If the RTE extension uses L=0, but the repeated extension data did not contain a long extension, then extension coding continues afterwards for the frame following the RTE extension (as if an L=0 separator had been coded). If an RTE extension with L=0 appears in the last frame, then the rest of the padding (if any) MUST be set to zero by the encoder, while the decoder MUST ignore any additional non-zero padding.

IDs 3-119: Unassigned These extensions are to be defined in their own respective documents and the IDs are to be assigned by IANA. The meaning of the L flag is already defined for all of these unassigned IDs because a decoder must know how to skip extensions it does not know about. Due to potential for interaction between extensions, new extensions are to be assigned with the "Standards Action" policy defined by .

IDs 120-126: I-D Experimental We reserve these 7 IDs for experimental extensions, such that extensions defined in Internet-Drafts can be tested before they become RFC without causing possible interoperability issues should their bitstream definitions change. When using an experimental ID, it is RECOMMENDED to use a two-byte prefix that attempts to encode an experiment number (first byte) and a version number (second byte). Experimental extension documents SHOULD attempt to choose an experiment number that does not collide with other ongoing experiments.

ID 127: Extended Extensions The last ID is reserved for future extensions to the extension mechanism. As with all other long extensions, the meaning of the L flag is pre-defined to ensure decoders can skip extended extensions they do not know about. The contents of the payload for this extension will be defined by a future specification.

IANA Considerations This document defines a new registry "Opus Extension IDs" in a new "Opus" group, that allocates individual IDs to individual extensions to be defined in the future. The existing "Opus Channel Mapping Families" registry will also be moved to the newly created "Opus" group. Moreover, this document already defines the following IDs:

Extension ID	Description	Reference
0	Original padding definition	Defined in
1	Frame separator	Defined in .
2	Repeat These Extensions	Defined in .
3-119	Unassigned	To be assigned with the "Standards Action" policy
120-126	Experimental Internet-Draft implementations	Defined in , following the "Experimental Use" policy
127	Reserved for future extensions	Reserved in

For forward compatibility, any extension MUST use the definition of the L flag dictated by its ID value (see ). Extension definitions MUST specify whether or not it is permitted for the extension to appear multiple times for a given Opus frame within the packet.

Opus Media Type Update This document updates the audio/opus media type registration to add the following two optional parameters: extensions: specifies a comma-separated list of supported extension IDs on the receiver side. sprop-extensions: specifies a comma-separated list of supported extension IDs on the sender side. extN-*: To facilitate parameter forwarding, extension document that require receiver extension parameters SHOULD name them "ext", followed by the extension ID, a hyphen, and the parameter name. sprop-extN-*: Extension-specific sender-side parameters defined similarly as above. All names starting with "ext" and "sprop-ext" are reserved for use by Opus extensions. Extension IDs 0, 1, and 2 MUST be supported by any receiver that recognizes Opus extensions, and do not need to be included in the extensions or sprop-extensions lists.

Mapping to SDP Parameters The media type parameters described above map to declarative SDP and SDP offer-answer in the same way as other optional parameters in . As per Section 6.3, media-level format parameters MUST be explicitly specified and MUST NOT be carried over blindly from another offer or answer. Regardless of any a=fmtp SDP attribute specified, the receiver MUST be capable of receiving any signal.