]> Packed CBOR Universität Bremen TZI
Postfach 330440 Bremen D-28359 Germany +49-421-218-63921 cabo@tzi.org
TUD Dresden University of Technology
Helmholtzstr. 10 Dresden D-01069 Germany mikolai.guetschow@tu-dresden.de
Internet-Draft The Concise Binary Object Representation (CBOR, RFC 8949 == STD 94) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. CBOR does not provide any forms of data compression. CBOR data items, in particular when generated from legacy data models, often allow considerable gains in compactness when applying data compression. While traditional data compression techniques such as DEFLATE (RFC 1951) can work well for CBOR encoded data items, their disadvantage is that the recipient needs to decompress the compressed form before it can make use of the data. This specification describes Packed CBOR, a set of CBOR tags and simple values that enable a simple transformation of an original CBOR data item into a Packed CBOR data item that is almost as easy to consume as the original CBOR data item. A separate decompression step is therefore often not required at the recipient. (This cref will be removed by the RFC editor:)
The present revision -19 is a work-in-progress release in preparation for another cbor-packed side meeting. This revision resolves the use of the tunables A/B/C by setting A=16, B=8, and C=8, and choosing requested simple values and tag numbers, in preparation for continuing the early allocation process. About This Document Status information for this document may be found at . Discussion of this document takes place on the CBOR Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .
Introduction The Concise Binary Object Representation (CBOR, ) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. CBOR does not provide any forms of data compression. CBOR data items, in particular when generated from legacy data models, often allow considerable gains in compactness when applying data compression. While traditional data compression techniques such as DEFLATE can work well for CBOR encoded data items, their disadvantage is that the recipient needs to decompress the compressed form before it can make use of the data. This specification describes Packed CBOR, a set of CBOR tags and simple values that enable a simple transformation of an original CBOR data item into a Packed CBOR data item that is almost as easy to consume as the original CBOR data item. A separate decompression step is therefore often not required at the recipient. This document defines the Packed CBOR format by specifying the transformation from a Packed CBOR data item to the original CBOR data item; it does not define an algorithm for a packer. Different packers can differ in the amount of effort they invest in arriving at a reduced-redundancy packed form; often, they simply employ the sharing that is natural for a specific application. Packed CBOR can make use of two kinds of optimization:
  • item sharing: substructures (data items) that occur repeatedly in the original CBOR data item can be collapsed to a simple reference to a common representation of that data item. The processing required during consumption is limited to following that reference (plus carrying out integration tags (), if these are in use).
  • argument sharing: application of a function with two arguments, one of which is shared. Data items (strings, containers) that share a prefix or suffix, or more generally data items that can be constructed from a function taking a shared argument and a rump data item, can be replaced by a reference to the shared argument plus a rump data item. For strings and the default "concatenation" function, the processing required during consumption is similar to following the argument reference plus that for an indefinite-length string.
A specific application protocol that employs Packed CBOR might employ both kinds of optimization or limit its use to item sharing only.
Extensibility Approach Packed CBOR is defined in two main parts:
  • Data items for referencing packing tables (), the set of which defined here which is intended to be the stable, common component of all uses of Packed CBOR, and
  • Mechanisms for setting up packing tables (), which carries the main extension point, populated in this document by two table setup tags. Such setup information is usually conveyed in a tag and then applies to the content of the tag. Setup information can also be contained in environmental information that applies to an encoded CBOR data item, e.g., a media type can set up a static dictionary that applies to CBOR data items in representations that are of that media type.
Sections , , and provide additional extension points, each of which is populated by one or more extensions in this document or elsewhere. These extensions can be selected by an application protocol that makes use of Packed CBOR. Beyond the extensibility approach shown in the present document, new CBOR tags (or media types etc.) could also be defined such that they (1) modify (or completely swap out) the way the referencing data items (simple values and tags) defined in this document operate and/or (2) define new referencing data items. (From the point of view of the present specification, these tags or media types then act as setup tags setting up tables that control subtrees with semantics different from the present specification; from the point of view of the specification defining these tags or media types this simply initiates the use of the referencing data items for their specific purposes.) An example for this is not shown in the present document so that there is a coherent interpretation of the referencing data items defined here; such new definitions of referencing data items probably should specify how they interact with parts of Packed CBOR that they do not replace. An unpacker can only carry out the tags (and the environmental information) that it knows how to interpret. An unpacker that encounters tags that are unknown to it can simply make these tags available to the application, which then can abort processing if unknown (or unimplemented) tags are found, or if their interpretation would require functionality of the unpacker that is not available. As a shortcut, the application might also provide the unpacker with a list of tags that the application can process, allowing the unpacker to abort processing when a tag unknown to it and not on this list is encountered.
Terminology and Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in () () when, and only when, they appear in all capitals, as shown here.
Original data item:
A CBOR data item that is intended to be expressed by a packed data item; the result of all reconstructions.
Packed data item:
A CBOR data item that involves packed references (packed CBOR).
Packed reference:
A shared item reference or an argument reference, expressed by a reference data item.
Reference data item:
A data item (tag or simple value) that serves as a packed reference.
Reference site:
The context of a reference data item.
Shared item reference:
A reference to a shared item as defined in .
Argument reference:
A reference that combines a shared argument with a rump item as defined in .
Rump:
The data item contained in an argument reference that is combined with the argument to yield the reconstruction.
Straight reference:
An argument reference that uses the argument as the left-hand side and the rump as the right-hand side.
Inverted reference:
An argument reference that uses the rump as the left-hand side and the argument as the right-hand side.
Function tag:
A tag used in an argument reference for the argument (straight references) or the rump (inverted references), causing the application of a function indicated by the function tag in order to reconstruct the data item.
Integration tag:
A tag defined by an application protocol to be used as a shared item table element in order to signal a non-default procedure to integrate the shared item into the reference site.
Stand-in item:
A data item (a tag or a simple value) defined by an application protocol to stand in for a more complex data item. Stand-in items are fundamentally independent of Packed CBOR but can be employed by the application protocol as part of a Packed CBOR argument reference.
Packing tables:
The pair of a shared item table and an argument table.
Active set (of packing tables):
The packing tables in effect at the data item under consideration.
Reconstruction:
The result of applying a packed reference in the context of given packing tables; we speak of the reconstruction of a packed reference as that result.
The definitions of apply. Specifically: The term "byte" is used in its now customary sense as a synonym for "octet"; "byte strings" are CBOR data items carrying a sequence of zero or more (binary) bytes, while "text strings" are CBOR data items carrying a sequence of zero or more Unicode code points (more precisely: Unicode scalar values), encoded in UTF-8 . In this specification, the term "argument" is not used in the specific sense assigned to it in Section of RFC 8949 , but in its general sense as an argument of a function. Where arithmetic is explained, this document uses the notation familiar from the programming language C, except that
  • ".." denotes a range that includes both ends given,
  • in the HTML and PDF forms, subtraction and negation are rendered as a hyphen ("-", as are various dashes), and
  • superscript notation denotes exponentiation. For example, 2 to the power of 64 is notated: 264. In the plain-text version of this specification, superscript notation is not available and therefore is rendered by a surrogate notation. That notation is not optimized for this RFC; it is unfortunately ambiguous with C's exclusive-or and requires circumspection from the reader of the plain-text version.
Examples of CBOR data items are shown in CBOR Extended Diagnostic Notation (Section of RFC 8949 in conjunction with ➔ possibly update to ).
Packed CBOR This section describes the packing tables, their structure, and how they are referenced.
Packing Tables At any point within a data item making use of Packed CBOR, there is an active set of packing tables that applies. There are two packing tables in an active set:
  • Shared item table
  • Argument table
Without any table setup, these two tables are empty arrays. Table setup can cause these arrays to be non-empty, where the elements are (potentially themselves packed) data items. Each of the tables is indexed by an unsigned integer (starting from 0). Such an index may be derived from information in tags and their content as well as from CBOR simple values. Table setup mechanisms (see ) may include all information needed for table setup within the packed CBOR data item, or they may refer to external information. This external information may be immutable, or it may be intended to potentially grow over time. In the latter case, the table setup mechanism needs to define how both backward and forward compatibility is addressed, e.g., how a reference to a new item should be handled when the unpacker uses an older version of the external information. If, during unpacking, an index is used that references an item that is unpopulated in (e.g., outside the size of) the table in use, this MAY be treated as an error by the unpacker and abort the unpacking. Alternatively, the unpacker MAY provide an implementation specific value, enclosed in the tag 1112, to the application and leave the error handling to the application. In the simplest case, this could be 1112(undefined), using the simple value >undefined< as per Section of RFC 8949 ; however, the same value cannot be used repeatedly as a map key within the same map. An unpacker SHOULD document which of these two alternatives has been chosen. CBOR based protocols that include the use of packed CBOR MAY require that unpacking errors are tolerated in some positions.
Referencing Shared Items Shared items are stored in the shared item table of the active set. The shared data items are referenced by using the reference data items in . The table index (an unsigned integer) is derived either from the simple value number or the (unsigned or negative) integer N provided as the content of tag 6. When reconstructing the original data item, such a reference is replaced by the referenced data item, which is then recursively unpacked. Referencing Shared Values
Reference Table Index
Simple value 0..15 0..15
Tag 6(N) (unsigned integer N ≥ 0) 16 + 2×N
Tag 6(N) (negative integer N < 0) 16 − 2×N − 1
As examples, the first 22 elements of the shared item table are referenced by simple(0), simple(1), ... simple(15), 6(0), 6(-1), 6(1), 6(-2), 6(2), 6(-3). (The alternation between unsigned and negative integers for even/odd table index values — "zigzag encoding" — makes systematic use of shorter integer encodings first.) Taking into account the encoding of these referring data items, there are 16 one-byte references, 48 two-byte references, 464 three-byte references, 130560 four-byte references, etc. As CBOR integers can grow to very large (or very negative) values, there is no practical limit to how many shared items might be used in a Packed CBOR item. Note that the semantics of Tag 6 depend on its tag content: An integer turns the tag into a shared item reference, whereas an array of an integer and a data item turns it into an argument reference (). All other forms of arguments for Tag 6 are reserved for future updates to the present specification. Note also that the tag content of Tag 6 may itself be packed, so it may need to be unpacked to make this determination.
Referencing Argument Items The argument table serves as a common table that can be used for argument references, i.e., for concatenation as well as references involving a function tag. When referencing an argument, a distinction is made between straight and inverted references; if no function tag is involved, a straight reference combines a prefix out of the argument table with the rump data item, and an inverted reference combines a rump data item with a suffix out of the argument table. Straight Referencing (e.g., Prefix) Arguments
Straight Reference Table Index
Tag 128..135(rump) 0..7
Tag 6([unsigned integer N, rump]) (N ≥ 0) 8 + N
Inverted Referencing (e.g., Suffix) Arguments
Inverted Reference Table Index
Tag 136..143(rump) 0..7
Tag 6([negative integer N, rump]) (N < 0) 8 - N - 1
Argument data items are referenced by using the reference data items in and . For the 16 tags 128 to 143 included, the table index (an unsigned integer) is derived from the tag number, together with the information if the reference is straight or inverted. For tag 6, the table index is derived from the integer N in the first element of the tag content (unsigned integer for straight, negative integer for inverted references). The "rump item" is the second element of the two-element array that is the tag content. When reconstructing the original data item, such a reference is replaced by a data item constructed from the argument data item found in the table (argument, which might need to be recursively unpacked first) and the rump data item (rump, again possibly needing to be recursively unpacked). Separate from the tag used as a reference, a tag ("function tag") may be involved to supply a function to be used in resolving the reference. It is crucial not to confuse reference tag and, if present, function tag. A straight reference uses the argument as the provisional left-hand side and the rump data item as the provisional right-hand side. An inverted reference uses the rump data item as the provisional left-hand side and the argument as the provisional right-hand side. In both cases, the provisional left-hand side is examined. If it is a tag ("function tag"), it is "unwrapped": The function tag's tag number is used to indicate the function to be applied, and the tag content (which, again, might need to be recursively unpacked) is kept as the unwrapped left-hand side. If the provisional left-hand side is not a tag, it is kept as the final left-hand side, and the function to be applied is concatenation, as defined below. The following procedure applies to the data items of both the provisional right-hand side and the unwrapped left-hand side (if applicable), independent of each other: If the data item is one of the explicitly allowed stand-in items (), the item that the stand-in item stands for is recursively unpacked. If the resulting unpacked data item is again an allowed stand-in item, the previous step is repeated. If the data item is neither a stand-in item, nor further unpackable, it is taken as the final right-hand or left-hand side, respectively. If a function tag was given, the reference is replaced by the result of applying the indicated unpacking function with the final left-hand side as its first argument and the final right-hand side as its second. The unpacking function is defined by the definition of the tag number supplied. If that definition does not define an unpacking function, the result of the unpacking is not valid. If no function tag was given, the reference is replaced by the final left-hand side "concatenated" with the final right-hand side, where concatenation is defined as in . As a contrived (but short) example, if the argument table is ["foobar", h'666f6f62', "fo"], each of the following straight (prefix) references will unpack to "foobart": 128("t"), 129("art"), 130("obart") (the byte string h'666f6f62' == 'foob' is concatenated into a text string, and the last example is not an optimization). Taking into account the encoding, there are 8 two-byte references, 24 three-byte references, 224 four-byte references, 65280 five-byte references, etc. The numbers for inverted (suffix) references are the same. (As CBOR integers can grow to very large (or very negative) values, there is no practical limit to how many argument items might be used in a Packed CBOR item.)
Concatenation The concatenation function is defined as follows:
  • If both left-hand side and right-hand side are arrays, the result of the concatenation is an array with all elements of the left-hand-side array followed by the elements of the right-hand side array.
  • If both left-hand side and right-hand side are maps, the result of the concatenation is a map that is initialized with a copy of the left-hand-side map, and then filled in with the members of the right-hand side map, replacing any existing members that have the same key. In order to be able to remove a map entry from the left-hand-side map, as a special case, any members to be replaced with a value of undefined (0xf7) from the right-hand-side map are instead removed, and right-hand-side members with the value undefined are never filled in into the concatenated map.
  • If both left-hand side and right-hand side are one of the string types (not necessarily the same), the bytes of the left-hand side are concatenated with the bytes of the right-hand side. Byte strings concatenated with text strings need to contain valid UTF-8 data. The result of the concatenation gets the type of the unwrapped rump data item; this way a single argument table entry can be used to build both byte and text strings, depending on what type of rump is being used.
  • If one side is one of the string types, and the other side is an array, the result of the concatenation is equivalent to the application of the "join" function () to the string as the left-hand side and the array as the right-hand side. The original right-hand side of the concatenation determines the string type of the result.
  • Other type combinations of left-hand side and right-hand side are not valid.
Discussion This specification uses up a number of Simple Values and Tags, in particular one of the rare one-byte tags and a good chunk of the one-byte simple values. Since the objective is reduced bulk, this is warranted only based on a consensus that this specific format could be useful for a wide area of applications, while maintaining reasonable simplicity in particular at the side of the consumer. Instead of evolving the set of reference data items, this specification derives its evolvability from treating the table setup mechanism as an extension point, which can in effect provide evolved semantics to the reference data items as they reference the table. A maliciously crafted Packed CBOR data item might contain a reference loop. A consumer/unpacker MUST protect against that.
Allocation These specification parameters allow the current specification to be precise while the quantitative allocation discussion is ongoing. They will be replaced by specific chosen numbers when the present specification is finalized. The sense of the WG has been to be more conservative in allocating CBOR resources to Packed CBOR than previous drafts of this document were. In addition, provides a liberal way to use the allocations for reference data items in packed items that use different unpacking mechanisms from the ones described in this document. 16 1+0 simple values are allocated to shared item references. During early development of CBOR, when the bit allocation and thus the ranges of simple values were originally defined, a range of 16 allocations was kept aside for item sharing. The allocations for 1+0 simple values were therefore performed from the top of the range down, i.e., with the block of false/true/null/undefined being originally assigned to 24..27 (after the introduction of indefinite length encoding, 20..23). No further allocation has been performed in this range in the 12 years since. 8 1+1 tags are allocated to straight argument references, and 8 further 1+1 tags allocated to inverted argument references. A single 1+0 tag is allocated to data item references beyond the above, with the tag content deciding which of the above reference mechanisms is to be used. Note the nature of Packed CBOR means that all these allocations can be used for pretty much unlimited purposes by simply defining another table setup mechanism (media type or table-building tag).
Table Setup The reference data items described in assume that packing tables have been set up. By default, both tables are empty (zero-length arrays). Table setup can happen in one of two ways:
  • By the application environment, e.g., a media type. These can define tables that amount to a static dictionary that can be used in a CBOR data item for this application environment. Note that, without this information, a data item that uses such a static dictionary can be decoded at the CBOR level, but not fully unpacked. The table setup mechanisms provided by this document are defined in such a way that an unpacker can at least recognize if this is the case.
  • By one or more table-building tags enclosing the packed content. Each tag is usually defined to build an augmented table by adding to the packing tables that already apply to the tag, and to apply the resulting augmented table when unpacking the tag content. Usually, the semantics of the tag will be to prepend items to one or more of the tables. (The specific behavior of any such tag, in the presence of a table applying to it, needs to be carefully specified.) Note that it may be useful to leave a particular efficiency tier alone and only prepend to a higher tier; e.g., a tag could insert shared items at table index 16 and shift anything that was already there further along in the array while leaving index 0 to 15 alone. Explicit additions by tag can combine with application-environment supplied tables that apply to the entire CBOR data item. Reference data items in the newly constructed (low-numbered) parts of the table are usually interpreted in the number space of that table (which includes the, now higher-numbered, inherited parts), while reference data items in any existing, inherited (higher-numbered) part continue to use the (more limited) number space of the inherited table.
Where external information is used in a table setup mechanism that is not immutable, care needs to be taken so that, over time, references to existing table entries stay valid (i.e., the information is only extended), and that a maximum size of this information is given. This allows an unpacker to recognize references to items that are not yet defined in the version of the external reference that it uses, providing backward and possibly limited (degraded) forward compatibility. For table setup, the present specification only defines two simple table-building tags, which operate by prepending to the (by default empty) tables.
Basic Packed CBOR Two tags are predefined by this specification for packing table setup. They are defined in CDDL as in , assuming the allocation of tag numbers 113 ('q') and 1113 for these tags:
CDDL for Packed CBOR Table Setup Tags Defined in this Document
These tags extend the two tables for shared items and for arguments that apply to the entire tag, which, unless an enclosing table setup tag or a table-setting application environment (e.g., a media type) applies, are empty tables:
Tag 113 ("Basic-Packed-CBOR"):
The array given as the first element of the tag content is prepended to both the tables for shared items and for arguments.
Tag 1113 ("Split-Basic-Packed-CBOR"):
The arrays given as the first and second element of the tag content are prepended individually to the tables for shared items and for arguments, respectively.
As discussed in the introduction to this section, references in the supplied new arrays use the new number space (where inherited items are shifted by the new items given), while the inherited items themselves use the inherited number space (so their semantics do not change by the mere action of inheritance). The original CBOR data item can be reconstructed by recursively replacing shared item and argument references encountered in the rump by their reconstructions.
Function Tags Function tags that occur in an argument or a rump supply the semantics for reconstructing a data item from their tag content and the non-dominating rump or argument, respectively. The present specification defines three function tags.
Join Function Tags Tag 106 ('j') defines the "join" unpacking function, based on the concatenation function (). The join function expects an item that can be concatenated as its left-hand side, and an array of such items as its right-hand side. Joining works by sequentially applying the concatenation function to the elements of the right-hand-side array, interspersing the left-hand side as the "joiner". An example in functional notation: join(", ", ["a", "b", "c"]) returns "a, b, c". For a right-hand side of one or more elements, the first element determines the type of the result when text strings and byte strings are mixed in the argument. For a right-hand side of one element, the joiner is not used, and that element returned. For a right-hand side of zero elements, a neutral element is generated based on the type of the joiner (empty text/byte string for a text/byte string, empty array for an array, empty map for a map). For an example, we assume this unpacked data item: A packed form of this using straight references could be: Tag 105 ('i') defines the "ijoin" unpacking function, which is exactly like that of tag 106, except that the left-hand side and right-hand side are interchanged ('i'). A packed form of the first example using inverted references and the ijoin tag could be: A packed form of an array with many URIs that reference SenML items from the same place could be: Note that for these examples, the implicit join semantics for mixed string-array concatenation as defined in actually obviate the need for an explicit join/ijoin tag; the examples do serve to demonstrate the explicit usage of the tag.
Record Function Tag Tag 114 ('r') defines the "record" function, which combines an array of keys with an array of values into a map. The record function expects an array as its left-hand side, whose items are treated as key items for the resulting map, and an array of equal or shorter length as its right-hand side, whose items are treated as value items for the resulting map. The map is constructed by grouping key and value items with equal position in the provided arrays into pairs that constitute the resulting map. The value item array MUST NOT be longer than the key item array. The value item array MAY be shorter than the key item array, in which case the one or more unmatched value items towards the end are treated as absent. Additionally, value items that are the CBOR simple value undefined (simple(23), encoding 0xf7) are also treated as absent. Key items whose matching value items are absent are not included in the resulting map. For an example, we assume this unpacked data item: A straightforward packed form of this using the record function tag could be: A slightly more concise packed form can be achieved by manipulating the key item order (recall that the order of key/value pairs in maps carries no semantics):
Integration Tags Integration tags fulfill a similar purpose for shared item references as function tags do for argument references. An integration tag can be used as an element of a shared item table, supplying extended semantics on how to integrate its tag content into the context from which the shared item is referenced. A regular shared item reference can be used to reference an integration tag. (Note that the generation of an integration tag can in turn be automatic in the table setup mechanism specified by an application environment () or a table setup tag, so the integration tag may never actually physically occur in the interchanged data.) Application protocol specifications need to be explicit about which integration tags are in use; otherwise, the unpacker will not know whether a tag in a shared item table position is an integration tag or is intended to be shared literally. (The set of integration tags in use can also be defined as part of the table setup mechanism.) The present specification defines one integration tag.
Splicing Integration Tag Tag 1115, the splicing integration tag, can be used with a tag content that is an array. It specifies that the tag content is "spliced" into the surrounding array of a reference item referencing that shared item, i.e. the surrounding array is replaced by one that enumerates the elements of the shared item at the site of the shared item reference. Example: a rump of [1, 2, 3, simple(0), 7, 8, 9], where the shared item table contains 1115([4, 5, 6]) as its first item is unpacked as [1, 2, 3, 4, 5, 6, 7, 8, 9]. Example application: Splicing integration tags could be generated implicitly in the implicit table setup defined in , removing the need to allow nested arrays for names.
Additional Stand-in Items Application specifications that employ Packed CBOR may also enable the use of additional "stand-in" items (tags or simple values) beyond the reference items defined by Packed CBOR. These are data items used in place of original representation items such as strings or arrays, where the tag or simple value is defined to stand for a data item that can actually be used in the position of the stand-in item. Examples would be tags such as 21 to 23 (base64url, base64, uppercase hex: Section of RFC 8949 ) or 108 (lowercase hex: ), which stand for text string items but internally employ more compact byte string representations that may also be more natural as application data items. These additional stand-in items are fundamentally independent of Packed CBOR, but they also can be used as the right-hand-side of reference items (see ). Note that application protocol specifications need to be explicit about which stand-in items are provided for; otherwise, inconsistent interpretations at different places in a system can lead to check/use vulnerabilities.
Tag Validity: Equivalence Principle In Section of RFC 8949 , the validity of tags is defined in terms of type and value of their tag content. The CBOR Tag registry ( as defined in Section of RFC 8949 ) allows recording the "data item" for a registered tag, which is usually an abbreviated description of the top-level data type allowed for the tag content. In other words, in the registry, the validity of a tag of a given tag number is described in terms of the top-level structure of the data carried in the tag content. The description of a tag might add further constraints for the data item. But in any case, a tag definition can only specify validity based on the structure of its tag content. In Packed CBOR, a reference data item (represented as a tag or a simple value) might be "standing in" for the actual tag content of an outer tag, or for a structural component of that. In this case, the formal structure of the outer tag's content before unpacking usually no longer fulfills the validity conditions of the outer tag. The underlying problem is not unique to Packed CBOR. For instance, describes tags 64..87 that "stand in" for CBOR arrays (the native form of which has major type 4). For the other tags defined , which require some array structure of the tag content, a footnote was added:
[...] The second element of the outer array in the data item is a native CBOR array (major type 4) or Typed Array (one of tag 64..87)
The top-down approach to handle the "rendezvous" between the outer tag and the tag content representation (e.g., using an inner tag) does not support extensibility: any further Typed Array tags being defined do not inherit the exception granted to tag number 64..87; they would need to formally update all existing tag definitions that can accept typed arrays or be of limited use with these existing tags. Instead, the tag validity mechanism needs to be extended by a bottom-up component: A tag (or simple value) definition needs to be able to declare that the tag can "stand in" for, (is, in terms of tag validity, equivalent to) some structure. E.g., tag 64..87 could have declared their equivalence to the CBOR major type 4 arrays they stand in for.
Tag Content Equivalence A tag or simple value definition MAY declare Tag Content Equivalence to some existing structure, under some conditions defined by that definition. This, in effect, extends all existing tag definitions that accept the named structure to accept the newly defined item under the conditions given for the Tag Content Equivalence. A number of limitations apply to Tag Content Equivalence, which therefore should be applied deliberately and sparingly:
  • Tag Content Equivalence is a new concept, which may not be implemented by an existing generic decoder. A generic decoder not implementing tag equivalence might raise tag validity errors where Tag Content Equivalence says there should be none.
  • A CBOR protocol MAY specify the use of Tag Content Equivalence, effectively limiting the protocol's full use to those generic encoders that implement it. Existing CBOR protocols that do not address Tag Content Equivalence implicitly have a new variant that allows Tag Content Equivalence (e.g., to support Packed CBOR with an existing protocol). A CBOR protocol that does address Tag Content Equivalence MAY be explicit about what kinds of Tag Content Equivalence it supports (e.g., only the reference tags employed by Packed CBOR and certain table setup tags).
  • There is currently no way to express Tag Content Equivalence in CDDL. For Packed CBOR, CDDL would typically be used to describe the unpacked CBOR represented by it; further restricting the Packed CBOR is likely to lead to interoperability problems. (Note that, by definition, there is no need to describe Tag Equivalence on the receptacle [outer tag] side; only for the item that declares Tag Content Equivalence.)
  • The registry "" currently does not have a way to record any equivalence claimed for a tag. A convention would be to alert to Tag Content Equivalence in the "Semantics (short form)" field of the registry.Needs to be done for the tag registrations here.
Tag Content Equivalence of Tags and Simple Values Defined in Packed CBOR The reference data items (tags and simple values) in this specification declare their equivalence to the unpacked shared items or function results they represent. The table setup tags 113 and 1113 declare their equivalence to the unpacked CBOR data item represented by them.
IANA Considerations RFC Editor: please replace RFCXXXX with the RFC number of this RFC and remove this note. For all assignments described in this section, the "reference" column is the present document, i.e., RFCXXXX.
CBOR Tags Registry In the registry "" , IANA is requested to allocate the tags defined in . Values for Tag Numbers
Tag Data Item Semantics
6 int (for shared); [int, any] (for argument) Reference Data Item (for Packed CBOR: shared/argument)
105 concatenation item (text string, byte string, array, or map) Packed CBOR: ijoin function
106 array of concatenation item (text string, byte string, array, or map) Packed CBOR: join function
113 array (shared-and-argument-items, rump) Packed CBOR: table setup
114 array Packed CBOR: record function
128..135 any Reference Data Item (for Packed CBOR: straight argument)
136..143 function tag or concatenation item (text string, byte string, array, or map) Reference Data Item (for Packed CBOR: inverted argument)
1112 any Packed CBOR: reference error
1113 array (shared-items, argument-items, rump) Packed CBOR: table setup
1115 any Packed CBOR: splicing integration tag
CBOR Simple Values Registry In the registry "" , IANA is requested to allocate the simple values defined in . Simple Values
Value Semantics
0..15 Reference Data Item (for Packed CBOR: shared)
Security Considerations The security considerations of apply. Loops in the Packed CBOR can be used as a denial of service attack unless mitigated, see . As the unpacking is deterministic, packed forms can be used as signing inputs when deterministically encoded . (Note that where external dictionaries are added to cbor-packed as in , this requires additional consideration.) When tables are obtained from the application environment, e.g., a media type, any evolution of the application environment (such as an update to the media type specification) needs to reliably ensure that existing references continue to unpack in the same way. Therefore, application environments that provide packing tables need to explicitly specify if these packing tables may evolve, and, if yes, provide a design for this kind of evolvability. For instance, provides a way to reserve entries in a packing table that can be filled in by revisions of the application environment; to avoid false unpacking, this needs to be the only update that can be applied to such a table-setting application environment.
References Normative References Concise Binary Object Representation (CBOR) The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. These design goals make it different from earlier binary serializations such as ASN.1 and MessagePack. This document obsoletes RFC 7049, providing editorial improvements, new details, and errata fixes while keeping full compatibility with the interchange format of RFC 7049. It does not create a new version of the format. Concise Binary Object Representation (CBOR) Tags IANA Concise Binary Object Representation (CBOR) Simple Values IANA Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures This document proposes a notational convention to express Concise Binary Object Representation (CBOR) data structures (RFC 7049). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON. Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. CBOR Extended Diagnostic Notation (EDN) Universität Bremen TZI This document formalizes and consolidates the definition of the Extended Diagnostic Notation (EDN) of the Concise Binary Object Representation (CBOR), addressing implementer experience. Replacing EDN's previous informal descriptions, it updates RFC 8949, obsoleting its Section 8, and RFC 8610, obsoleting its Appendix G. It also specifies and uses registry-based extension points, using one to support text representations of epoch-based dates/times and of IP addresses and prefixes. // (This cref will be removed by the RFC editor:) The present -19 // includes the definition of the cri'' application- extension. cri'' // was previously defined in draft-ietf-core-href; however the latter // document overtook the present document in the approval process. // As the definition of cri'' is dependent on the present document // (and conversely has essentially no dependency on the technical // content of draft-ietf-core-href beyond its mere existence), the // text (including IANA considerations) has been moved here. -19 is // intended for use at the CBOR WG meeting at IETF 124. Informative References UTF-8, a transformation format of ISO 10646 ISO/IEC 10646-1 defines a large character set called the Universal Character Set (UCS) which encompasses most of the world's writing systems. The originally proposed encodings of the UCS, however, were not compatible with many current applications and protocols, and this has led to the development of UTF-8, the object of this memo. UTF-8 has the characteristic of preserving the full US-ASCII range, providing compatibility with file systems, parsers and other software that rely on US-ASCII values but are transparent to other values. This memo obsoletes and replaces RFC 2279. Concise Binary Object Representation (CBOR) The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. These design goals make it different from earlier binary serializations such as ASN.1 and MessagePack. JSON Merge Patch This specification defines the JSON merge patch format and processing rules. The merge patch format is primarily intended for use with the HTTP PATCH method as a means of describing a set of modifications to a target resource's content. Concise Binary Object Representation (CBOR) Sequences This document describes the Concise Binary Object Representation (CBOR) Sequence format and associated media type "application/cbor-seq". A CBOR Sequence consists of any number of encoded CBOR data items, simply concatenated in sequence. Structured syntax suffixes for media types allow other media types to build on them and make it explicit that they are built on an existing media type as their foundation. This specification defines and registers "+cbor-seq" as a structured syntax suffix for CBOR Sequences. Naming Things with Hashes This document defines a set of ways to identify a thing (a digital object in this case) using the output from a hash function. It specifies a new URI scheme for this purpose, a way to map these to HTTP URLs, and binary and human-speakable formats for these names. The various formats are designed to support, but not require, a strong link to the referenced object, such that the referenced object may be authenticated to the same degree as the reference to it. The reason for this work is to standardise current uses of hash outputs in URLs and to support new information-centric applications and other uses of hash outputs in protocols. DEFLATE Compressed Data Format Specification version 1.3 This specification defines a lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding, with efficiency comparable to the best currently available general-purpose compression methods. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Concise Binary Object Representation (CBOR) Tags for Typed Arrays The Concise Binary Object Representation (CBOR), as defined in RFC 7049, is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. This document makes use of this extensibility to define a number of CBOR tags for typed arrays of numeric data, as well as additional tags for multi-dimensional and homogeneous arrays. It is intended as the reference document for the IANA registration of the CBOR tags defined. CBOR Common Deterministic Encoding (CDE) Universität Bremen TZI CBOR (STD 94, RFC 8949) defines the concept of "Deterministically Encoded CBOR" in its Section 4.2, determining one specific way to encode each particular CBOR value. This definition is instantiated by "core requirements", providing some flexibility for application specific decisions; this makes it harder than necessary to offer Deterministic Encoding as a selectable feature of generic CBOR encoders. The present specification documents the Best Current Practice for CBOR _Common Deterministic Encoding_ (CDE), which can be shared by a large set of applications with potentially diverging detailed application requirements. The document also discusses the desire for partial implementations, which can be another reason for constraining CBOR encoders, and singles out the encoding constraint "definite-length-only" as a likely constraint to be used in application protocol and media type definitions. This specification updates RFC 8949 in that it provides clarifications and definitions of additional terms as well as more examples and explanatory text; it does not make technical changes to RFC 8949. // This revision -13 merges all active pull requests in preparation // for the 2025-cbor-17 interim on 2025-10-15. Notable CBOR Tags Universität Bremen TZI The Concise Binary Object Representation (CBOR, RFC 8949) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. In CBOR, one point of extensibility is the definition of CBOR tags. RFC 8949's original edition, RFC 7049, defined a basic set of 16 tags as well as a registry that can be used to contribute additional tag definitions [IANA.cbor-tags]. Since RFC 7049 was published, at the time of writing some 250 definitions of tags and ranges of tags have been added to that registry. The present document provides a roadmap to a large subset of these tag definitions. Where applicable, it points to an IETF standards or standard development document that specifies the tag. Where no such document exists, the intention is to collect specification information from the sources of the registrations. After some more development, the present document is intended to be useful as a reference document for the IANA registrations of the CBOR tags the definitions of which have been collected. A Concise Binary Object Representation (CBOR) of DNS Messages TUD Dresden University of Technology Universität Bremen TZI HAW Hamburg TUD Dresden University of Technology & Barkhausen Institut This document specifies a compact data format of DNS messages using the Concise Binary Object Representation [RFC8949]. The primary purpose is to keep DNS messages small in constrained networks. Packed CBOR: Table set up by reference Packed CBOR is a mechanism for transforming Concise Binary Object Representation (CBOR) data into a more compact form. This document introduces a means for setting up its tables by means of dereferenceable identifiers, and introduces a pattern of using it without sending long identifiers. Arbitrary-Exponent Numbers Specification for Registration of CBOR Tags 264 and 265
Examples The (JSON-compatible) CBOR data structure depicted in , 400 bytes of binary CBOR, could be packed into the CBOR data item depicted in , 308 bytes, only employing item sharing. With support for argument sharing and the record function tag 114, the data item can be packed into 298 bytes as depicted in . Note that this particular example does not lend itself to prefix compression, so it uses the simple common-table setup form (tag 113).
Example original CBOR data item, 400 bytes
Example packed CBOR data item with item sharing only, 308 bytes
Example packed CBOR data item using item sharing and the record function tag, 302 bytes
The (JSON-compatible) CBOR data structure below has been packed with shared item and (partial) prefix compression only and employs the split-table setup form (tag 1113).
Example original CBOR data item, 1210 bytes
Example packed CBOR data item, 507 bytes
List of Figures
:
:
:
:
:
:
List of Tables
:
:
:
:
:
Acknowledgements CBOR packing was part of the original proposal that turned into CBOR, but did not make it into , the predecessor of RFC 8949 . Various attempts to come up with a specification over the years did not proceed. In 2017, proposed investigating compact representations of W3C Thing Descriptions, which prompted the author to come up with what turned into the present design. This work was supported in part by the German Federal Ministry of Education and Research (BMBF) within the project Concrete Contracts.