Encodes and decodes types from/into [[ RCF 8949; https://www.rfc-editor.org/rfc/rfc8949.html ]] compatible `CBOR` binary. Also provided are conversion to and from JSON and the CBOR diagnostic format. **Allocations:** In general, when in the following table it says allocations are done on the `temp_allocator`, these allocations are still attempted to be deallocated. This allows you to use an allocator with freeing implemented as the `temp_allocator` which is handy with big CBOR. - *Encoding*: If the `.Deterministic_Map_Sorting` flag is set on the encoder, this allocates on the given `temp_allocator` some space for the keys of maps in order to sort them and then write them. Other than that there are no allocations (only for the final bytes if you use `cbor.encode_into_bytes`. - *Decoding*: Allocates everything on the given allocator and input given can be deleted after decoding. *No* temporary allocations are done. - *Marshal*: Same allocation strategy as encoding. - *Unmarshal*: Allocates everything on the given allocator and input given can be deleted after unmarshalling. Some temporary allocations are done on the given `temp_allocator`. **Determinism:** CBOR defines a deterministic en/decoder, which among other things uses the smallest type possible for integers and floats, and sorts map keys by their (encoded) lexical bytewise order. You can enable this behaviour using a combination of flags, also available as the `cbor.ENCODE_FULLY_DETERMINISTIC` constant. If you just want the small size that comes with this, but not the map sorting (which has a performance cost) you can use the `cbor.ENCODE_SMALL` constant for the flags. A deterministic float is a float in the smallest type (f16, f32, f64) that hasn't changed after conversion. A deterministic integer is an integer in the smallest representation (u8, u16, u32, u64) it fits in. **Untrusted Input:** By default input is treated as untrusted, this means the sizes that are encoded in the CBOR are not blindly trusted. If you were to trust these sizes, and allocate space for them an attacker would be able to cause massive allocations with small payloads. The decoder has a `max_pre_alloc` field that specifies the maximum amount of bytes (roughly) to pre allocate, a KiB by default. This does mean reallocations are more common though, you can, if you know the input is trusted, add the `.Trusted_Input` flag to the decoder. **Tags:** CBOR describes tags that you can wrap values with to assign a number to describe what type of data will follow. More information and a list of default tags can be found here: [[RFC 8949 Section 3.4;https://www.rfc-editor.org/rfc/rfc8949.html#name-tagging-of-items]]. A list of registered extension types can be found here: [[IANA CBOR assignments;https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml]]. Tags can either be assigned to a distinct Odin type (used by default), or be used with struct tags (`cbor_tag:"base64"`, or `cbor_tag:"1"` for example). By default, the following tags are supported/provided by this implementation: - *1/epoch*: Assign this tag to `time.Time` or integer fields to use the defined seconds since epoch format. - *24/cbor*: Assign this tag to string or byte fields to store encoded CBOR (not decoding it). - *34/base64*: Assign this tag to string or byte fields to store and decode the contents in base64. - *2 & 3*: Used automatically by the implementation to encode and decode big numbers into/from `core:math/big`. - *55799*: Self described CBOR, used when `.Self_Described_CBOR` flag is used to wrap the entire binary. This shows other implementations that we are dealing with CBOR by just looking at the first byte of input. - *1010*: An extension tag that defines a string type followed by its value, this is used by this implementation to support Odin's unions. Users can provide their own tag implementations using the `cbor.tag_register_type(...)` to register a tag for a distinct Odin type used automatically when it is encountered during marshal and unmarshal. Or with `cbor.tag_register_number(...)` to register a tag number along with an identifier for convenience that can be used with struct tags, e.g. `cbor_tag:"69"` or `cbor_tag:"my_tag"`. You can look at the default tags provided for pointers on how these implementations work. Example: package main import "base:intrinsics" import "core:encoding/cbor" import "core:fmt" import "core:reflect" import "core:time" Possibilities :: union { string, int, } Data :: struct { str: string, neg: cbor.Negative_U16, // Store a CBOR value directly. now: time.Time `cbor_tag:"epoch"`, // Wrapped in the epoch tag. ignore_this: ^Data `cbor:"-"`, // Ignored by implementation. renamed: f32 `cbor:"renamed :)"`, // Renamed when encoded. my_union: Possibilities, // Union support. my_raw: [8]u32 `cbor_tag:"raw"`, // Custom tag that just writes the value as bytes. } main :: proc() { // Example custom tag implementation that instead of breaking down all parts, // just writes the value as a big byte blob. This is an advanced feature but very powerful. RAW_TAG_NR :: 200 cbor.tag_register_number({ marshal = proc(_: ^cbor.Tag_Implementation, e: cbor.Encoder, v: any) -> cbor.Marshal_Error { cbor._encode_u8(e.writer, RAW_TAG_NR, .Tag) or_return return cbor.err_conv(cbor._encode_bytes(e, reflect.as_bytes(v))) }, unmarshal = proc(_: ^cbor.Tag_Implementation, d: cbor.Decoder, _: cbor.Tag_Number, v: any) -> (cbor.Unmarshal_Error) { hdr := cbor._decode_header(d.reader) or_return maj, add := cbor._header_split(hdr) if maj != .Bytes { return .Bad_Tag_Value } bytes := cbor.err_conv(cbor._decode_bytes(d, add, maj)) or_return intrinsics.mem_copy_non_overlapping(v.data, raw_data(bytes), len(bytes)) return nil }, }, RAW_TAG_NR, "raw") now := time.Time{_nsec = 1701117968 * 1e9} data := Data{ str = "Hello, World!", neg = 300, now = now, ignore_this = &Data{}, renamed = 123123.125, my_union = 3, my_raw = {1=1, 2=2, 3=3}, } // Marshal the struct into binary CBOR. binary, err := cbor.marshal(data, cbor.ENCODE_FULLY_DETERMINISTIC) fmt.assertf(err == nil, "marshal error: %v", err) defer delete(binary) // Decode the binary data into a `cbor.Value`. decoded, derr := cbor.decode(string(binary)) fmt.assertf(derr == nil, "decode error: %v", derr) defer cbor.destroy(decoded) // Turn the CBOR into a human readable representation defined as the diagnostic format in [[RFC 8949 Section 8;https://www.rfc-editor.org/rfc/rfc8949.html#name-diagnostic-notation]]. diagnosis, eerr := cbor.to_diagnostic_format(decoded) fmt.assertf(eerr == nil, "to diagnostic error: %v", eerr) defer delete(diagnosis) fmt.println(diagnosis) } Output: { "my_raw": 200(h'00001000200030000000000000000000'), "my_union": 1010([ "int", 3 ]), "neg": -301, "now": 1(1701117968), "renamed :)": 123123.12500000, "str": "Hello, World!" }

Collection Info

View Source
Collection
core
Path
encoding/cbor
Entries
100

Source Files

Constants

16

DEFAULT_MAX_PRE_ALLOC #

Source
DEFAULT_MAX_PRE_ALLOC :: mem.Kilobyte

The default maximum amount of bytes to allocate on a buffer/container at once to prevent malicious input from causing massive allocations.

ENCODE_FULLY_DETERMINISTIC #

Source
ENCODE_FULLY_DETERMINISTIC :: Encoder_Flags{.Deterministic_Int_Size, .Deterministic_Float_Size, .Deterministic_Map_Sorting}

Flags for fully deterministic output (if you are not using streaming/indeterminate length).

ENCODE_SMALL #

Source
ENCODE_SMALL :: Encoder_Flags{.Deterministic_Int_Size, .Deterministic_Float_Size}

Flags for the smallest encoding output.

INITIAL_STREAMED_BYTES_CAPACITY #

Source
INITIAL_STREAMED_BYTES_CAPACITY :: 16

If we are decoding a stream of either text or bytes, the initial capacity will be this value.

INITIAL_STREAMED_CONTAINER_CAPACITY #

Source
INITIAL_STREAMED_CONTAINER_CAPACITY :: 8

If we are decoding a stream of either a map or list, the initial capacity will be this value.

INITIALIZE_DEFAULT_TAGS #

Source
INITIALIZE_DEFAULT_TAGS :: #config(CBOR_INITIALIZE_DEFAULT_TAGS, !ODIN_DEFAULT_TO_PANIC_ALLOCATOR && !ODIN_DEFAULT_TO_NIL_ALLOCATOR)

Controls initialization of default tag implementations.

TAG_BASE64_NR #

Source
TAG_BASE64_NR :: 34

The contents of this tag are base64 encoded during marshal and decoded during unmarshal. Use the struct tag `cbor_tag:"34"` or `cbor_tag:"base64"` to have your field string or bytes field en/decoded as base64.

TAG_CBOR_NR #

Source
TAG_CBOR_NR :: 24

Sometimes it is beneficial to carry an embedded CBOR data item that is not meant to be decoded immediately at the time the enclosing data item is being decoded. Tag number 24 (CBOR data item) can be used to tag the embedded byte string as a single data item encoded in CBOR format. Use the struct tag `cbor_tag:"24"` or `cbor_tag:"cbor"` to keep a non-decoded field (string or bytes) of raw CBOR.

TAG_EPOCH_TIME_NR #

Source
TAG_EPOCH_TIME_NR :: 1

UTC time in seconds, unmarshalled into a `core:time` `time.Time` or integer. Use the struct tag `cbor_tag:"1"` or `cbor_tag:"epoch"` to have your `time.Time` field en/decoded as epoch time.

TAG_NEGATIVE_BIG_NR #

Source
TAG_NEGATIVE_BIG_NR :: 3

Using `core:math/big`, big integers are properly encoded and decoded during marshal and unmarshal. These fields use this tag by default, no struct tag required.

TAG_OBJECT_TYPE #

Source
TAG_OBJECT_TYPE :: 1010

A tag that is used to assign a textual type to the object following it. The tag's value must be an array of 2 items, where the first is text (describing the following type) and the second is any valid CBOR value. See the registration: https://datatracker.ietf.org/doc/draft-rundgren-cotx/05/ We use this in Odin to marshal and unmarshal unions.

TAG_SELF_DESCRIBED_CBOR #

Source
TAG_SELF_DESCRIBED_CBOR :: 55799

A tag that is used to detect the contents of a binary buffer (like a file) are CBOR. This tag would wrap everything else, decoders can then check for this header and see if the given content is definitely CBOR. Added by the encoder if it has the flag `.Self_Described_CBOR`, decoded by default.

TAG_UNSIGNED_BIG_NR #

Source
TAG_UNSIGNED_BIG_NR :: 2

Using `core:math/big`, big integers are properly encoded and decoded during marshal and unmarshal. These fields use this tag by default, no struct tag required.

Config Values

1

INITIALIZE_DEFAULT_TAGS #

Source
INITIALIZE_DEFAULT_TAGS :: #config(CBOR_INITIALIZE_DEFAULT_TAGS, !ODIN_DEFAULT_TO_PANIC_ALLOCATOR && !ODIN_DEFAULT_TO_NIL_ALLOCATOR)

Controls initialization of default tag implementations.

Types

37

Add #

Source
Add :: Add

The lower 3 bits of the header which denotes additional information for the type of value.

Header #

Source
Header :: Header

Known/common headers are defined, undefined headers can still be valid. Higher 3 bits is for the major type and lower 5 bits for the additional information.

Major #

Source
Major :: Major

The higher 3 bits of the header which denotes what type of value it is.

Simple #

Source
Simple :: distinct Simple

A distinct atom-like number, range from `0..=19` and `32..=max(u8)`.

Tag_Implementation #

Source
Tag_Implementation :: Tag_Implementation

A tag implementation that handles marshals and unmarshals for the tag it is registered on.

Tag_Marshal_Proc #

Source
Tag_Marshal_Proc :: Tag_Marshal_Proc

Procedure responsible for marshalling the tag in the given `any` into the given encoder.

Unsupported_Type_Error #

Source
Unsupported_Type_Error :: Unsupported_Type_Error

Error that is returned when a type couldn't be marshalled into or out of, as much information as possible/available is added.

Procedures

35

decode_from_decoder #

Source
decode_from_decoder :: proc(d: Decoder, allocator := context.allocator, loc := #caller_location) -> (v: Value, err: Decode_Error) {…}

Reads a CBOR value from the given decoder. See docs on the proc group `decode` for more information.

decode_from_reader #

Source
decode_from_reader :: proc(r: Stream, flags: bit_set[Decoder_Flag] = {}, allocator := context.allocator, loc := #caller_location) -> (v: Value, err: Decode_Error) {…}

Reads a CBOR value from the given reader. See docs on the proc group `decode` for more information.

decode_from_string #

Source
decode_from_string :: proc(s: string, flags: bit_set[Decoder_Flag] = {}, allocator := context.allocator, loc := #caller_location) -> (v: Value, err: Decode_Error) {…}

Decodes the given string as CBOR. See docs on the proc group `decode` for more information.

destroy #

Source
destroy :: proc(val: Value, allocator := context.allocator) {…}

Recursively frees all memory allocated when decoding the passed value.

encode_into_builder #

Source
encode_into_builder :: proc(b: ^Builder, v: Value, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, temp_allocator := context.temp_allocator, loc := #caller_location) -> Encode_Error {…}

Encodes the CBOR value into binary CBOR written to the given builder. See the docs on the proc group `encode_into` for more info.

encode_into_bytes #

Source
encode_into_bytes :: proc(v: Value, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, allocator := context.allocator, temp_allocator := context.temp_allocator, loc := #caller_location) -> (data: []u8, err: Encode_Error) {…}

Encodes the CBOR value into binary CBOR allocated on the given allocator. See the docs on the proc group `encode_into` for more info.

encode_into_encoder #

Source
encode_into_encoder :: proc(e: Encoder, v: Value, loc := #caller_location) -> Encode_Error {…}

Encodes the CBOR value into binary CBOR written to the given encoder. See the docs on the proc group `encode_into` for more info.

encode_into_writer #

Source
encode_into_writer :: proc(w: Stream, v: Value, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, temp_allocator := context.temp_allocator, loc := #caller_location) -> Encode_Error {…}

Encodes the CBOR value into binary CBOR written to the given writer. See the docs on the proc group `encode_into` for more info.

from_json #

Source
from_json :: proc(val: Value, allocator := context.allocator) -> (Value, Allocator_Error) #optional_ok {…}

Converts from JSON to CBOR. Everything is copied to the given allocator, the passed in JSON value can be deleted after.

marshal_into_builder #

Source
marshal_into_builder :: proc(b: ^Builder, v: any, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, temp_allocator := context.temp_allocator) -> Marshal_Error {…}

Marshals the given value into a CBOR byte stream written to the given builder. See docs on the `marshal_into` proc group for more info.

marshal_into_bytes #

Source
marshal_into_bytes :: proc(v: any, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, allocator := context.allocator, temp_allocator := context.temp_allocator, loc := #caller_location) -> (bytes: []u8, err: Marshal_Error) {…}

Marshals the given value into a CBOR byte stream (allocated using the given allocator). See docs on the `marshal_into` proc group for more info.

marshal_into_encoder #

Source
marshal_into_encoder :: proc(e: Encoder, v: any) -> (err: Marshal_Error) {…}

Marshals the given value into a CBOR byte stream written to the given encoder. See docs on the `marshal_into` proc group for more info.

marshal_into_writer #

Source
marshal_into_writer :: proc(w: Stream, v: any, flags: bit_set[Encoder_Flag] = ENCODE_SMALL, temp_allocator := context.temp_allocator) -> Marshal_Error {…}

Marshals the given value into a CBOR byte stream written to the given writer. See docs on the `marshal_into` proc group for more info.

tag_register_number #

Source
tag_register_number :: proc "contextless" (impl: Tag_Implementation, nr: u64, id: string) {…}

Register a custom tag implementation to be used when marshalling that tag number or marshalling a field with the struct tag `cbor_tag:"nr"`.

tag_register_type #

Source
tag_register_type :: proc "contextless" (impl: Tag_Implementation, nr: u64, type: typeid) {…}

Register a custom tag implementation to be used when marshalling that type and unmarshalling that tag number.

tags_register_defaults #

Source
tags_register_defaults :: proc "contextless" () {…}

Registers tags that have implementations provided by this package. This is done by default and can be controlled with the `CBOR_INITIALIZE_DEFAULT_TAGS` define.

to_diagnostic_format_string #

Source
to_diagnostic_format_string :: proc(val: Value, padding: int = 0, allocator := context.allocator, loc := #caller_location) -> (string, Allocator_Error) #optional_ok {…}

Turns the given CBOR value into a human-readable string. See docs on the proc group `diagnose` for more info.

to_diagnostic_format_writer #

Source
to_diagnostic_format_writer :: proc(w: Stream, val: Value, padding: int = 0) -> Error {…}

Writes the given CBOR value into the writer as human-readable text. See docs on the proc group `diagnose` for more info.

to_json #

Source
to_json :: proc(val: Value, allocator := context.allocator) -> (Value, Allocator_Error) #optional_ok {…}

Converts from CBOR to JSON. NOTE: overflow on integers or floats is not handled. Everything is copied to the given allocator, the passed in CBOR value can be `destroy`'ed after. If a CBOR map with non-string keys is encountered it is turned into an array of tuples.

unmarshal_from_bytes #

Source
unmarshal_from_bytes :: proc(
	bytes:          []u8, 
	ptr:            ^$T, 
	flags:          bit_set[Decoder_Flag] = Decoder_Flags{}, 
	allocator := context.allocator, 
	temp_allocator := context.temp_allocator, 
	loc := #caller_location, 
) -> (err: Unmarshal_Error) {…}

Unmarshals from a slice of bytes, see docs on the proc group `Unmarshal` for more info.

unmarshal_from_decoder #

Source
unmarshal_from_decoder :: proc(d: Decoder, ptr: ^$T, allocator := context.allocator, temp_allocator := context.temp_allocator, loc := #caller_location) -> (err: Unmarshal_Error) {…}

unmarshal_from_reader #

Source
unmarshal_from_reader :: proc(
	r:              Stream, 
	ptr:            ^$T, 
	flags:          bit_set[Decoder_Flag] = Decoder_Flags{}, 
	allocator := context.allocator, 
	temp_allocator := context.temp_allocator, 
	loc := #caller_location, 
) -> (err: Unmarshal_Error) {…}

unmarshal_from_string #

Source
unmarshal_from_string :: proc(
	s:              string, 
	ptr:            ^$T, 
	flags:          bit_set[Decoder_Flag] = Decoder_Flags{}, 
	allocator := context.allocator, 
	temp_allocator := context.temp_allocator, 
	loc := #caller_location, 
) -> (err: Unmarshal_Error) {…}

Unmarshals from a string, see docs on the proc group `Unmarshal` for more info.

Procedure Groups

11

decode_from #

Source
decode_from :: proc{
	decode_from_string,
	decode_from_reader,
	decode_from_decoder,
}

Decodes both deterministic and non-deterministic CBOR into a `Value` variant. `Text` and `Bytes` can safely be cast to cstrings because of an added 0 byte. Allocations are done using the given allocator, *no* allocations are done on the `context.temp_allocator`. A value can be (fully and recursively) deallocated using the `destroy` proc in this package. Disable streaming/indeterminate lengths with the `.Disallow_Streaming` flag. Shrink excess bytes in buffers and containers with the `.Shrink_Excess` flag. Mark the input as trusted input with the `.Trusted_Input` flag, this turns off the safety feature of not pre-allocating more than `max_pre_alloc` bytes before reading into the bytes. You should only do this when you own both sides of the encoding and are sure there can't be malicious bytes used as an input.

encode_into #

Source
encode_into :: proc{
	encode_into_bytes,
	encode_into_builder,
	encode_into_writer,
	encode_into_encoder,
}

Encodes the CBOR value into a binary CBOR. Flags can be used to control the output (mainly determinism, which coincidently affects size). The default flags `ENCODE_SMALL` (`.Deterministic_Int_Size`, `.Deterministic_Float_Size`) will try to put ints and floats into their smallest possible byte size without losing equality. Adding the `.Self_Described_CBOR` flag will wrap the value in a tag that lets generic decoders know the contents are CBOR from just reading the first byte. Adding the `.Deterministic_Map_Sorting` flag will sort the encoded maps by the byte content of the encoded key. This flag has a cost on performance and memory efficiency because all keys in a map have to be precomputed, sorted and only then written to the output. Empty flags will do nothing extra to the value. The allocations for the `.Deterministic_Map_Sorting` flag are done using the given temp_allocator. but are followed by the necessary `delete` and `free` calls if the allocator supports them. This is helpful when the CBOR size is so big that you don't want to collect all the temporary allocations until the end.

marshal_into #

Source
marshal_into :: proc{
	marshal_into_bytes,
	marshal_into_builder,
	marshal_into_writer,
	marshal_into_encoder,
}

Marshal a value into binary CBOR. Flags can be used to control the output (mainly determinism, which coincidently affects size). The default flags `ENCODE_SMALL` (`.Deterministic_Int_Size`, `.Deterministic_Float_Size`) will try to put ints and floats into their smallest possible byte size without losing equality. Adding the `.Self_Described_CBOR` flag will wrap the value in a tag that lets generic decoders know the contents are CBOR from just reading the first byte. Adding the `.Deterministic_Map_Sorting` flag will sort the encoded maps by the byte content of the encoded key. This flag has a cost on performance and memory efficiency because all keys in a map have to be precomputed, sorted and only then written to the output. Empty flags will do nothing extra to the value. The allocations for the `.Deterministic_Map_Sorting` flag are done using the given `temp_allocator`. but are followed by the necessary `delete` and `free` calls if the allocator supports them. This is helpful when the CBOR size is so big that you don't want to collect all the temporary allocations until the end.

to_diagnostic_format #

Source
to_diagnostic_format :: proc{
	to_diagnostic_format_string,
	to_diagnostic_format_writer,
}

to_diagnostic_format either writes or returns a human-readable representation of the value, optionally formatted, defined as the diagnostic format in [[RFC 8949 Section 8;https://www.rfc-editor.org/rfc/rfc8949.html#name-diagnostic-notation]]. Incidentally, if the CBOR does not contain any of the additional types defined on top of JSON this will also be valid JSON.

unmarshal #

Source
unmarshal :: proc{
	unmarshal_from_reader,
	unmarshal_from_string,
	unmarshal_from_bytes,
}

Unmarshals the given CBOR into the given pointer using reflection. Types that require allocation are allocated using the given allocator. Some temporary allocations are done on the given `temp_allocator`, but, if you want to, this can be set to a "normal" allocator, because the necessary `delete` and `free` calls are still made. This is helpful when the CBOR size is so big that you don't want to collect all the temporary allocations until the end. Disable streaming/indeterminate lengths with the `.Disallow_Streaming` flag. Shrink excess bytes in buffers and containers with the `.Shrink_Excess` flag. Mark the input as trusted input with the `.Trusted_Input` flag, this turns off the safety feature of not pre-allocating more than `max_pre_alloc` bytes before reading into the bytes. You should only do this when you own both sides of the encoding and are sure there can't be malicious bytes used as an input.