1 /****************************************************************************
3 ** Copyright (C) 2016 Intel Corporation
5 ** Permission is hereby granted, free of charge, to any person obtaining a copy
6 ** of this software and associated documentation files (the "Software"), to deal
7 ** in the Software without restriction, including without limitation the rights
8 ** to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 ** copies of the Software, and to permit persons to whom the Software is
10 ** furnished to do so, subject to the following conditions:
12 ** The above copyright notice and this permission notice shall be included in
13 ** all copies or substantial portions of the Software.
15 ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 ** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 ** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
23 ****************************************************************************/
26 #define _DEFAULT_SOURCE 1
28 #define _POSIX_C_SOURCE 200809L
29 #ifndef __STDC_LIMIT_MACROS
30 # define __STDC_LIMIT_MACROS 1
35 #include "compilersupport_p.h"
36 #include "math_support_p.h"
46 * \defgroup CborToJson Converting CBOR to JSON
47 * \brief Group of functions used to convert CBOR to JSON.
49 * This group contains two functions that are can be used to convert one
50 * CborValue object to an equivalent JSON representation. This module attempts
51 * to follow the recommendations from RFC 7049 section 4.1 "Converting from
52 * CBOR to JSON", though it has a few differences. They are noted below.
54 * These functions produce a "minified" JSON output, with no spacing,
55 * indentation or line breaks. If those are necessary, they need to be applied
56 * in a post-processing phase.
58 * Note that JSON cannot support all CBOR types with fidelity, so the
59 * conversion is usually lossy. For that reason, TinyCBOR supports adding a set
60 * of metadata JSON values that can be used by a JSON-to-CBOR converter to
61 * restore the original data types.
63 * The TinyCBOR library does not provide a way to convert from JSON
64 * representation back to encoded form. However, it provides a tool called
65 * \c json2cbor which can be used for that purpose. That tool supports the
66 * metadata format that these functions may produce.
68 * Either of the functions in this section will attempt to convert exactly one
69 * CborValue object to JSON. Those functions may return any error documented
70 * for the functions for CborParsing. In addition, if the C standard library
71 * stream functions return with error, the text conversion will return with
74 * These functions also perform UTF-8 validation in CBOR text strings. If they
75 * encounter a sequence of bytes that not permitted in UTF-8, they will return
76 * CborErrorInvalidUtf8TextString. That includes encoding of surrogate points
79 * \warning The metadata produced by these functions is not guaranteed to
80 * remain stable. A future update of TinyCBOR may produce different output for
81 * the same input and parsers may be unable to handle them.
83 * \sa CborParsing, CborPretty, cbor_parser_init()
87 * \addtogroup CborToJson
89 * <h2 class="groupheader">Conversion limitations</h2>
91 * When converting from CBOR to JSON, there may be information loss. This
92 * section lists the possible scenarios.
94 * \par Number precision:
95 * ALL JSON numbers, due to its JavaScript heritage, are IEEE 754
96 * double-precision floating point. This means JSON is not capable of
97 * representing integers numbers outside the range [-(2<sup>53</sup>)+1,
98 * 2<sup>53</sup>-1] and is not capable of representing NaN or infinite. If the
99 * CBOR data contains a number outside the valid range, the conversion will
100 * lose precision. If the input was NaN or infinite, the result of the
101 * conversion will be "null". In addition, the distinction between half-,
102 * single- and double-precision is lost.
105 * If enabled, the original value and original type are stored in the metadata.
107 * \par Non-native types:
108 * CBOR's type system is richer than JSON's, which means some data values
109 * cannot be represented when converted to JSON. The conversion silently turns
110 * them into strings: CBOR simple types become "simple(nn)" where \c nn is the
111 * simple type's value, with the exception of CBOR undefined, which becomes
112 * "undefined", while CBOR byte strings are converted to an Base16, Base64, or
116 * If enabled, the original type is stored in the metadata.
118 * \par Presence of tags:
119 * JSON has no support for tagged values, so by default tags are dropped when
120 * converting to JSON. However, if the CborConvertObeyByteStringTags option is
121 * active (default), then certain known tags are honored and are used to format
122 * the conversion of the tagged byte string to JSON.
125 * If the CborConvertTagsToObjects option is active, then the tag and the
126 * tagged value are converted to to a JSON object. Otherwise, if enabled, the
127 * last (innermost) tag is stored in the metadata.
129 * \par Non-string keys in maps:
130 * JSON requires all Object keys to be strings, while CBOR does not. By
131 * default, if a non-string key is found, the conversion fails with error
132 * CborErrorJsonObjectKeyNotString. If the CborConvertStringifyMapKeys option
133 * is active, then the conversion attempts to create a string representation
134 * using CborPretty. Note that the \c json2cbor tool is not able to parse this
135 * back to the original form.
137 * \par Duplicate keys in maps:
138 * Neither JSON nor CBOR allow duplicated keys, but current TinyCBOR does not
139 * validate that this is the case. If there are duplicated keys in the input,
140 * they will be repeated in the output, which may JSON tools may flag as
141 * invalid. In addition to that, if the CborConvertStringifyMapKeys option is
142 * active, it is possible that a non-string key in a CBOR map will be converted
143 * to a string form that is identical to another key.
146 * When metadata support is active, the conversion will add extra key-value
147 * pairs to the JSON output so it can store the metadata. It is possible that
148 * the keys for the metadata clash with existing keys in the JSON map.
151 extern FILE *open_memstream(char **bufptr, size_t *sizeptr);
153 enum ConversionStatusFlags {
154 TypeWasNotNative = 0x100, /* anything but strings, boolean, null, arrays and maps */
155 TypeWasTagged = 0x200,
156 NumberPrecisionWasLost = 0x400,
157 NumberWasNaN = 0x800,
158 NumberWasInfinite = 0x1000,
159 NumberWasNegative = 0x2000, /* always used with NumberWasInifite or NumberWasTooBig */
164 typedef struct ConversionStatus {
166 uint64_t originalNumber;
170 static CborError value_to_json(FILE *out, CborValue *it, int flags, CborType type, ConversionStatus *status);
172 static CborError dump_bytestring_base16(char **result, CborValue *it)
174 static const char characters[] = "0123456789abcdef";
178 CborError err = cbor_value_calculate_string_length(it, &n);
182 /* a Base16 (hex) output is twice as big as our buffer */
183 buffer = (uint8_t *)malloc(n * 2 + 1);
184 *result = (char *)buffer;
186 /* let cbor_value_copy_byte_string know we have an extra byte for the terminating NUL */
188 err = cbor_value_copy_byte_string(it, buffer + n - 1, &n, it);
189 assert(err == CborNoError);
191 for (i = 0; i < n; ++i) {
192 uint8_t byte = buffer[n + i];
193 buffer[2*i] = characters[byte >> 4];
194 buffer[2*i + 1] = characters[byte & 0xf];
199 static CborError generic_dump_base64(char **result, CborValue *it, const char alphabet[65])
202 uint8_t *buffer, *out, *in;
203 CborError err = cbor_value_calculate_string_length(it, &n);
207 /* a Base64 output (untruncated) has 4 bytes for every 3 in the input */
208 size_t len = (n + 5) / 3 * 4;
209 out = buffer = (uint8_t *)malloc(len + 1);
210 *result = (char *)buffer;
212 /* we read our byte string at the tail end of the buffer
213 * so we can do an in-place conversion while iterating forwards */
214 in = buffer + len - n;
216 /* let cbor_value_copy_byte_string know we have an extra byte for the terminating NUL */
218 err = cbor_value_copy_byte_string(it, in, &n, it);
219 assert(err == CborNoError);
221 uint_least32_t val = 0;
222 for (i = 0; n - i >= 3; i += 3) {
223 /* read 3 bytes x 8 bits = 24 bits */
227 __builtin_memcpy(&val, in + i - 1, sizeof(val));
228 val = cbor_ntohl(val);
231 val = (in[i] << 16) | (in[i + 1] << 8) | in[i + 2];
234 /* write 4 chars x 6 bits = 24 bits */
235 *out++ = alphabet[(val >> 18) & 0x3f];
236 *out++ = alphabet[(val >> 12) & 0x3f];
237 *out++ = alphabet[(val >> 6) & 0x3f];
238 *out++ = alphabet[val & 0x3f];
241 /* maybe 1 or 2 bytes left */
243 /* we can read in[i + 1] even if it's past the end of the string because
244 * we know (by construction) that it's a NUL byte */
247 __builtin_memcpy(&val16, in + i, sizeof(val16));
248 val = cbor_ntohs(val16);
250 val = (in[i] << 8) | in[i + 1];
254 /* the 65th character in the alphabet is our filler: either '=' or '\0' */
256 out[3] = alphabet[64];
258 /* write the third char in 3 chars x 6 bits = 18 bits */
259 out[2] = alphabet[(val >> 6) & 0x3f];
261 out[2] = alphabet[64]; /* filler */
263 out[1] = alphabet[(val >> 12) & 0x3f];
264 out[0] = alphabet[(val >> 18) & 0x3f];
272 static CborError dump_bytestring_base64(char **result, CborValue *it)
274 static const char alphabet[] = "ABCDEFGH" "IJKLMNOP" "QRSTUVWX" "YZabcdef"
275 "ghijklmn" "opqrstuv" "wxyz0123" "456789+/" "=";
276 return generic_dump_base64(result, it, alphabet);
279 static CborError dump_bytestring_base64url(char **result, CborValue *it)
281 static const char alphabet[] = "ABCDEFGH" "IJKLMNOP" "QRSTUVWX" "YZabcdef"
282 "ghijklmn" "opqrstuv" "wxyz0123" "456789-_";
283 return generic_dump_base64(result, it, alphabet);
286 static CborError add_value_metadata(FILE *out, CborType type, const ConversionStatus *status)
288 int flags = status->flags;
289 if (flags & TypeWasTagged) {
290 /* extract the tagged type, which may be JSON native */
291 type = flags & FinalTypeMask;
292 flags &= ~(FinalTypeMask | TypeWasTagged);
294 if (fprintf(out, "\"tag\":\"%" PRIu64 "\"%s", status->lastTag,
295 flags & ~TypeWasTagged ? "," : "") < 0)
302 /* print at least the type */
303 if (fprintf(out, "\"t\":%d", type) < 0)
306 if (flags & NumberWasNaN)
307 if (fprintf(out, ",\"v\":\"nan\"") < 0)
309 if (flags & NumberWasInfinite)
310 if (fprintf(out, ",\"v\":\"%sinf\"", flags & NumberWasNegative ? "-" : "") < 0)
312 if (flags & NumberPrecisionWasLost)
313 if (fprintf(out, ",\"v\":\"%c%" PRIx64 "\"", flags & NumberWasNegative ? '-' : '+',
314 status->originalNumber) < 0)
316 if (type == CborSimpleType)
317 if (fprintf(out, ",\"v\":%d", (int)status->originalNumber) < 0)
322 static CborError find_tagged_type(CborValue *it, CborTag *tag, CborType *type)
324 CborError err = CborNoError;
325 *type = cbor_value_get_type(it);
326 while (*type == CborTagType) {
327 cbor_value_get_tag(it, tag); /* can't fail */
328 err = cbor_value_advance_fixed(it);
332 *type = cbor_value_get_type(it);
337 static CborError tagged_value_to_json(FILE *out, CborValue *it, int flags, ConversionStatus *status)
342 if (flags & CborConvertTagsToObjects) {
343 cbor_value_get_tag(it, &tag); /* can't fail */
344 err = cbor_value_advance_fixed(it);
348 if (fprintf(out, "{\"tag%" PRIu64 "\":", tag) < 0)
351 CborType type = cbor_value_get_type(it);
352 err = value_to_json(out, it, flags, type, status);
355 if (flags & CborConvertAddMetadata && status->flags) {
356 if (fprintf(out, ",\"tag%" PRIu64 "$cbor\":{", tag) < 0 ||
357 add_value_metadata(out, type, status) != CborNoError ||
361 if (fputc('}', out) < 0)
363 status->flags = TypeWasNotNative | CborTagType;
368 err = find_tagged_type(it, &status->lastTag, &type);
371 tag = status->lastTag;
373 /* special handling of byte strings? */
374 if (type == CborByteStringType && (flags & CborConvertByteStringsToBase64Url) == 0 &&
375 (tag == CborNegativeBignumTag || tag == CborExpectedBase16Tag || tag == CborExpectedBase64Tag)) {
379 if (tag == CborNegativeBignumTag) {
381 err = dump_bytestring_base64url(&str, it);
382 } else if (tag == CborExpectedBase64Tag) {
383 err = dump_bytestring_base64(&str, it);
384 } else { /* tag == CborExpectedBase16Tag */
385 err = dump_bytestring_base16(&str, it);
389 err = fprintf(out, "\"%s%s\"", pre, str) < 0 ? CborErrorIO : CborNoError;
391 status->flags = TypeWasNotNative | TypeWasTagged | CborByteStringType;
395 /* no special handling */
396 err = value_to_json(out, it, flags, type, status);
397 status->flags |= TypeWasTagged | type;
401 static CborError stringify_map_key(char **key, CborValue *it, int flags, CborType type)
403 (void)flags; /* unused */
404 (void)type; /* unused */
405 #ifdef WITHOUT_OPEN_MEMSTREAM
406 (void)key; /* unused */
407 (void)it; /* unused */
408 return CborErrorJsonNotImplemented;
412 FILE *memstream = open_memstream(key, &size);
413 if (memstream == NULL)
414 return CborErrorOutOfMemory; /* could also be EMFILE, but it's unlikely */
415 CborError err = cbor_value_to_pretty_advance(memstream, it);
417 if (unlikely(fclose(memstream) < 0 || *key == NULL))
418 return CborErrorInternalError;
423 static CborError array_to_json(FILE *out, CborValue *it, int flags, ConversionStatus *status)
425 const char *comma = "";
426 while (!cbor_value_at_end(it)) {
427 if (fprintf(out, "%s", comma) < 0)
431 CborError err = value_to_json(out, it, flags, cbor_value_get_type(it), status);
438 static CborError map_to_json(FILE *out, CborValue *it, int flags, ConversionStatus *status)
440 const char *comma = "";
442 while (!cbor_value_at_end(it)) {
444 if (fprintf(out, "%s", comma) < 0)
448 CborType keyType = cbor_value_get_type(it);
449 if (likely(keyType == CborTextStringType)) {
451 err = cbor_value_dup_text_string(it, &key, &n, it);
452 } else if (flags & CborConvertStringifyMapKeys) {
453 err = stringify_map_key(&key, it, flags, keyType);
455 return CborErrorJsonObjectKeyNotString;
460 /* first, print the key */
461 if (fprintf(out, "\"%s\":", key) < 0)
464 /* then, print the value */
465 CborType valueType = cbor_value_get_type(it);
466 err = value_to_json(out, it, flags, valueType, status);
468 /* finally, print any metadata we may have */
469 if (flags & CborConvertAddMetadata) {
470 if (!err && keyType != CborTextStringType) {
471 if (fprintf(out, ",\"%s$keycbordump\":true", key) < 0)
474 if (!err && status->flags) {
475 if (fprintf(out, ",\"%s$cbor\":{", key) < 0 ||
476 add_value_metadata(out, valueType, status) != CborNoError ||
489 static CborError value_to_json(FILE *out, CborValue *it, int flags, CborType type, ConversionStatus *status)
499 err = cbor_value_enter_container(it, &recursed);
501 it->ptr = recursed.ptr;
502 return err; /* parse error */
504 if (fputc(type == CborArrayType ? '[' : '{', out) < 0)
507 err = (type == CborArrayType) ?
508 array_to_json(out, &recursed, flags, status) :
509 map_to_json(out, &recursed, flags, status);
511 it->ptr = recursed.ptr;
512 return err; /* parse error */
515 if (fputc(type == CborArrayType ? ']' : '}', out) < 0)
517 err = cbor_value_leave_container(it, &recursed);
519 return err; /* parse error */
521 status->flags = 0; /* reset, there are never conversion errors for us */
525 case CborIntegerType: {
526 double num; /* JS numbers are IEEE double precision */
528 cbor_value_get_raw_integer(it, &val); /* can't fail */
531 if (cbor_value_is_negative_integer(it)) {
532 num = -num - 1; /* convert to negative */
533 if ((uint64_t)(-num - 1) != val) {
534 status->flags = NumberPrecisionWasLost | NumberWasNegative;
535 status->originalNumber = val;
538 if ((uint64_t)num != val) {
539 status->flags = NumberPrecisionWasLost;
540 status->originalNumber = val;
543 if (fprintf(out, "%.0f", num) < 0) /* this number has no fraction, so no decimal points please */
548 case CborByteStringType:
549 case CborTextStringType: {
551 if (type == CborByteStringType) {
552 err = dump_bytestring_base64url(&str, it);
553 status->flags = TypeWasNotNative;
556 err = cbor_value_dup_text_string(it, &str, &n, it);
560 err = (fprintf(out, "\"%s\"", str) < 0) ? CborErrorIO : CborNoError;
566 return tagged_value_to_json(out, it, flags, status);
568 case CborSimpleType: {
570 cbor_value_get_simple_type(it, &simple_type); /* can't fail */
571 status->flags = TypeWasNotNative;
572 status->originalNumber = simple_type;
573 if (fprintf(out, "\"simple(%" PRIu8 ")\"", simple_type) < 0)
579 if (fprintf(out, "null") < 0)
583 case CborUndefinedType:
584 status->flags = TypeWasNotNative;
585 if (fprintf(out, "\"undefined\"") < 0)
589 case CborBooleanType: {
591 cbor_value_get_boolean(it, &val); /* can't fail */
592 if (fprintf(out, val ? "true" : "false") < 0)
597 case CborDoubleType: {
602 status->flags = TypeWasNotNative;
603 cbor_value_get_float(it, &f);
607 case CborHalfFloatType:
608 status->flags = TypeWasNotNative;
609 cbor_value_get_half_float(it, &f16);
610 val = decode_half(f16);
612 cbor_value_get_double(it, &val);
615 int r = fpclassify(val);
616 if (r == FP_NAN || r == FP_INFINITE) {
617 if (fprintf(out, "null") < 0)
619 status->flags |= r == FP_NAN ? NumberWasNaN :
620 NumberWasInfinite | (val < 0 ? NumberWasNegative : 0);
622 uint64_t ival = (uint64_t)fabs(val);
623 if ((double)ival == fabs(val)) {
624 /* print as integer so we get the full precision */
625 r = fprintf(out, "%s%" PRIu64, val < 0 ? "-" : "", ival);
626 status->flags |= TypeWasNotNative; /* mark this integer number as a double */
628 /* this number is definitely not a 64-bit integer */
629 r = fprintf(out, "%." DBL_DECIMAL_DIG_STR "g", val);
637 case CborInvalidType:
638 return CborErrorUnknownType;
641 return cbor_value_advance_fixed(it);
645 * \enum CborToJsonFlags
646 * The CborToJsonFlags enum contains flags that control the conversion of CBOR to JSON.
648 * \value CborConvertAddMetadata Adds metadata to facilitate restoration of the original CBOR data.
649 * \value CborConvertTagsToObjects Converts CBOR tags to JSON objects
650 * \value CborConvertIgnoreTags (default) Ignore CBOR tags, except for byte strings
651 * \value CborConvertObeyByteStringTags (default) Honor formatting of CBOR byte strings if so tagged
652 * \value CborConvertByteStringsToBase64Url Force the conversion of all CBOR byte strings to Base64url encoding, despite any tags
653 * \value CborConvertRequireMapStringKeys (default) Require CBOR map keys to be strings, failing the conversion if they are not
654 * \value CborConvertStringifyMapKeys Convert non-string keys in CBOR maps to a string form
655 * \value CborConvertDefaultFlags Default conversion flags.
659 * \fn CborError cbor_value_to_json(FILE *out, const CborValue *value, int flags)
661 * Converts the current CBOR type pointed by \a value to JSON and writes that
662 * to the \a out stream. If an error occurs, this function returns an error
663 * code similar to CborParsing. The \a flags parameter indicates one of the
664 * flags from CborToJsonFlags that control the conversion.
666 * \sa cbor_value_to_json_advance(), cbor_value_to_pretty()
670 * Converts the current CBOR type pointed by \a value to JSON and writes that
671 * to the \a out stream. If an error occurs, this function returns an error
672 * code similar to CborParsing. The \a flags parameter indicates one of the
673 * flags from CborToJsonFlags that control the conversion.
675 * If no error ocurred, this function advances \a value to the next element.
677 * \sa cbor_value_to_json(), cbor_value_to_pretty_advance()
679 CborError cbor_value_to_json_advance(FILE *out, CborValue *value, int flags)
681 ConversionStatus status;
682 return value_to_json(out, value, flags, cbor_value_get_type(value), &status);