api.md revision 268831
1# API documentation 2 3**Table of Contents** *generated with [DocToc](http://doctoc.herokuapp.com/)* 4 5- [Synopsis](#synopsis) 6- [Description](#description) 7 - [Parser functions](#parser-functions) 8 - [Emitting functions](#emitting-functions) 9 - [Conversion functions](#conversion-functions) 10 - [Generation functions](#generation-functions) 11 - [Iteration functions](#iteration-functions) 12 - [Validation functions](#validation-functions) 13 - [Utility functions](#utility-functions) 14- [Parser functions](#parser-functions-1) 15 - [ucl_parser_new](#ucl_parser_new) 16 - [ucl_parser_register_macro](#ucl_parser_register_macro) 17 - [ucl_parser_register_variable](#ucl_parser_register_variable) 18 - [ucl_parser_add_chunk](#ucl_parser_add_chunk) 19 - [ucl_parser_add_string](#ucl_parser_add_string) 20 - [ucl_parser_add_file](#ucl_parser_add_file) 21 - [ucl_parser_get_object](#ucl_parser_get_object) 22 - [ucl_parser_get_error](#ucl_parser_get_error) 23 - [ucl_parser_free](#ucl_parser_free) 24 - [ucl_pubkey_add](#ucl_pubkey_add) 25 - [ucl_parser_set_filevars](#ucl_parser_set_filevars) 26 - [Parser usage example](#parser-usage-example) 27- [Emitting functions](#emitting-functions-1) 28 - [ucl_object_emit](#ucl_object_emit) 29 - [ucl_object_emit_full](#ucl_object_emit_full) 30- [Conversion functions](#conversion-functions-1) 31- [Generation functions](#generation-functions-1) 32 - [ucl_object_new](#ucl_object_new) 33 - [ucl_object_typed_new](#ucl_object_typed_new) 34 - [Primitive objects generation](#primitive-objects-generation) 35 - [ucl_object_fromstring_common](#ucl_object_fromstring_common) 36- [Iteration functions](#iteration-functions-1) 37 - [ucl_iterate_object](#ucl_iterate_object) 38- [Validation functions](#validation-functions-1) 39 - [ucl_object_validate](#ucl_object_validate) 40 41# Synopsis 42 43`#include <ucl.h>` 44 45# Description 46 47Libucl is a parser and `C` API to parse and generate `ucl` objects. Libucl consist of several groups of functions: 48 49### Parser functions 50Used to parse `ucl` files and provide interface to extract `ucl` object. Currently, `libucl` can parse only full `ucl` documents, for instance, it is impossible to parse a part of document and therefore it is impossible to use `libucl` as a streaming parser. In future, this limitation can be removed. 51 52### Emitting functions 53Convert `ucl` objects to some textual or binary representation. Currently, libucl supports the following exports: 54 55- `JSON` - valid json format (can possibly lose some original data, such as implicit arrays) 56- `Config` - human-readable configuration format (lossless) 57- `YAML` - embedded yaml format (has the same limitations as `json` output) 58 59### Conversion functions 60Help to convert `ucl` objects to C types. These functions are used to convert `ucl_object_t` to C primitive types, such as numbers, strings or boolean values. 61 62### Generation functions 63Allow creation of `ucl` objects from C types and creating of complex `ucl` objects, such as hashes or arrays from primitive `ucl` objects, such as numbers or strings. 64 65### Iteration functions 66Iterate over `ucl` complex objects or over a chain of values, for example when a key in an object has multiple values (that can be treated as implicit array or implicit consolidation). 67 68### Validation functions 69Validation functions are used to validate some object `obj` using json-schema compatible object `schema`. Both input and schema must be UCL objects to perform validation. 70 71### Utility functions 72Provide basic utilities to manage `ucl` objects: creating, removing, retaining and releasing reference count and so on. 73 74# Parser functions 75 76Parser functions operates with `struct ucl_parser`. 77 78### ucl_parser_new 79 80~~~C 81struct ucl_parser* ucl_parser_new (int flags); 82~~~ 83 84Creates new parser with the specified flags: 85 86- `UCL_PARSER_KEY_LOWERCASE` - lowercase keys parsed 87- `UCL_PARSER_ZEROCOPY` - try to use zero-copy mode when reading files (in zero-copy mode text chunk being parsed without copying strings so it should exist till any object parsed is used) 88- `UCL_PARSER_NO_TIME` - treat time values as strings without parsing them as floats 89 90### ucl_parser_register_macro 91 92~~~C 93void ucl_parser_register_macro (struct ucl_parser *parser, 94 const char *macro, ucl_macro_handler handler, void* ud); 95~~~ 96 97Register new macro with name .`macro` parsed by handler `handler` that accepts opaque data pointer `ud`. Macro handler should be of the following type: 98 99~~~C 100bool (*ucl_macro_handler) (const unsigned char *data, 101 size_t len, void* ud);` 102~~~ 103 104Handler function accepts macro text `data` of length `len` and the opaque pointer `ud`. If macro is parsed successfully the handler should return `true`. `false` indicates parsing failure and the parser can be terminated. 105 106### ucl_parser_register_variable 107 108~~~C 109void ucl_parser_register_variable (struct ucl_parser *parser, 110 const char *var, const char *value); 111~~~ 112 113Register new variable $`var` that should be replaced by the parser to the `value` string. 114 115### ucl_parser_add_chunk 116 117~~~C 118bool ucl_parser_add_chunk (struct ucl_parser *parser, 119 const unsigned char *data, size_t len); 120~~~ 121 122Add new text chunk with `data` of length `len` to the parser. At the moment, `libucl` parser is not a streamlined parser and chunk *must* contain the *valid* ucl object. For example, this object should be valid: 123 124~~~json 125{ "var": "value" } 126~~~ 127 128while this one won't be parsed correctly: 129 130~~~json 131{ "var": 132~~~ 133 134This limitation may possible be removed in future. 135 136### ucl_parser_add_string 137~~~C 138bool ucl_parser_add_string (struct ucl_parser *parser, 139 const char *data, size_t len); 140~~~ 141 142This function acts exactly like `ucl_parser_add_chunk` does but if `len` argument is zero, then the string `data` must be zero-terminated and the actual length is calculated up to `\0` character. 143 144### ucl_parser_add_file 145 146~~~C 147bool ucl_parser_add_file (struct ucl_parser *parser, 148 const char *filename); 149~~~ 150 151Load file `filename` and parse it with the specified `parser`. This function uses `mmap` call to load file, therefore, it should not be `shrunk` during parsing. Otherwise, `libucl` can cause memory corruption and terminate the calling application. This function is also used by the internal handler of `include` macro, hence, this macro has the same limitation. 152 153### ucl_parser_get_object 154 155~~~C 156ucl_object_t* ucl_parser_get_object (struct ucl_parser *parser); 157~~~ 158 159If the `ucl` data has been parsed correctly this function returns the top object for the parser. Otherwise, this function returns the `NULL` pointer. The reference count for `ucl` object returned is increased by one, therefore, a caller should decrease reference by using `ucl_object_unref` to free object after usage. 160 161### ucl_parser_get_error 162 163~~~C 164const char *ucl_parser_get_error(struct ucl_parser *parser); 165~~~ 166 167Returns the constant error string for the parser object. If no error occurred during parsing a `NULL` object is returned. A caller should not try to free or modify this string. 168 169### ucl_parser_free 170 171~~~C 172void ucl_parser_free (struct ucl_parser *parser); 173~~~ 174 175Frees memory occupied by the parser object. The reference count for top object is decreased as well, however if the function `ucl_parser_get_object` was called previously then the top object won't be freed. 176 177### ucl_pubkey_add 178 179~~~C 180bool ucl_pubkey_add (struct ucl_parser *parser, 181 const unsigned char *key, size_t len); 182~~~ 183 184This function adds a public key from text blob `key` of length `len` to the `parser` object. This public key should be in the `PEM` format and can be used by `.includes` macro for checking signatures of files included. `Openssl` support should be enabled to make this function working. If a key cannot be added (e.g. due to format error) or `openssl` was not linked to `libucl` then this function returns `false`. 185 186### ucl_parser_set_filevars 187 188~~~C 189bool ucl_parser_set_filevars (struct ucl_parser *parser, 190 const char *filename, bool need_expand); 191~~~ 192 193Add the standard file variables to the `parser` based on the `filename` specified: 194 195- `$FILENAME` - a filename of `ucl` input 196- `$CURDIR` - a current directory of the input 197 198For example, if a `filename` param is `../something.conf` then the variables will have the following values: 199 200- `$FILENAME` - "../something.conf" 201- `$CURDIR` - ".." 202 203if `need_expand` parameter is `true` then all relative paths are expanded using `realpath` call. In this example if `..` is `/etc/dir` then variables will have these values: 204 205- `$FILENAME` - "/etc/something.conf" 206- `$CURDIR` - "/etc" 207 208## Parser usage example 209 210The following example loads, parses and extracts `ucl` object from stdin using `libucl` parser functions (the length of input is limited to 8K): 211 212~~~C 213char inbuf[8192]; 214struct ucl_parser *parser = NULL; 215int ret = 0, r = 0; 216ucl_object_t *obj = NULL; 217FILE *in; 218 219in = stdin; 220parser = ucl_parser_new (0); 221while (!feof (in) && r < (int)sizeof (inbuf)) { 222 r += fread (inbuf + r, 1, sizeof (inbuf) - r, in); 223} 224ucl_parser_add_chunk (parser, inbuf, r); 225fclose (in); 226 227if (ucl_parser_get_error (parser)) { 228 printf ("Error occurred: %s\n", ucl_parser_get_error (parser)); 229 ret = 1; 230} 231else { 232 obj = ucl_parser_get_object (parser); 233} 234 235if (parser != NULL) { 236 ucl_parser_free (parser); 237} 238if (obj != NULL) { 239 ucl_object_unref (obj); 240} 241return ret; 242~~~ 243 244# Emitting functions 245 246Libucl can transform UCL objects to a number of tectual formats: 247 248- configuration (`UCL_EMIT_CONFIG`) - nginx like human readable configuration file where implicit arrays are transformed to the duplicate keys 249- compact json: `UCL_EMIT_JSON_COMPACT` - single line valid json without spaces 250- formatted json: `UCL_EMIT_JSON` - pretty formatted JSON with newlines and spaces 251- compact yaml: `UCL_EMIT_YAML` - compact YAML output 252 253Moreover, libucl API allows to select a custom set of emitting functions allowing 254efficient and zero-copy output of libucl objects. Libucl uses the following structure to support this feature: 255 256~~~C 257struct ucl_emitter_functions { 258 /** Append a single character */ 259 int (*ucl_emitter_append_character) (unsigned char c, size_t nchars, void *ud); 260 /** Append a string of a specified length */ 261 int (*ucl_emitter_append_len) (unsigned const char *str, size_t len, void *ud); 262 /** Append a 64 bit integer */ 263 int (*ucl_emitter_append_int) (int64_t elt, void *ud); 264 /** Append floating point element */ 265 int (*ucl_emitter_append_double) (double elt, void *ud); 266 /** Opaque userdata pointer */ 267 void *ud; 268}; 269~~~ 270 271This structure defines the following callbacks: 272 273- `ucl_emitter_append_character` - a function that is called to append `nchars` characters equal to `c` 274- `ucl_emitter_append_len` - used to append a string of length `len` starting from pointer `str` 275- `ucl_emitter_append_int` - this function applies to integer numbers 276- `ucl_emitter_append_double` - this function is intended to output floating point variable 277 278The set of these functions could be used to output text formats of `UCL` objects to different structures or streams. 279 280Libucl provides the following functions for emitting UCL objects: 281 282### ucl_object_emit 283 284~~~C 285unsigned char *ucl_object_emit (const ucl_object_t *obj, enum ucl_emitter emit_type); 286~~~ 287 288Allocate a string that is suitable to fit the underlying UCL object `obj` and fill it with the textual representation of the object `obj` according to style `emit_type`. The caller should free the returned string after using. 289 290### ucl_object_emit_full 291 292~~~C 293bool ucl_object_emit_full (const ucl_object_t *obj, enum ucl_emitter emit_type, 294 struct ucl_emitter_functions *emitter); 295~~~ 296 297This function is similar to the previous with the exception that it accepts the additional argument `emitter` that defines the concrete set of output functions. This emit function could be useful for custom structures or streams emitters (including C++ ones, for example). 298 299# Conversion functions 300 301Conversion functions are used to convert UCL objects to primitive types, such as strings, numbers, or boolean values. There are two types of conversion functions: 302 303- safe: try to convert an ucl object to a primitive type and fail if such a conversion is not possible 304- unsafe: return primitive type without additional checks, if the object cannot be converted then some reasonable default is returned (NULL for strings and 0 for numbers) 305 306Also there is a single `ucl_object_tostring_forced` function that converts any UCL object (including compound types - arrays and objects) to a string representation. For objects, arrays, booleans and numeric types this function performs emitting to a compact json format actually. 307 308Here is a list of all conversion functions: 309 310- `ucl_object_toint` - returns `int64_t` of UCL object 311- `ucl_object_todouble` - returns `double` of UCL object 312- `ucl_object_toboolean` - returns `bool` of UCL object 313- `ucl_object_tostring` - returns `const char *` of UCL object (this string is NULL terminated) 314- `ucl_object_tolstring` - returns `const char *` and `size_t` len of UCL object (string does not need to be NULL terminated) 315- `ucl_object_tostring_forced` - returns string representation of any UCL object 316 317Strings returned by these pointers are associated with the UCL object and exist over its lifetime. A caller should not free this memory. 318 319# Generation functions 320 321It is possible to generate UCL objects from C primitive types. Moreover, libucl allows creation and modifying complex UCL objects, such as arrays or associative objects. 322 323## ucl_object_new 324~~~C 325ucl_object_t * ucl_object_new (void) 326~~~ 327 328Creates new object of type `UCL_NULL`. This object should be released by caller. 329 330## ucl_object_typed_new 331~~~C 332ucl_object_t * ucl_object_typed_new (unsigned int type) 333~~~ 334 335Create an object of a specified type: 336- `UCL_OBJECT` - UCL object - key/value pairs 337- `UCL_ARRAY` - UCL array 338- `UCL_INT` - integer number 339- `UCL_FLOAT` - floating point number 340- `UCL_STRING` - NULL terminated string 341- `UCL_BOOLEAN` - boolean value 342- `UCL_TIME` - time value (floating point number of seconds) 343- `UCL_USERDATA` - opaque userdata pointer (may be used in macros) 344- `UCL_NULL` - null value 345 346This object should be released by caller. 347 348## Primitive objects generation 349Libucl provides the functions similar to inverse conversion functions called with the specific C type: 350- `ucl_object_fromint` - converts `int64_t` to UCL object 351- `ucl_object_fromdouble` - converts `double` to UCL object 352- `ucl_object_fromboolean` - converts `bool` to UCL object 353- `ucl_object_fromstring` - converts `const char *` to UCL object (this string should be NULL terminated) 354- `ucl_object_fromlstring` - converts `const char *` and `size_t` len to UCL object (string does not need to be NULL terminated) 355 356Also there is a function to generate UCL object from a string performing various parsing or conversion operations called `ucl_object_fromstring_common`. 357 358## ucl_object_fromstring_common 359~~~C 360ucl_object_t * ucl_object_fromstring_common (const char *str, 361 size_t len, enum ucl_string_flags flags) 362~~~ 363 364This function is used to convert a string `str` of size `len` to a UCL object applying `flags` conversions. If `len` is equal to zero then a `str` is assumed as NULL-terminated. This function supports the following flags (a set of flags can be specified using logical `OR` operation): 365 366- `UCL_STRING_ESCAPE` - perform JSON escape 367- `UCL_STRING_TRIM` - trim leading and trailing whitespaces 368- `UCL_STRING_PARSE_BOOLEAN` - parse passed string and detect boolean 369- `UCL_STRING_PARSE_INT` - parse passed string and detect integer number 370- `UCL_STRING_PARSE_DOUBLE` - parse passed string and detect integer or float number 371- `UCL_STRING_PARSE_TIME` - parse time values as floating point numbers 372- `UCL_STRING_PARSE_NUMBER` - parse passed string and detect number (both float, integer and time types) 373- `UCL_STRING_PARSE` - parse passed string (and detect booleans, numbers and time values) 374- `UCL_STRING_PARSE_BYTES` - assume that numeric multipliers are in bytes notation, for example `10k` means `10*1024` and not `10*1000` as assumed without this flag 375 376If parsing operations fail then the resulting UCL object will be a `UCL_STRING`. A caller should always check the type of the returned object and release it after using. 377 378# Iteration functions 379 380Iteration are used to iterate over UCL compound types: arrays and objects. Moreover, iterations could be performed over the keys with multiple values (implicit arrays). To iterate over an object, an array or a key with multiple values there is a function `ucl_iterate_object`. 381 382## ucl_iterate_object 383~~~C 384const ucl_object_t* ucl_iterate_object (const ucl_object_t *obj, 385 ucl_object_iter_t *iter, bool expand_values); 386~~~ 387 388This function accepts opaque iterator pointer `iter`. In the first call this iterator *must* be initialized to `NULL`. Iterator is changed by this function call. `ucl_iterate_object` returns the next UCL object in the compound object `obj` or `NULL` if all objects have been iterated. The reference count of the object returned is not increased, so a caller should not unref the object or modify its content (e.g. by inserting to another compound object). The object `obj` should not be changed during the iteration process as well. `expand_values` flag speicifies whether `ucl_iterate_object` should expand keys with multiple values. The general rule is that if you need to iterate through the *object* or *explicit array*, then you always need to set this flag to `true`. However, if you get some key in the object and want to extract all its values then you should set `expand_values` to `false`. Mixing of iteration types is not permitted since the iterator is set according to the iteration type and cannot be reused. Here is an example of iteration over the objects using libucl API (assuming that `top` is `UCL_OBJECT` in this example): 389 390~~~C 391ucl_object_iter_t it = NULL, it_obj = NULL; 392const ucl_object_t *cur, *tmp; 393 394/* Iterate over the object */ 395while ((obj = ucl_iterate_object (top, &it, true))) { 396 printf ("key: \"%s\"\n", ucl_object_key (obj)); 397 /* Iterate over the values of a key */ 398 while ((cur = ucl_iterate_object (obj, &it_obj, false))) { 399 printf ("value: \"%s\"\n", 400 ucl_object_tostring_forced (cur)); 401 } 402} 403~~~ 404 405# Validation functions 406 407Currently, there is only one validation function called `ucl_object_validate`. It performs validation of object using the specified schema. This function is defined as following: 408 409## ucl_object_validate 410~~~C 411bool ucl_object_validate (const ucl_object_t *schema, 412 const ucl_object_t *obj, struct ucl_schema_error *err); 413~~~ 414 415This function uses ucl object `schema`, that must be valid in terms of `json-schema` draft v4, to validate input object `obj`. If this function returns `true` then validation procedure has been succeed. Otherwise, `false` is returned and `err` is set to a specific value. If a caller sets `err` to NULL then this function does not set any error just returning `false`. Error is the structure defined as following: 416 417~~~C 418struct ucl_schema_error { 419 enum ucl_schema_error_code code; /* error code */ 420 char msg[128]; /* error message */ 421 ucl_object_t *obj; /* object where error occurred */ 422}; 423~~~ 424 425Caller may use `code` field to get a numeric error code: 426 427~~~C 428enum ucl_schema_error_code { 429 UCL_SCHEMA_OK = 0, /* no error */ 430 UCL_SCHEMA_TYPE_MISMATCH, /* type of object is incorrect */ 431 UCL_SCHEMA_INVALID_SCHEMA, /* schema is invalid */ 432 UCL_SCHEMA_MISSING_PROPERTY,/* missing properties */ 433 UCL_SCHEMA_CONSTRAINT, /* constraint found */ 434 UCL_SCHEMA_MISSING_DEPENDENCY, /* missing dependency */ 435 UCL_SCHEMA_UNKNOWN /* generic error */ 436}; 437~~~ 438 439`msg` is a string description of an error and `obj` is an object where error has occurred. Error object is not allocated by libucl, so there is no need to free it after validation (a static object should thus be used). 440