1Revision history for Perl extension JSON::XS 2 3TODO: maybe detetc and croak on more invalid inputs (e.g. +-inf/nan) 4TODO: maybe avoid the reblessing and better support readonly objects. 5TODO: http://stevehanov.ca/blog/index.php?id=104 compression 6 72.33 Wed Aug 1 21:03:52 CEST 2012 8 - internal encode/decode XS wrappers did not expect stack 9 moves caused by callbacks (analyzed and testcase by Jesse Luehrs). 10 - add bencode as to/from option in bin/json_xs. 11 - add -e option to json_xs, and none and string in/out formats. 12 132.32 Thu Aug 11 19:06:38 CEST 2011 14 - fix a bug in the initial whitespace accumulation. 15 162.31 Wed Jul 27 17:53:05 CEST 2011 17 - don't accumulate initial whitespace in the incremental buffer 18 (this can be useful to allow whitespace-keepalive on a tcp 19 connection without triggering the max_size limit). 20 - properly croak on some invalid inputs that are not strings 21 (e.g. undef) when trying to decode a json text (reported 22 and analyzed by Goro Fuji). 23 242.3 Wed Aug 18 01:26:47 CEST 2010 25 - make sure decoder doesn't change the decoding in the incremental 26 parser (testcase provided by Hendrik Schumacher). 27 - applied patch by DaTa for Data::Dumper support in json_xs. 28 - added -t dump support to json_xs, using Data::Dump. 29 - added -f eval support to json_xs. 30 312.29 Wed Mar 17 02:39:12 CET 2010 32 - fix a memory leak when callbacks set using filter_json_object 33 or filter_json_single_key_object were called (great testcase 34 by Eric Wilhelm). 35 362.28 Thu Mar 11 20:30:46 CET 2010 37 - implement our own atof function - perl's can be orders of 38 magnitudes slower than even the system one. on the positive 39 side, ours seems to be more exact in general than perl's. 40 (testcase provided by Tim Meadowcroft). 41 - clarify floating point conversion issues a bit. 42 - update jpsykes csrf article url. 43 - updated benchmark section - JSON::PP became much faster! 44 452.27 Thu Jan 7 07:35:08 CET 2010 46 - support relaxed option inside the incremental parser 47 (testcase provided by IKEGAMI via Makamaka). 48 492.26 Sat Oct 10 03:26:19 CEST 2009 50 - big integers could become truncated (based on patch 51 by Strobl Anton). 52 - output format change: indent now adds a final newline, which is 53 more expected and more true to the documentation. 54 552.25 Sat Aug 8 12:04:41 CEST 2009 56 - the perl debugger completely breaks lvalue subs - try to work 57 around the issue. 58 - ignore RMAGICAL hashes w.r.t. CANONICAL. 59 - try to work around a possible char signedness issue on aix. 60 - require common sense. 61 622.24 Sat May 30 08:25:45 CEST 2009 63 - the incremental parser did not update its parse offset 64 pointer correctly when parsing utf8-strings (nicely 65 debugged by Martin Evans). 66 - appending a non-utf8-string to the incremental parser 67 in utf8 mode failed to upgrade the string. 68 - wording of parse error messages has been improved. 69 702.232 Sun Feb 22 11:12:25 CET 2009 71 - use an exponential algorithm to extend strings, to 72 help platforms with bad or abysmal==windows memory 73 allocater performance, at the expense of some memory 74 wastage (use shrink to recover this extra memory). 75 (nicely analysed by Dmitry Karasik). 76 772.2311 Thu Feb 19 02:12:54 CET 2009 78 - add a section "JSON and ECMAscript" to explain some 79 incompatibilities between the two (problem was noted by 80 various people). 81 - add t/20_faihu.t. 82 832.231 Thu Nov 20 04:59:08 CET 2008 84 - work around 5.10.0 magic bugs where manipulating magic values 85 (such as $1) would permanently damage them as perl would 86 ignore the magicalness, by making a full copy of the string, 87 reported by Dmitry Karasik. 88 - work around spurious warnings under older perl 5.8's. 89 902.23 Mon Sep 29 05:08:29 CEST 2008 91 - fix a compilation problem when perl is not using char * as, well, 92 char *. 93 - use PL_hexdigit in favour of rolling our own. 94 952.2222 Sun Jul 20 18:49:00 CEST 2008 96 - same game again, broken 5.10 finds yet another assertion 97 failure, and the workaround causes additional runtime warnings. 98 Work around the next assertion AND the warning. 5.10 seriously 99 needs to adjust it's attitude against working code. 100 1012.222 Sat Jul 19 06:15:34 CEST 2008 102 - you work around one -DDEBUGGING assertion bug in perl 5.10 103 just to hit the next one. work around this one, too. 104 1052.22 Tue Jul 15 13:26:51 CEST 2008 106 - allow higher nesting levels in incremental parser. 107 - error out earlier in some cases in the incremental parser 108 (as suggested by Yuval Kogman). 109 - improve incr-parser test (Yuval Kogman). 110 1112.21 Tue Jun 3 08:43:23 CEST 2008 112 - (hopefully) work around a perl 5.10 bug with -DDEBUGGING. 113 - remove the experimental status of the incremental parser interface. 114 - move =encoding around again, to avoid bugs with search.cpan.org. 115 when can we finally have utf-8 in pod??? 116 - add ->incr_reset method. 117 1182.2 Wed Apr 16 20:37:25 CEST 2008 119 - lifted the log2 rounding restriction of max_depth and max_size. 120 - make booleans mutable by creating a copy instead of handing out 121 the same scalar (reported by pasha sadri). 122 - added support for incremental json parsing (still EXPERIMENTAL). 123 - implemented and added a json_xs command line utility that can convert 124 from/to a number of serialisation formats - tell me if you need more. 125 - implement allow_unknown/get_allow_unknown methods. 126 - fixed documentation of max_depth w.r.t. higher and equal. 127 - moved down =encoding directive a bit, too much breaks if it's the first 128 pod directive :/. 129 - removed documentation section on other modules, it became somewhat 130 outdated and is nowadays mostly of historical interest. 131 1322.1 Wed Mar 19 23:23:18 CET 2008 133 - update documentation here and there: add a large section 134 about utf8/latin1/ascii flags, add a security consideration 135 and extend and clarify the JSON and YAML section. 136 - medium speed enhancements when encoding/decoding non-ascii chars. 137 - minor speedup in number encoding case. 138 - extend and clarify the section on incompatibilities 139 between YAML and JSON. 140 - switch to static inline from just inline when using gcc. 141 - add =encoding utf-8 to the manpage, now that perl 5.10 supports it. 142 - fix some issues with UV to JSON conversion of unknown impact. 143 - published the yahoo locals search result used in benchmarks as the 144 original url changes so comparison is impossible. 145 1462.01 Wed Dec 5 11:40:28 CET 2007 147 - INCOMPATIBLE API CHANGE: to_json and from_json have been 148 renamed to encode_json/decode_json for JSON.pm compatibility. 149 The old functions croak and might be replaced by JSON.pm 150 comaptible versions in some later release. 151 1522.0 Tue Dec 4 11:30:46 CET 2007 153 - this is supposed to be the first version of JSON::XS 154 compatible with version 2.0+ of the JSON module. 155 Using the JSON module as frontend to JSON::XS should be 156 as fast as using JSON::XS directly, so consider using it 157 instead. 158 - added get_* methods for all "simple" options. 159 - make JSON::XS subclassable. 160 1611.53 Tue Nov 13 23:58:33 CET 2007 162 - minor doc clarifications. 163 - fixed many doc typos (patch by Thomas L. Shinnick). 164 1651.52 Mon Oct 15 03:22:06 CEST 2007 166 - remove =encoding pod directive again, it confuses too many pod 167 parsers :/. 168 1691.51 Sat Oct 13 03:55:56 CEST 2007 170 - encode empty arrays/hashes in a compact way when pretty is enabled. 171 - apparently JSON::XS was used to find some bugs in the 172 JSON_checker testsuite, so add (the corrected) JSON_checker tests to 173 the testsuite. 174 - quite a bit of doc updates/extension. 175 - require 5.8.2, as this seems to be the first unicode-stable version. 176 1771.5 Tue Aug 28 04:05:38 CEST 2007 178 - add support for tied hashes, based on ideas and testcase by 179 Marcus Holland-Moritz. 180 - implemented relaxed parsing mode where some extensions are being 181 accepted. generation is still JSON-only. 182 1831.44 Wed Aug 22 01:02:44 CEST 2007 184 - very experimental process-emulation support, slowing everything down. 185 the horribly broken perl threads are still not supported - YMMV. 186 1871.43 Thu Jul 26 13:26:37 CEST 2007 188 - convert big json numbers exclusively consisting of digits to NV 189 only when there is no loss of precision, otherwise to string. 190 1911.42 Tue Jul 24 00:51:18 CEST 2007 192 - fix a crash caused by not handling missing array elements 193 (report and testcase by Jay Kuri). 194 1951.41 Tue Jul 10 18:21:44 CEST 2007 196 - fix compilation with NDEBUG (assert side-effect), 197 affects convert_blessed only. 198 - fix a bug in decode filters calling ENTER; SAVETMPS; 199 one time too often. 200 - catch a typical error in TO_JSON methods. 201 - antique-ised XS.xs again to work with outdated 202 C compilers (windows...). 203 2041.4 Mon Jul 2 10:06:30 CEST 2007 205 - add convert_blessed setting. 206 - encode did not catch all blessed objects, encoding their 207 contents in most cases. This has been fixed by introducing 208 the allow_blessed setting. 209 - added filter_json_object and filter_json_single_key_object 210 settings that specify a callback to be called when 211 all/specific json objects are encountered. 212 - assume that most object keys are simple ascii words and 213 optimise this case, penalising the general case. This can 214 speed up decoding by 30% in typical cases and gives 215 a smaller and faster perl hash. 216 - implemented simpleminded, optional resource size checking 217 in decode_json. 218 - remove objToJson/jsonToObj aliases, as the next version 219 of JSON will not have them either. 220 - bit the bullet and converted the very simple json object 221 into a more complex one. 222 - work around a bug where perl wrongly claims an integer 223 is not an integer. 224 - unbundle JSON::XS::Boolean into own pm file so Storable 225 and similar modules can resolve the overloading when thawing. 226 2271.3 Sun Jun 24 01:55:02 CEST 2007 228 - make JSON::XS::true and false special overloaded objects 229 and return those instead of 1 and 0 for those json atoms 230 (JSON::PP compatibility is NOT achieved yet). 231 - add JSON::XS::is_bool predicate to test for those special 232 values. 233 - add a reference to 234 http://jpsykes.com/47/practical-csrf-and-json-security. 235 - removed require 5.8.8 again, it is just not very expert-friendly. 236 Also try to be more compatible with slightly older versions, 237 which are not recommended (because they are buggy). 238 2391.24 Mon Jun 11 05:40:49 CEST 2007 240 - added informative section on JSON-as-YAML. 241 - get rid of some c99-isms again. 242 - localise dec->cur in decode_str, speeding up 243 string decoding considerably (>15% on my amd64 + gcc). 244 - increased SHORT_STRING_LEN to 16kb: stack space is 245 usually plenty, and this actually saves memory 246 when !shrinking as short strings will fit perfectly. 247 2481.23 Wed Jun 6 20:13:06 CEST 2007 249 - greatly improved small integer encoding and decoding speed. 250 - implement a number of µ-optimisations. 251 - updated benchmarks. 252 2531.22 Thu May 24 00:07:25 CEST 2007 254 - require 5.8.8 explicitly as older perls do not seem to offer 255 the required macros. 256 - possibly made it compile on so-called C compilers by microsoft. 257 2581.21 Wed May 9 18:40:32 CEST 2007 259 - character offset reported for trailing garbage was random. 260 2611.2 Wed May 9 18:35:01 CEST 2007 262 - decode did not work with magical scalars (doh!). 263 - added latin1 flag to produce JSON texts in the latin1 subset 264 of unicode. 265 - flag trailing garbage as error. 266 - new decode_prefix method that returns the number 267 of characters consumed by a decode. 268 - max octets/char in perls UTF-X is actually 13, not 11, 269 as pointed out by Glenn Linderman. 270 - fixed typoe reported by YAMASHINA Hio. 271 2721.11 Mon Apr 9 07:05:49 CEST 2007 273 - properly 0-terminate sv's returned by encode to help 274 C libraries that expect that 0 to be there. 275 - partially "port" JSON from C to microsofts fucking broken 276 pseudo-C. They should be burned to the ground for pissing 277 on standards. And I should be stoned for even trying to 278 support this filthy excuse for a c compiler. 279 2801.1 Wed Apr 4 01:45:00 CEST 2007 281 - clarify documentation (pointed out by Quinn Weaver). 282 - decode_utf8 sometimes did not correctly flag errors, 283 leading to segfaults. 284 - further reduced default nesting depth to 512 due to the test 285 failure by that anonymous "chris" whose e-mail address seems 286 to be impossible to get. Tests on other freebsd systems indicate 287 that this is likely a problem in his/her configuration and not this 288 module. 289 - renamed json => JSON in error messages. 290 - corrected the character offset in some error messages. 291 2921.01 Sat Mar 31 16:15:40 CEST 2007 293 - do not segfault when from_json/decode gets passed 294 a non-string object (reported by Florian Ragwitz). 295 This has no effect on normal operation. 296 2971.0 Thu Mar 29 04:43:34 CEST 2007 298 - the long awaited (by me) 1.0 version. 299 - add \0 (JSON::XS::false) and \1 (JSON::XS::true) mappings to JSON 300 true and false. 301 - add some more notes to shrink, as suggested by Alex Efros. 302 - improve testsuite. 303 - halve the default nesting depth limit, to hopefully make it 304 work on Freebsd (unfortunately, the cpan tester did not 305 send me his report, so I cannot ask about the stack limit on fbsd). 306 3070.8 Mon Mar 26 00:10:48 CEST 2007 308 - fix a memleak when decoding hashes. 309 - export jsonToBj and objToJson as aliases 310 to to_json and from_json, to reduce incompatibilities 311 between JSON/JSON::PC and JSON::XS. (experimental). 312 - implement a maximum nesting depth for both en- and de-coding. 313 - added a security considerations sections. 314 3150.7 Sun Mar 25 01:46:30 CET 2007 316 - code cleanup. 317 - fix a memory overflow bug when indenting. 318 - pretty-printing now up to 15% faster. 319 - improve decoding speed of strings by 320 up to 50% by specialcasing short strings. 321 - further decoding speedups for strings using 322 lots of \u escapes. 323 - improve utf8 decoding speed for U+80 .. U+7FF. 324 3250.5 Sat Mar 24 20:41:51 CET 2007 326 - added the UTF-16 encoding example hinted at in previous 327 versions. 328 - minor documentation fixes. 329 - fix a bug in and optimise canonicalising fastpath 330 (reported by Craig Manley). 331 - remove a subtest that breaks with bleadperl (reported 332 by Andreas König). 333 3340.31 Sat Mar 24 02:14:34 CET 2007 335 - documentation updates. 336 - do some casting to hopefully fix Andreas' problem. 337 - nuke bogus json rpc stuff. 338 3390.3 Fri Mar 23 19:33:21 CET 2007 340 - remove spurious PApp::Util reference (John McNamara). 341 - adapted lots of tests from other json modules 342 (idea by Chris Carline). 343 - documented mapping from json to perl and vice versa. 344 - improved the documentation by adding more examples. 345 - added short escaping forms, reducing the created 346 json texts a bit. 347 - added shrink flag. 348 - when flag methods are called without enable argument 349 they will by default enable their flag. 350 - considerably improved string encoding speed (at least 351 with gcc 4). 352 - added a test that covers lots of different characters. 353 - clarified some error messages. 354 - error messages now use correct character offset 355 with F_UTF8. 356 - improve the "no bytes" and "no warnings" hacks in 357 case the called functions do... stuff. 358 - croak when encoding to ascii and an out-of-range 359 (non-unicode) codepoint is encountered. 360 3610.2 Fri Mar 23 00:23:34 CET 2007 362 - the "could not sleep without debugging release". 363 it should basically work now, with many bugs as 364 no production tests have been run yet. 365 - added more testcases. 366 - the expected shitload of bugfixes. 367 - handle utf8 flag correctly in decode. 368 - fix segfault in decoder. 369 - utf8n_to_uvuni sets retlen to -1, but retlen is an 370 unsigned types (argh). 371 - fix decoding of utf-8 strings. 372 - improved error diagnostics. 373 - fix decoding of 'null'. 374 - fix parsing of empty array/hashes 375 - silence warnings when we prepare the croak message. 376 3770.1 Thu Mar 22 22:13:43 CET 2007 378 - first release, very untested, basically just to claim 379 the namespace. 380 3810.01 Thu Mar 22 06:08:12 CET 2007 382 - original version; cloned from Convert-Scalar 383 384