Parsing JSON numbers in ECMAScript
I've been worried about numbers and JSON for a while. Not that JSON has a problem: numbers are clearly specified, and in languages (such as Erlang) which support arbitrary precision out-of-the box, parsing JSON is not an issue.
In ECMAScript however, JSON.parse
will generate JSON numbers with the specification's type (native IEE754 doubles), with no way to override the default. So if instead you'd like to parse numbers into large integers using BigInt
(a native ECMAScript feature) or your preferred arbitrary-precision library, you were out of luck.
In my current project (budget-insight
) I had a need for an arbitrary precision parser for superagent (still my preferred fetch API). I surveyed existing streaming parsers, noting on the way that (for example) the respected Oboe
parser uses parseFloat
, or that @dominictarr's JSONStream
uses @creationix' jsonparse
.
In the end I decided to propose a solution for jsonparse
by allowing any reviver for JSON numbers. The Merge Request was accepted but hasn't been published on npmjs yet; you can find it in @shimaore/jsonparse
in the meantime.
To use it simply instantiate jsonparse
with a reviver:
const jsonparse = require(`@shimaore/jsonparse`);
const Decimal = require(`decimal.js-light`);
var parser = new jsonparse();
parser.numberReviver = (text) => new Decimal(text);
You will then receive numbers as arbitrary-precision decimal numbers, thanks to @MikeMcl excellent decimal.js-light
.
Coming back to the original problem, I created a plugin for superagent
to use jsonparse
instead of the default parser (JSON.parse
, obviously), which has the double advantage of
- providing a streaming parsing to superagent, and
- allowing to parse (using my changes to
jsonparse
) JSON numbers as arbitrary-precision numbers.
The result is superagent-jsonparse
:
const request = require('superagent');
const jsonParser = require('superagent-jsonparse');
const Decimal = require('decimal.js-light');
{body} = await request
.get('https://example.com/data.json')
.accept('json')
.use(jsonParser( (text) => new Decimal(text) ))
// body contains arbitrary-precision Decimal for all JSON numbers
You may also use superagent-jsonparse
as a streaming parser if you'd like, although (given that jsonparse
is about 4 times slower than native JSON.parse
on Node.js, for example), the only advantage would be to not block during the execution of JSON.parse
inside superagent
if the input JSON is large.