Is there a Node module for an async JSON parser that does not load the entire JSON string into memory? Is there a Node module for an async JSON parser that does not load the entire JSON string into memory? node.js node.js

Is there a Node module for an async JSON parser that does not load the entire JSON string into memory?


I've written a module that does this: BFJ (Big-Friendly JSON). It exports a bunch of functions that operate at different levels of abstraction, but are all asynchronous and streaming at their core.

At the highest level are two functions for reading from and writing to the file system, bfj.read and bfj.write. They each return a promise, so you call them like this:

var bfj = require('bfj');// Asynchronously read from a JSON file on diskbfj.read(path)  .then(data => {    // :)  })  .catch(error => {    // :(  });// Asynchronously write to a JSON file on diskbfj.write(path, data)  .then(data => {    // :)  })  .catch(error => {    // :(  });

Also at this level is a function for serializing data to a JSON string, called bfj.stringify:

// Asynchronously serialize data to a JSON stringbfj.stringify(data)  .then(json => {    // :)  })  .catch(error => {    // :(  });

Beneath those are two more generic functions for reading from and writing to streams, bfj.parse and bfj.streamify. These serve as foundations for the higher level functions, but you can also call them directly:

// Asynchronously parse JSON from a readable streambfj.parse(readableStream).  .then(data => {    // :)  })  .catch(error => {    // :(  });// Asynchronously serialize data to a writable stream of JSONbfj.streamify(data).  .pipe(writableStream);

At the lowest level there are two functions analagous to SAX parsers/serializers, bfj.walk and bfj.eventify. It's unlikely you'd want to call these directly, they're just the guts of the implementation for the higher levels.

It's open-source and MIT-licensed. For more information, check the readme.


jsonparse is a streamed json parser, the sample code already shown the minimum of using Node stream.

  1. Change client.request() to fs.createReadStream().
  2. Setup on('data') listeners on the file read stream similar to what's in on('response') in the example.