jsonl Stringer

(Since 1.6.0) jsonl/Stringer is a Transform stream. It consumes a stream of JavaScript objects and converts it to a JSONL (AKA NDJSON) file. It is very useful when you want to edit a stream with filters and custom code, and save it back to a file.

⚠️ Deprecated — use stream-chain/jsonl/stringerStream instead. This module is a thin delegation to stream-chain's JSONL stringer (same result, plus prefix / suffix / space / emptyValue options). stream-json is a JSON token library; JSONL yields whole objects per line and belongs in stream-chain alongside the other substrate components, so stream-json's JSONL is slated for removal in a future major version. Migrate now by importing from stream-chain directly. For writing a JSONL file directly, stream-chain also ships stringerToFile() from stream-chain/jsonl/file/stringer.js (a block-writing file sink) — use it instead of piping the stringer into a file write.

jsonl/Stringer has three major differences with Stringer:

• It consumes a stream of JavaScript objects, while Stringer consumes a stream of raw tokens.
• It produces a valid JSONL file, where all values are separated by newline symbols.
• It is faster.

Introduction

const jsonlStringer = require('stream-json/jsonl/stringer.js');
const jsonlParser = require('stream-json/jsonl/parser.js');
const {chain} = require('stream-chain');

const fs = require('fs');
const zlib = require('zlib');

chain([
  fs.createReadStream('data.jsonl.gz'),
  zlib.createGunzip(),
  jsonlParser(),
  // select only active values
  data => (data.value.active ? data.value : chain.none),
  jsonlStringer(),
  zlib.createGzip(),
  fs.createWriteStream('edited.jsonl.gz')
]);

API

jsonl/Stringer has no special API. It is based on Transform with its writable part operating in object mode and its readable part operating in text mode.

constructor(options)

options is an optional object described in details in node.js' Stream documentation. Additionally, the following optional custom flags are recognized:

replacer can be one of two optional objects:

• It can be a function, which takes two parameters and returns a value.
• It can be an array of strings or numbers to define a whitelist for object properties.
• See JSON.stringify() for more details.

Static methods and properties

jsonlStringer.stringer(options)

Alias of the factory function.

jsonlStringer.asStream(options)

Same as jsonlStringer(options) — since jsonlStringer already returns a stream, this is an identity alias for API consistency.

const jsonlStringer = require('stream-json/jsonl/stringer.js');
const jsonlParser = require('stream-json/jsonl/parser.js');
const {chain} = require('stream-chain');

const fs = require('fs');

const pipeline = chain([fs.createReadStream('sample.jsonl'), jsonlParser.asStream(), data => data.value, jsonlStringer()]);

The code above roundtrips data from sample.jsonl.

Web Streams

jsonlStringer ships in two substrate-specific entries:

• Node — stream-json/jsonl/stringer.js. jsonlStringer() is itself a Transform stream (as before). jsonlStringer.asWebStream(options) returns a Web TransformStream<T, string> (delegates to stream-chain/jsonl/stringerWebStream).
• Web — stream-json/web/jsonl/stringer.js. The factory itself returns a Web TransformStream<T, string>. Pulls in no Node-stream imports.

The Web variant supports additional options: separator (default '\n'), prefix, suffix, space, emptyValue, plus strategy / writableStrategy / readableStrategy for QueuingStrategy configuration.

// Web
import jsonlStringer from 'stream-json/web/jsonl/stringer.js';

const ts = jsonlStringer();
objectSource.pipeTo(ts.writable);
let result = '';
for await (const chunk of ts.readable) result += chunk;
console.log(result);

Or paired with the Web JSONL parser via chain:

import {chain} from 'stream-chain/web';
import jsonlParser from 'stream-json/web/jsonl/parser.js';
import jsonlStringer from 'stream-json/web/jsonl/stringer.js';

const pipeline = chain([source, jsonlParser(), ({value}) => value, jsonlStringer()]);
for await (const chunk of pipeline.readable) console.log(chunk);