Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Test

on:
push:
branches: [main, v2]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18.x, 20.x, 22.x]
steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm test
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
package-lock.json
4 changes: 0 additions & 4 deletions .travis.yml

This file was deleted.

2 changes: 1 addition & 1 deletion LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# LICENCE

The MIT License (MIT)
Copyright (c) 2013 Rodrigo González, Sapienlab
Copyright (c) 2013 Rodrigo González, SASUD

Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
Expand Down
7 changes: 0 additions & 7 deletions Makefile

This file was deleted.

280 changes: 87 additions & 193 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,242 +1,136 @@
[![Build Status](https://secure.travis-ci.org/sapienlab/jsonpack.png)](http://travis-ci.org/sapienlab/jsonpack)
# jsonpack <img src=https://raw.githubusercontent.com/sapienlab/jsonpack/master/icon.png alt="icon for jsonpack compressor">
# jsonpack

A compression algorithm for JSON
[![Test](https://github.com/rgcl/jsonpack/actions/workflows/test.yml/badge.svg)](https://github.com/rgcl/jsonpack/actions/workflows/test.yml)
[![npm](https://img.shields.io/npm/v/jsonpack)](https://www.npmjs.com/package/jsonpack)

## Introduction
A URL-safe JSON serializer. Produces compact ASCII output usable directly in URLs and `localStorage` — no base64, no binary, no extra encoding step.

jsonpack is a JavaScript program to pack and unpack JSON data.
## Installation

```bash
npm install jsonpack
```

It can compress to 55% of original size if the data has a recursive structure, example
[Earthquake GeoJSON](http://earthquake.usgs.gov/earthquakes/feed/geojson/2.5/month) or
[Twitter API](http://search.twitter.com/search.json?q=Twitter%20API&result_type=mixed).
## Usage

This lib works in both Node.js and browsers (older browsers missing [ES5's JSON.stringify](http://caniuse.com/json) support will need a [shim](http://bestiejs.github.io/json3/)).
**CommonJS**
```js
const { pack, unpack } = require('jsonpack');
```

**Quick example**
```javascript
// big JSON
var json = {...}
**ESM**
```js
import { pack, unpack } from 'jsonpack';
```

// pack the big JSON
var packed = jsonpack.pack(json);
**Example**
```js
const { pack, unpack } = require('jsonpack');

// do stuff...
const data = {
type: 'FeatureCollection',
features: [
{ type: 'Feature', geometry: { type: 'Point', coordinates: [-73.98, 40.74] }, properties: { name: 'A' } },
{ type: 'Feature', geometry: { type: 'Point', coordinates: [-73.99, 40.75] }, properties: { name: 'B' } },
// ... hundreds more
]
};

// And then unpack the packed
var json = jsonpack.unpack(packed);
const packed = pack(data);
// repeated keys like "type", "Feature", "geometry", "Point", "coordinates"
// are stored once in a dictionary and referenced by index

const restored = unpack(packed);
```

## Installation
## API

**jsonpack** can be installed via [cpm][cpm], [volo][volo] or [npm][npm], or simply [downloaded][download].
### `pack(json, options?)`

Via cpm:
Serializes a JSON value into a compact URL-safe string.

```bash
$ cpm install jsonpack
```
- `json` — any JSON-serializable value, or a JSON string
- `options.verbose` — log each step to console (default: `false`)
- `options.debug` — return internal representation instead of string (default: `false`)

Via volo:
`Date` objects are preserved: `unpack(pack(date))` returns a `Date` instance, not a string.

```bash
$ volo add sapienlab/jsonpack
```
Returns a `string`.

Via npm:
### `unpack(packed, options?)`

```bash
$ npm install jsonpack
```
Restores the original value from a packed string.

## API
- `packed` — string produced by `pack()`
- `options.verbose` — log each step to console (default: `false`)

### Attributes

#### jsonpack.JSON
A object that implements the JSON.parse() and JSON.stringify() members.
By default is the native JSON implemented in ECMAscript 5.

### Members

#### jsonpack.pack(json, options)
Retrieve a packed representation of the json

** Parameters **

* json {Object|string}: A valid JSON Object or their string representation
* parameters {[Object]}: A optional object
* verbose (devault is false): If is true, print a log message to the console at each step of packing
* example: `jsonpack.pack(json, { verbose: true });` packs with verbose only
* debug {[boolean=false]}: If is true, return a object with the internal representation of the
parser dictionary and the AST
* example: `jsonpack.pack(json, { debug: true });` packs with debug only

** Returns:**

* string: the packed string representation of the data
* object: if parameters.debug is true

##### Examples

* Example 1: Node.js

```javascript
// Example in node.js, read a file with JSON content and save another file
// with the packed representation of that JSON
var jsonpack = require('jsonpack/main'),
fs = require('fs');

// read a file called myBigJSON.json and execute with
// jsonContent as the content of the file
fs.readFile('../data/bigData.json', 'utf8', function(error, jsonContent) {

// packed now is a string with the packed version of jsonContent
var packed = jsonpack.pack(jsonContent);

// save the packed in a file
fs.writeFile('../data/packed.txt', packed);

});
```
Returns the original value.

* Example 2: Browser/Node.js with AMD
---

```javascript
require(['jsonpack', 'text!../data/bigData.json'], function(jsonpack, jsonContent) {
## Why jsonpack

// packed the data
var packed = jsonpack.pack(jsonContent);

// Do stuff with the packed string
console.log(packed);
});
```
The standard way to embed JSON in a URL or `localStorage` is:

* Example 3: Browser

```html
<script src="path/to/jsonpack/main.js" />
<script>
var json = {
type : 'world',
name: 'earth',
children: [{
type: 'continent',
name: 'America',
children: [{
type : 'country',
name : 'Chile',
children: [{
type : 'commune',
name : 'Antofagasta'
}]
}]
}, {
type: 'continent',
name : 'Europe'
}]
};

var packed = jsonpack.pack(json);

console.log(packed);
// print:
// "type|world|name|earth|children|continent|America|country|Chile|commune|Antofagasta|Europe^^^$0|1|2|3|4|@$0|5|2|6|4|@$0|7|2|8|4|@$0|9|2|A]]]]]|$0|5|2|B]]]"


</script>
```js
encodeURIComponent(JSON.stringify(data))
```

#### jsonpack.unpack(packed, options)
It works, but it *expands* your data — `{`, `"`, `:` become `%7B`, `%22`, `%3A`. A typical API response grows to **140–170% of its original size**.

Unpack the data in the *packed* parameter
The alternative with the best compression, [lz-string](https://github.com/pieroxy/lz-string), shrinks data dramatically but decodes **4–6× slower**.

** Parameters **
jsonpack sits in between: **< 7 KB minified, zero dependencies** (no transitive dependencies either).

* packed {string} : The result of call jsonpack.packed(...)
* options {[Object]}: Optional object
* verbose (default: false) print a log message to the console at each step of packing
### Benchmark

** Return: ** Object, the clone of the original JSON
Measured across 8 real-world datasets (GeoJSON, e-commerce, API responses, deeply nested structures):

##### Examples
![URL-safe JSON serializers: compression vs speed](charts/01-scatter-compression-vs-speed.png)

* Example 1: Node.js
| | encodeURI(JSON) | **jsonpack** | lz-string (URI) |
|---|:---:|:---:|:---:|
| Avg output size | 152% of original | **68% of original** | 39% of original |
| Avg unpack speed | 262 MB/s | **171 MB/s** | 41 MB/s |
| Zero dependencies | ✓ | ✓ | ✓ |
| URL-safe output | ✓ | ✓ | ✓ |

```javascript
// Example in node.js, read a file with packed content and save another file
// with the string representation of the original JSON
var jsonpack = require('jsonpack/main'),
fs = require('fs');

// read a file called packedjson and execute with
// packed as the content of the file
fs.readFile('../data/packed.txt', 'utf8', function(error, packed) {

// data now is a JavaScript Object of the original JSON
var data = jsonpack.unpack(jsonContent);

// save the JSON in a file. data is a Javascript Object, so must be
// stringifed (and pretty print the JSON with 2 space indents).
fs.writeFile('../data/unpacked.json', JSON.stringify(data, null, 2));

});
```
#### Compression by dataset

* Example 2: Browser/Node.js with AMD
![Compression ratio by dataset](charts/02-ratio-per-dataset.png)

```javascript
require(['jsonpack', 'text!../data/packed'], function(jsonpack, packed) {
#### Unpack speed by dataset

// unpacked the data
// json now is a clone of the original JSON
var json = jsonpack.unpack(packed);

// Do stuff with the JavaScript object
console.log(json);
});
![Unpack speed by dataset](charts/03-unpack-speed-per-dataset.png)

```
Full benchmark methodology and raw results: [rgcl/jsonpack-benchmark](https://github.com/rgcl/jsonpack-benchmark)

* Example 3: Browser
### When to use jsonpack

```html
<script src="path/to/jsonpack/main.js" />
<script>
var packed = "type|world|name|earth|children|continent|America|country|Chile|commune|Antofagasta|Europe^^^$0|1|2|3|4|@$0|5|2|6|4|@$0|7|2|8|4|@$0|9|2|A]]]]]|$0|5|2|B]]]"
**Use jsonpack when:**
- You need to store JSON in a URL query string or `localStorage`
- Output size matters (jsonpack produces ~55% less data than `encodeURIComponent`)
- Fast decoding matters (jsonpack decodes 4× faster than lz-string)
- You can't afford to grow your bundle

// unpack the packed to a clone of the original JSON
var json = jsonpack.unpack(packed);

console.log(json);

</script>
```
**Use lz-string instead when** output size is the only constraint and decoding speed doesn't matter.

## FAQ
### This library is stable?
Yes, was tested in Node.js, Chrome and Firefox.
**Use `encodeURIComponent` when** the data is small, changes rarely, or you want zero abstraction.

### How to contribute?
I'm not a native English speaker, so create a issue or better a pull request for all of my grammatical errors :)
As well, if you have a code issue or suggestion, create a issue, Thanks!
### How it works

### What about the icon?
The icon is a generic (LGPL) icon by David Vignoni - http://www.icon-king.com/
jsonpack builds a dictionary of all unique values (strings, integers, floats, dates) in the JSON and replaces them with base-36 indices. The result is a flat, ASCII-only string. Repeated keys and values — common in structured data like API responses and GeoJSON — are stored once and referenced everywhere.

## LICENCE
`Date` objects are stored as ISO 8601 strings in the dictionary and marked with a special token in the structure, so `unpack` can restore them as `Date` instances. This is one area where jsonpack goes beyond what `JSON.parse(JSON.stringify())` offers natively.

The MIT License (MIT)
Copyright (c) 2013 Rodrigo González, Sapienlab
---

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
## Notes

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
- Pack and unpack are synchronous. For large payloads in a browser, run them in a [Web Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API).
- Requires Node.js ≥ 14.
- The packed format is not binary-compatible with other JSON compression libraries.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
## Licence

[cpm]: https://github.org/kriszyp/cpm
[volo]: http://volojs.org/
[npm]: http://npmjs.org/
[download]: https://github.com/sapienlab/jsonpack/archive/master.zip
MIT © 2013 Rodrigo González, SASUD
Binary file added charts/01-scatter-compression-vs-speed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added charts/02-ratio-per-dataset.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added charts/03-unpack-speed-per-dataset.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
19 changes: 19 additions & 0 deletions index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
export interface PackOptions {
verbose?: boolean;
debug?: boolean;
}

export interface UnpackOptions {
verbose?: boolean;
}

/**
* Packs a JSON value into a compact string representation.
* Date objects are preserved and restored as Date instances on unpack.
*/
export function pack(json: unknown, options?: PackOptions): string;

/**
* Unpacks a string produced by `pack` back into its original JSON value.
*/
export function unpack<T = unknown>(packed: string, options?: UnpackOptions): T;
Loading
Loading