Skip to content

simd-json-evaluation.md

Latest commit

 

History

History
54 lines (34 loc) · 3.12 KB

simd-json-evaluation.md

File metadata and controls

54 lines (34 loc) · 3.12 KB
Raw

SIMD JSON Deserialization Evaluation & Architecture Findings

This document summarizes the investigation, benchmarks, and architectural evaluation of integrating SIMD-accelerated JSON parsers (such as simd-json) into OxyRoute (issue #163).

1. Background & Motivation

In high-throughput HTTP microservices, deserializing incoming JSON payloads into Python-accessible data structures can consume 30–50% of CPU time on compute-heavy routes.

Standard serde_json processes bytes sequentially using scalar CPU instructions. In contrast, SIMD (Single Instruction, Multiple Data) parsers leverage 128-bit (SSE4.2/NEON), 256-bit (AVX2), and 512-bit (AVX-512) vector registers to parse delimiters, string escapes, and numbers in parallel at rates up to 2.5–3.5 GB/s.

2. Benchmark & Comparative Findings

Payload Scale Profiling

Testing parsing throughput across payload sizes:

Payload Size Typical Entity serde_json + json_to_py SIMD Accelerated DOM Relative Speedup
1 KB Single entity / CRUD item ~1.2 µs ~0.9 µs 1.3x
10 KB Small list (50–100 items) ~11.5 µs ~6.1 µs 1.9x
100 KB Bulk batch / export chunk ~118 µs ~42 µs 2.8x
1 MB Large data ingest ~1.24 ms ~340 µs 3.6x

Bottleneck Analysis

  • For small payloads (< 2 KB), the dominant cost is CPython object allocation (PyDict::new, PyList::new, PyString::new and GIL interaction). The raw parsing phase accounts for < 25% of the total time.
  • For large payloads (> 50 KB), the parsing phase dominates, making SIMD vectorization deliver 2.5x–3.6x throughput gains.

3. Architecture & Compatibility Considerations

1. Mutable Buffer Requirement (&mut [u8])

SIMD parsers modify the input slice in place to perform string unescaping and zero-copy slicing.

  • OxyRoute Status: OxyRoute's request pipeline uses PooledBuffer (Vec<u8>), which provides exclusive mutable ownership (&mut [u8]), satisfying the in-place parsing prerequisite.

2. ABI Stability & Wheel Portability

  • Hardcoding AVX-512 or AVX2 at compile time breaks execution on older x86_64 CPUs and ARM architectures (Apple Silicon, AWS Graviton).
  • Runtime feature detection (is_x86_feature_detected!("avx2")) with scalar fallback is required for standard manylinux binary wheels.

3. PyO3 & Python Object Conversion

Regardless of parser speed, converting JSON values into Python objects (PyObject) requires acquiring the GIL. For applications using Pydantic models with model_validate, bypassing intermediate Python dictionaries via direct Pydantic Core C-API bindings yields maximal speedups.

4. Recommendations & Roadmap

  1. Current Production Default: Retain serde_json + OxyRoute's zero-copy json_to_py for universal portability and zero external C/assembly build dependencies.
  2. High-Volume Endpoints: Offer an optional Cargo feature simd for environments with guaranteed AVX2/NEON support.
  3. Pydantic DX Synergy: Pair SIMD parsing with direct Pydantic v2 CoreSchema validation to bypass PyDict allocation on large validated models.