Skip to content

[C++] Replace RapidJSON with simdjson in JSON chunker #50944

Closed

Description

@Reranko05

Describe the enhancement requested

The JSON chunker currently uses RapidJSON to identify complete JSON values and determine JSON document boundaries when newlines_in_values is enabled.

Replace the RapidJSON-based boundary detection with simdjson while preserving the existing chunking behavior, including handling of:

  • Pretty-printed JSON values containing raw newlines.
  • Multiple JSON values in a single block.
  • JSON values straddling block boundaries.
  • Incomplete trailing JSON values.
  • Invalid JSON input and existing error propagation behavior.

The new implementation should detect structural JSON boundaries and validate complete JSON candidates with simdjson.

Additional context

Parent Issue: GH-35460.

Component(s)

C++

Activity

  1. self-assigned this
    on Aug 21, 2026
  2. added a commit that references this issue on Sep 16, 2026
  3. added this to the 26.0.0 milestone on Sep 16, 2026
  4. pitrou commented on Sep 16, 2026

    @pitrou
    Member

    Issue resolved by pull request 50945
    #50945

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions