Skip to content

The chunks of one document are extracted one at a time #588

Description

@WaylandYang

One document is one job, and inside the job the chunks are extracted one after another (extraction.rs, for chunk in chunks.iter()). A model call takes about 55 seconds on DeepSeek-V3 with the schema.org candidates, so a document's extraction time is chunks × 55 s no matter what the deployment's worker_concurrency or the model's max_concurrent say: those only let more documents run at once.

Measured on the #582 runs, three documents uploaded together: anthropic.txt (27 chunks) 25 min, removal-of-sam-altman-from-openai.txt (23) 21 min, openai.txt (73) 68 min. The two small ones finish and the base then waits three quarters of an hour on one document with two idle workers, while the model gate (10 concurrent by default) sits at one call.

What should happen

The chunks of one document are extracted concurrently, bounded by the model's concurrency gate (model_limits, the same one every other call goes through), and the results are applied in chunk order so that entity resolution and the ledger behave exactly as they do today: resolution is order-sensitive (a later chunk's namesake logic reads what earlier chunks bound, handled_by_name, response_claims), so the model calls are the parallel part and the resolution stays sequential over the replies in sequence order.

Expected effect on the same three documents: bounded by the gate, not by the longest document. With a gate of 10, openai.txt becomes roughly 73 / 10 × 55 s ≈ 7 min of calls plus resolution.

Not this

  • Not a bigger worker_concurrency: it changes nothing for a single document.
  • Not parallel resolution: the order-dependent parts stay in order.
  • Not a change to what is extracted: the same prompt, the same replies, the same ledger; a run before and after on the same documents should differ only in wall-clock time (and the run-to-run variance the model already has).

Acceptance

  • Three documents uploaded together finish in about the time of the longest document's chunks / gate calls, not chunks calls.
  • A failed chunk is reported the same way it is today (unextracted, the per-document summary), and a rate-limited call still backs off through the gate rather than retrying in a tight loop.
  • The extraction ledger and the fact/entity counts on a re-extraction are within the run-to-run variance already measured (A fact quotes the words that name each side #583's table).

Related: #526 (a worker held while the ontology index is built, the other reason a document waits), #583 (the runs that measured this).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions