You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One document is one job, and inside the job the chunks are extracted one after another (extraction.rs, for chunk in chunks.iter()). A model call takes about 55 seconds on DeepSeek-V3 with the schema.org candidates, so a document's extraction time is chunks × 55 s no matter what the deployment's worker_concurrency or the model's max_concurrent say: those only let more documents run at once.
Measured on the #582 runs, three documents uploaded together: anthropic.txt (27 chunks) 25 min, removal-of-sam-altman-from-openai.txt (23) 21 min, openai.txt (73) 68 min. The two small ones finish and the base then waits three quarters of an hour on one document with two idle workers, while the model gate (10 concurrent by default) sits at one call.
What should happen
The chunks of one document are extracted concurrently, bounded by the model's concurrency gate (model_limits, the same one every other call goes through), and the results are applied in chunk order so that entity resolution and the ledger behave exactly as they do today: resolution is order-sensitive (a later chunk's namesake logic reads what earlier chunks bound, handled_by_name, response_claims), so the model calls are the parallel part and the resolution stays sequential over the replies in sequence order.
Expected effect on the same three documents: bounded by the gate, not by the longest document. With a gate of 10, openai.txt becomes roughly 73 / 10 × 55 s ≈ 7 min of calls plus resolution.
Not this
Not a bigger worker_concurrency: it changes nothing for a single document.
Not parallel resolution: the order-dependent parts stay in order.
Not a change to what is extracted: the same prompt, the same replies, the same ledger; a run before and after on the same documents should differ only in wall-clock time (and the run-to-run variance the model already has).
Acceptance
Three documents uploaded together finish in about the time of the longest document's chunks / gate calls, not chunks calls.
A failed chunk is reported the same way it is today (unextracted, the per-document summary), and a rate-limited call still backs off through the gate rather than retrying in a tight loop.
One document is one job, and inside the job the chunks are extracted one after another (
extraction.rs,for chunk in chunks.iter()). A model call takes about 55 seconds on DeepSeek-V3 with the schema.org candidates, so a document's extraction time ischunks × 55 sno matter what the deployment'sworker_concurrencyor the model'smax_concurrentsay: those only let more documents run at once.Measured on the #582 runs, three documents uploaded together:
anthropic.txt(27 chunks) 25 min,removal-of-sam-altman-from-openai.txt(23) 21 min,openai.txt(73) 68 min. The two small ones finish and the base then waits three quarters of an hour on one document with two idle workers, while the model gate (10 concurrent by default) sits at one call.What should happen
The chunks of one document are extracted concurrently, bounded by the model's concurrency gate (
model_limits, the same one every other call goes through), and the results are applied in chunk order so that entity resolution and the ledger behave exactly as they do today: resolution is order-sensitive (a later chunk's namesake logic reads what earlier chunks bound,handled_by_name,response_claims), so the model calls are the parallel part and the resolution stays sequential over the replies in sequence order.Expected effect on the same three documents: bounded by the gate, not by the longest document. With a gate of 10,
openai.txtbecomes roughly73 / 10 × 55 s ≈ 7 minof calls plus resolution.Not this
worker_concurrency: it changes nothing for a single document.Acceptance
chunks / gatecalls, notchunkscalls.unextracted, the per-document summary), and a rate-limited call still backs off through the gate rather than retrying in a tight loop.Related: #526 (a worker held while the ontology index is built, the other reason a document waits), #583 (the runs that measured this).