docs: spike findings on prompt caching in Bedrock batch extraction

[noel-improv]

Spike outcome

AWS Bedrock does not support prompt caching with the batch inference API. Caching is on-demand only. The toolkit's batch extractors submit work through CreateModelInvocationJob (S3-in, S3-out, asynchronous), so a cache_control/cachePoint marker in a batch record is inert. The batch-path token-cost or latency win this spike investigated is not achievable.

This tests the central assumption in the DRAFT issue awslabs#326 (Proposed Solution point 2: "Bedrock Batch Inference supports prompt caching when the system prompt is consistent across records"). The spike refutes it. The issue's own Current State note already flagged that the per-record JSONL format has no cache hints.

Verification status

Confirmed two ways. Documentation: the prompt-caching user guide (verbatim: "It is not supported with the batch inference API"), the batch-inference limitations page, and the CreateModelInvocationJob API reference. Empirical: a two-arm test on us.anthropic.claude-sonnet-4-6.

The empirical result is stronger than the docs imply. The on-demand control passed — an InvokeModel call using the exact body shape a batch record carries (cache_control: ephemeral on the first content block) wrote then read ~4,240 cache tokens on a repeated prefix, proving the marker placement is valid. The batch arm then submitted those same records through CreateModelInvocationJob: all 100 failed with errorCode 400 ("your request did not allow prompt caching"), successRecordCount: 0. So batch does not silently drop the cache directive — it rejects the request. The identical body caching on-demand isolates the batch API as the sole cause.

One measured caveat: the effective cache-checkpoint minimum for Sonnet 4.6 behaved like ~4,096 tokens, not the 1,024 the AWS table lists (a ~1,400-token prefix did not cache).

What's in this PR

A findings document at lexical-graph/docs/spikes/prompt-caching-in-batch-extraction.md. No production code change, because there is nothing to add to the batch path.

Follow-up (out of scope here)

Caching is supported on the non-batch / on-demand extraction path (issue awslabs#326 point 1), which this ticket excludes. Two structural facts hold now: the instruction block leads both prompts (a single checkpoint at its end captures the reusable span), and neither prompt contains few-shot examples (the static content is instructions only). Backlog ticket deferred until the team reviews these findings.

Incidental

A pre-existing malformed system-role literal in the Claude batch branch ('role': 'system\"') was found during review and fixed independently in awslabs#331.

References

• AWS — Prompt caching
• AWS — Batch inference
• AWS — CreateModelInvocationJob

[github-actions]

Lexical Graph Coverage Report: The coverage is at 59.68% (target: 80%). Download the HTML report here.

[github-actions]

Lexical Graph Coverage Report: The coverage is at 59.67% (target: 80%). Download the HTML report here.

[github-actions]

Lexical Graph Coverage Report: The coverage is at 59.68% (target: 80%). Download the HTML report here.

[github-actions]

Lexical Graph Coverage Report: The coverage is at 59.68% (target: 80%). Download the HTML report here.

[github-actions]

Lexical Graph Coverage Report: The coverage is at 64.23% (target: 80%). Download the HTML report here.