diff --git a/.amplifier/evaluation/README.md b/.amplifier/evaluation/README.md index 2a8ee490..4200d756 100644 --- a/.amplifier/evaluation/README.md +++ b/.amplifier/evaluation/README.md @@ -5,6 +5,12 @@ single canonical definition per agent and per task. Each trial provisions an isolated Digital Twin Universe (DTU) environment, drives the agent, extracts its work, and grades the result. +Two third-party benchmarks live beside it in their own directories, `deep-swe/` +and `jobbench/`, because each owns a task format, prompt contract, and grading +path that must be reproduced exactly for its scores to mean anything. Both are +described below and each has its own README. All three share the same four +agent arms. + ## Layout ``` @@ -34,6 +40,7 @@ run.py entry point runs/ gitignored per-run outputs deep-swe/ separate harness for the deep-swe benchmark (see below) +jobbench/ separate harness for the JobBench benchmark (see below) ``` ## Task groups @@ -168,6 +175,62 @@ agents installed into pier's venv), and there are several correctness constraint that make numbers comparable or worthless. See `deep-swe/README.md` before running it. +## jobbench + +`jobbench/` is a self-contained harness for +[JobBench](https://github.com/Job-Bench/job-bench-eval), which measures +multi-source knowledge work rather than coding: reconciling contradictory +records, cross-referencing data, tracing citations, across 35 white-collar +occupations. The agent gets a folder of source files and must produce +deliverables (xlsx, docx, pdf, ipynb, sqlite, pptx); an LLM judge scores those +deliverables against a weighted, criterion-level rubric. + +It is separate from the matrix harness above because JobBench owns its own task +format, prompt contract, and judge, all of which must be reproduced exactly for +scores to mean anything. It does use DTUs, one per (agent, task), launched from +a per-agent golden Incus image so a container is warm in about 15 seconds +instead of provisioning from scratch. + +``` +jobbench/run.py entry point (fetch, list-tasks, dtu-check, + bake, run, grade) +jobbench/src/jobbench/ the harness package +jobbench/src/jobbench/agents/ agent adapters +jobbench/src/jobbench/judge.py the JobBench judge, with documented changes +jobbench/profiles/ golden-image bake profiles + trial template +jobbench/tests/ unit tests (synthetic fixtures only) +jobbench/THIRD-PARTY-NOTICES.md Apache-2.0 attribution for the above +``` + +`judge.py` is substantially derived from upstream, and `prompt.py` carries +upstream's prompt template verbatim. Both are Apache-2.0 and stay in-tree +rather than vendored under a separate directory, so the local adaptations +documented in their headers remain diffable against upstream. The license and +the file-by-file attribution are in `jobbench/THIRD-PARTY-NOTICES.md`; the rest +of the harness is MIT under this repo's top-level `LICENSE`. + +The same four arms as the matrix harness: `amplifier-agent`, +`amplifier-foundation`, `opencode-vanilla`, `opencode-amplifier`. + +``` +python run.py fetch --split main +python run.py bake --agent amplifier-agent +python run.py run --agent all --all-tasks --split main --max-parallel 4 +``` + +Two splits: `main` (65 tasks, what the public leaderboard reports, withholds +reference material the agent is expected to find via live web search) and `easy` +(63 tasks, self-contained, a different corpus rather than simplified versions of +the main tasks). Scoring is a weighted rubric score per task, all-or-nothing per +rubric. Results land under gitignored `jobbench/runs/`, not `runs/`. + +The judge defaults to `gpt-5.6-terra` at medium reasoning effort. The JobBench +authors validated their rubrics against `grok-4.3`, so scores produced here are +internally consistent and valid for agent-vs-agent comparison but are not +comparable to the published leaderboard. See `jobbench/README.md` for the +prompt-contract constraint, the baked baseline toolchain, and known issues +before running it. + ## Runtime-fetched data Some task groups store only a selector and pull content on the fly, so no @@ -178,6 +241,9 @@ benchmark content is committed here: (`ScaleAI/SWE-bench_Pro`) and the official scaleapi repo. - `tasks/automation-bench/` stores the `task` name plus `example_id` and fetches the prompt, tool set, seeded world state, and assertions from AutomationBench. +- `jobbench/` stores no task content at all. `run.py fetch` downloads the split + from HuggingFace (`JobBench/job-bench`) into a gitignored local cache, and the + per-run output tree that quotes task and rubric text is gitignored too. For automation-bench the harness clones `zapier/AutomationBench` (pinned commit) exactly once into a machine-local cache under the system temp dir, then extracts diff --git a/.amplifier/evaluation/deep-swe/.gitignore b/.amplifier/evaluation/deep-swe/.gitignore index a9cb2ce7..3ae55bd1 100644 --- a/.amplifier/evaluation/deep-swe/.gitignore +++ b/.amplifier/evaluation/deep-swe/.gitignore @@ -6,3 +6,8 @@ __pycache__/ .ruff_cache/ *.egg-info/ rendered-dockerfiles.txt + +# Spurious: this package declares no dependencies and is installed into pier's +# own venv (`uv pip install -e .`), so a lock file here describes nothing. It +# only appears as a side effect of ad-hoc `uv run` invocations. +uv.lock diff --git a/.amplifier/evaluation/deep-swe/run.py b/.amplifier/evaluation/deep-swe/run.py index 536dd5b7..303485eb 100644 --- a/.amplifier/evaluation/deep-swe/run.py +++ b/.amplifier/evaluation/deep-swe/run.py @@ -426,6 +426,11 @@ def summarize_trial(trial_dir: Path) -> tuple[str, float | None]: name = result.get("task_name") or trial_dir.name agent_result = result.get("agent_result") or {} cost = _num(agent_result.get("cost_usd")) + # Every token processed: fresh input + cache + output. This is additive and + # double-counts nothing because metrics.py normalizes `input_tokens` to + # fresh-only in BOTH branches (the amplifier sources natively fold + # cache_read into it; see `parse_events`). Matches metrics.json's + # `total_tokens`. token_parts = [ _num(agent_result.get(key)) for key in ("n_input_tokens", "n_cache_tokens", "n_output_tokens") diff --git a/.amplifier/evaluation/deep-swe/src/deepswe_agents/base.py b/.amplifier/evaluation/deep-swe/src/deepswe_agents/base.py index 0ef59023..e4088082 100644 --- a/.amplifier/evaluation/deep-swe/src/deepswe_agents/base.py +++ b/.amplifier/evaluation/deep-swe/src/deepswe_agents/base.py @@ -117,12 +117,16 @@ InstallStep(user="root", run="curl -LsSf https://astral.sh/uv/install.sh | sh"), ] +# opencode is installed UNPINNED, on purpose: the benchmark is meant to measure +# opencode as it actually ships today, so a pin here would quietly freeze the +# control arm (and the opencode-backed amplifier arm) at an old build. The +# retry loop is for install-endpoint flakiness only, not for version drift. OPENCODE_PRELUDE = [ InstallStep( user="root", run=( "for i in 1 2 3 4 5; do\n" - " curl -fsSL https://opencode.ai/install | VERSION=1.17.20 bash && break\n" + " curl -fsSL https://opencode.ai/install | bash && break\n" ' echo "opencode install attempt $i failed; retrying in $((i*10))s..." >&2\n' " sleep $((i*10))\n" "done\n" diff --git a/.amplifier/evaluation/deep-swe/src/deepswe_agents/metrics.py b/.amplifier/evaluation/deep-swe/src/deepswe_agents/metrics.py index a67abba7..5656ad1d 100644 --- a/.amplifier/evaluation/deep-swe/src/deepswe_agents/metrics.py +++ b/.amplifier/evaluation/deep-swe/src/deepswe_agents/metrics.py @@ -17,6 +17,35 @@ `not_available` discipline (never fabricate): every normalized field is either a real number or the exact string `"not_available"` -- never a silent 0. + +TOKEN ACCOUNTING. Every token field means exactly one thing, in both branches, +and the four are disjoint so they add up: + + input_tokens fresh input only, never previously cached + cache_read_tokens input served from cache + cache_write_tokens input written into cache + output_tokens generated output + total_tokens the sum of all four: every token actually processed + +Reaching that required normalizing the two sources, which do NOT agree on what +"input" means: + +- opencode's `session.tokens_input` column is already fresh-only. Verified on a + real run: `tokens_input=322` alongside `tokens_cache_read=19,299,708`. +- The amplifier stacks fold cache_read INTO their reported `input_tokens` (but + not cache_write), so `parse_events` subtracts it back out. Verified across + 114 events of a real run: 0/114 had input < cache_read, while 114/114 had + input < cache_read + cache_write -- e.g. `input=872` with `cache_read=0` and + `cache_write=12354`, which only a fresh-plus-cache_read reading explains. + +Why it matters: while `total_tokens` was `input + output`, an opencode trial +reported 95,147 against an amplifier trial's 1,218,757 on the same run -- an +apparent 12x gap that INVERTED the true ordering, since opencode had actually +processed ~19.6M tokens to amplifier's ~10.2M. The old figure silently dropped +opencode's entire 19.3M cache-read volume. + +`cost_usd` is unaffected by any of this: it is priced from the raw per-source +counts against `MODEL_RATES_PER_M`, never derived from `total_tokens`. """ from __future__ import annotations @@ -137,6 +166,10 @@ class _Usage: files_read: int = 0 min_ts: float | None = None max_ts: float | None = None + #: Records where reported input was somehow smaller than cache_read, i.e. + #: the source's convention is not what `parse_events` assumes. Surfaced in + #: `notes` rather than swallowed, because the resulting figure is wrong. + negative_fresh_input: int = 0 def observe_time(self, epoch: float | None) -> None: """Widen the earliest-to-latest span with one timestamp; None is a no-op.""" @@ -158,7 +191,16 @@ def as_dict(self) -> dict[str, Any]: "output_tokens": self.output_tokens, "cache_read_tokens": self.cache_read_tokens, "cache_write_tokens": self.cache_write_tokens, - "total_tokens": self.input_tokens + self.output_tokens, + # Every token the model actually processed. `input_tokens` is + # fresh-only in both branches by construction, so cache_read and + # cache_write are additive here and nothing is double-counted. + "total_tokens": ( + self.input_tokens + + self.cache_read_tokens + + self.cache_write_tokens + + self.output_tokens + ), + "negative_fresh_input": self.negative_fresh_input, "cost_usd": self.cost_usd, "cost_from_events": self.saw_cost, "llm_responses": self.llm_responses, @@ -337,6 +379,11 @@ def parse_opencode_db(db_paths: list[str], workspace_dir: str) -> dict[str, Any] # run with clear usage is never reported as 0 responses. usage.llm_responses += assistant or len(sessions) for s in sessions: + # opencode's `tokens_input` column is already FRESH-ONLY (it + # excludes both cache figures), which is the convention this module + # normalizes to, so it is accumulated as-is. The amplifier branch + # has to strip cache_read out to reach the same meaning; see + # `parse_events`. s_in = _to_int(s.get("tokens_input")) s_out = _to_int(s.get("tokens_output")) s_cr = _to_int(s.get("tokens_cache_read")) @@ -510,14 +557,28 @@ def parse_events(events_paths: list[str]) -> dict[str, Any]: # Field names differ by runtime/provider: amplifier-agent emits the # `_tokens`-suffixed names, the Python Anthropic provider emits the # bare names. Accept either. - usage.input_tokens += _to_int(_pick(event_usage, "input_tokens", "input")) + reported_in = _to_int(_pick(event_usage, "input_tokens", "input")) + ev_cache_read = _to_int(_pick(event_usage, "cache_read_tokens", "cache_read")) + ev_cache_write = _to_int(_pick(event_usage, "cache_write_tokens", "cache_write")) + # The amplifier stacks report an `input_tokens` that ALREADY + # contains cache_read (but not cache_write), while opencode reports + # a fresh-only figure. Strip cache_read here so `input_tokens` means + # exactly one thing -- genuinely-new, never-cached input -- no + # matter which source produced the record. Measured on a real run: + # 114/114 events had input >= cache_read and input < cache_read + + # cache_write, e.g. input=872 with cache_read=0, cache_write=12354. + fresh_in = reported_in - ev_cache_read + if fresh_in < 0: + # The invariant above broke, so the assumption no longer holds + # for this source. Clamp rather than emit a negative token + # count, and say so: a silently wrong number is the failure + # this whole normalization exists to prevent. + usage.negative_fresh_input += 1 + fresh_in = 0 + usage.input_tokens += fresh_in usage.output_tokens += _to_int(_pick(event_usage, "output_tokens", "output")) - usage.cache_read_tokens += _to_int( - _pick(event_usage, "cache_read_tokens", "cache_read") - ) - usage.cache_write_tokens += _to_int( - _pick(event_usage, "cache_write_tokens", "cache_write") - ) + usage.cache_read_tokens += ev_cache_read + usage.cache_write_tokens += ev_cache_write # cost_usd is only emitted by the amplifier-agent stack. Track # whether we ever saw it so a $0 from a stack that does not record # cost is not reported as a real, free run. @@ -721,6 +782,21 @@ def _finalize( if isinstance(record["cost_usd"], (int, float)): record["cost_usd"] = round(float(record["cost_usd"]), 6) + # A source whose input figure was smaller than its own cache_read violates + # the convention `parse_events` normalizes against, so the fresh-input + # figure for those records is a clamped 0 rather than the truth. Say so. + if parsed.get("negative_fresh_input"): + notes = [ + *notes, + ( + f"{parsed['negative_fresh_input']} record(s) reported input_tokens " + "below their own cache_read, which contradicts the " + "fresh-plus-cache_read convention this parser normalizes; fresh " + "input was clamped to 0 for those, so input_tokens is a FLOOR and " + "total_tokens may undercount." + ), + ] + record["source"] = source record["events_files"] = list(source_files) record["notes"] = " ".join([*notes, "total_wallclock_s is not measured by this harness."]) diff --git a/.amplifier/evaluation/deep-swe/tests/test_opencode_metrics.py b/.amplifier/evaluation/deep-swe/tests/test_opencode_metrics.py index 9e493cf3..7a0db832 100644 --- a/.amplifier/evaluation/deep-swe/tests/test_opencode_metrics.py +++ b/.amplifier/evaluation/deep-swe/tests/test_opencode_metrics.py @@ -138,6 +138,8 @@ def test_only_matching_workspace_sessions_are_counted(tmp_path): assert parsed["output_tokens"] == 200 assert parsed["cache_read_tokens"] == 50 assert parsed["cache_write_tokens"] == 75 + # All four are disjoint, so total is their sum: every token processed. + assert parsed["total_tokens"] == 1000 + 200 + 50 + 75 assert parsed["llm_responses"] == 7, "assistant turns from /root must not be counted" @@ -255,3 +257,168 @@ def test_unknown_model_gets_no_fabricated_rate_card(): config = json.loads(agent._opencode_config()) assert "cost" not in config["provider"]["anthropic"]["models"]["some-future-model"] + + +# --------------------------------------------------------------------------- +# Token accounting: the four fields are disjoint and sum to total_tokens +# +# The two sources disagree on what "input" means. opencode's `tokens_input` is +# fresh-only; the amplifier stacks fold cache_read into theirs. Left +# unnormalized with total_tokens = input + output, an opencode trial reported +# 95,147 against an amplifier trial's 1,218,757 on the same run -- an apparent +# 12x gap that inverted the true ordering, because opencode's 19.3M cache reads +# were dropped entirely. These tests pin the normalization on both branches. +# --------------------------------------------------------------------------- + + +def test_opencode_input_is_fresh_only_and_total_sums_all_four(tmp_path): + """opencode's column is already fresh-only, so it is accumulated as-is.""" + db = _make_db( + tmp_path / "opencode.db", + [ + { + "id": "s1", + "directory": "/app", + "tokens_input": 100, + "tokens_output": 50, + "tokens_cache_read": 9_000, + "tokens_cache_write": 900, + } + ], + assistant_turns={"s1": 3}, + ) + + parsed = parse_opencode_db([db], "/app") + + assert parsed["input_tokens"] == 100 + assert parsed["cache_read_tokens"] == 9_000 + assert parsed["cache_write_tokens"] == 900 + assert parsed["total_tokens"] == 100 + 9_000 + 900 + 50 + + +def test_opencode_cost_is_unaffected_by_the_token_normalization(tmp_path): + """Cost bills cache at cache rates and must not track total_tokens.""" + db = _make_db( + tmp_path / "opencode.db", + [ + { + "id": "s1", + "directory": "/app", + "tokens_input": 100, + "tokens_output": 50, + "tokens_cache_read": 9_000, + "tokens_cache_write": 900, + } + ], + assistant_turns={"s1": 1}, + ) + + parsed = parse_opencode_db([db], "/app") + + expected = compute_cost_from_tokens( + "claude-sonnet-5", + input_tokens=100, + output_tokens=50, + cache_read_tokens=9_000, + cache_write_tokens=900, + ) + assert expected is not None + assert parsed["cost_usd"] == pytest.approx(expected) + + +def _events_file(tmp_path, usage: dict, response_id: str = "msg_1"): + events = tmp_path / "events.jsonl" + events.write_text( + json.dumps( + { + "event": "llm:response", + "timestamp": "2026-08-05T15:23:15.602291443+00:00", + "workspace": "-workspace", + "data": { + "session_id": "fa3b1b70-e043-406a-8c8f-1fbc3edd24f0", + "timestamp": "2026-08-05T15:23:15.602291443+00:00", + "model": "claude-sonnet-5", + "provider": "anthropic", + "usage": usage, + "raw": {"id": response_id, "role": "assistant", "content": []}, + }, + } + ) + + "\n", + encoding="utf-8", + ) + return str(events) + + +def test_events_branch_strips_cache_read_out_of_input(tmp_path): + """The amplifier stacks fold cache_read into input; it must come back out. + + Real figures from a run: input=12724 with cache_read=11850 and + cache_write=504, i.e. 874 genuinely-new. Without the subtraction those + 11,850 cached tokens would be counted twice in total_tokens. + """ + from deepswe_agents.metrics import parse_events + + path = _events_file( + tmp_path, + { + "input_tokens": 12_724, + "output_tokens": 72, + "cache_read_tokens": 11_850, + "cache_write_tokens": 504, + }, + ) + + parsed = parse_events([path]) + + assert parsed["input_tokens"] == 12_724 - 11_850 + assert parsed["cache_read_tokens"] == 11_850 + assert parsed["cache_write_tokens"] == 504 + assert parsed["total_tokens"] == 874 + 11_850 + 504 + 72 + + +def test_events_branch_handles_cache_write_larger_than_input(tmp_path): + """cache_write is NOT part of input, so a huge write must not go negative. + + This is the shape that disproved the inclusive-of-everything reading: + input=872 alongside cache_write=12354 on a real first-turn event. + """ + from deepswe_agents.metrics import parse_events + + path = _events_file( + tmp_path, + { + "input_tokens": 872, + "output_tokens": 40, + "cache_read_tokens": 0, + "cache_write_tokens": 12_354, + }, + ) + + parsed = parse_events([path]) + + assert parsed["input_tokens"] == 872 + assert parsed["negative_fresh_input"] == 0 + assert parsed["total_tokens"] == 872 + 0 + 12_354 + 40 + + +def test_events_branch_clamps_and_flags_a_broken_convention(tmp_path): + """If input < cache_read the assumption broke: clamp, never emit a negative.""" + from deepswe_agents.metrics import normalize_metrics, parse_events + + path = _events_file( + tmp_path, + { + "input_tokens": 10, + "output_tokens": 5, + "cache_read_tokens": 500, + "cache_write_tokens": 0, + }, + ) + + parsed = parse_events([path]) + assert parsed["input_tokens"] == 0, "must clamp, not go negative" + assert parsed["negative_fresh_input"] == 1 + + record = normalize_metrics([path], source="test") + assert "clamped to 0" in record["notes"], "a wrong figure must announce itself" diff --git a/.amplifier/evaluation/jobbench/.gitignore b/.amplifier/evaluation/jobbench/.gitignore new file mode 100644 index 00000000..3a66dcd9 --- /dev/null +++ b/.amplifier/evaluation/jobbench/.gitignore @@ -0,0 +1,17 @@ +# Per-run outputs. These contain benchmark task text, agent-produced +# deliverables derived from it, and judge reports that quote rubrics verbatim. +runs/ + +# Dataset downloaded from Hugging Face at setup time. +dataset-cache/ + +# Local scratch and personal analysis tooling. +local/ + +# Python / uv +.venv/ +__pycache__/ +*.pyc + +# ruff +.ruff_cache/ diff --git a/.amplifier/evaluation/jobbench/README.md b/.amplifier/evaluation/jobbench/README.md new file mode 100644 index 00000000..5ccc4bf1 --- /dev/null +++ b/.amplifier/evaluation/jobbench/README.md @@ -0,0 +1,337 @@ +# JobBench harness + +Runs Amplifier agents against **JobBench** inside Digital Twin Universe (DTU) +containers, captures trajectories and token/cost metrics, and grades the +produced deliverables with the JobBench rubric judge. + +## Attribution + +JobBench -- the benchmark, dataset, rubrics, and judge -- is the work of the +JobBench authors: + +- Repository: +- Project site: +- Dataset: + +This directory contains our runner plus our modifications to the JobBench judge +script. The adaptations are documented at the top of `src/jobbench/judge.py`. + +`src/jobbench/judge.py` and the prompt template in `src/jobbench/prompt.py` are +derived from upstream and are licensed Apache-2.0; see `THIRD-PARTY-NOTICES.md` +for the file-by-file attribution and the full license text. The rest of this +harness is MIT under the repository's top-level `LICENSE`. + +Run `python run.py fetch` to download the dataset from Hugging Face before the +first run. + +## What JobBench measures + +Not coding. Multi-source knowledge work: reconciling contradictory records, +cross-referencing data, tracing citations, across 35 white-collar occupations. +The agent receives a folder of source files and must produce deliverables +(xlsx, docx, pdf, ipynb, sqlite, pptx). An LLM judge scores those deliverables +against a weighted, criterion-level rubric. + +``` +main split 65 tasks, 569 rubrics, 2066 criteria, 4500 total weight + includes reference material withheld from the sandbox that the + agent is expected to find via live web search +easy split 63 tasks, self-contained, no withheld material + a different corpus, not simplified versions of the main tasks +``` + +Scoring is all-or-nothing per rubric: full weight only if every criterion passes. + +## Prerequisites + +Host tooling. Every trial runs in a container, so all three are required for +anything past `fetch` and `list-tasks`: + +``` +uv dependency install; also what the bake profiles use + inside the container +incus src/jobbench/images.py shells out to it directly to + publish and delete the golden images +amplifier-digital-twin the DTU CLI (src/jobbench/dtu.py, CLI constant) -- + launch, exec, push, pull, destroy +``` + +Python >=3.11. `uv sync` installs the rest, including the judge's document +extractors (pandas, openpyxl, xlrd, python-pptx, pdfplumber, mammoth). + +Environment variables: + +``` +ANTHROPIC_API_KEY required. Passed into each trial container by + profiles/task.template.yaml passthrough.services; the + value never enters this Python process. +ANTHROPIC_BASE_URL required by the same passthrough block. +OPENAI_API_KEY required to grade. src/jobbench/grading.py reads it and + passes it to the judge as --api-key. Override per + invocation with --judge-api-key. +OPENAI_BASE_URL the judge endpoint, read the same way. Override with + --judge-api-base. +JOBBENCH_CACHE_DIR optional. Moves the dataset cache off the default + dataset-cache/ so one download is shared across + checkouts. +``` + +`src/jobbench/judge.py` also honors `JUDGE_API_BASE` / `JUDGE_API_KEY` / +`JUDGE_MODEL`, but only when invoked standalone. Driven through `run` or +`grade` it is always passed explicit flags, so those three are never consulted +on the normal path. + +## Quick start + +```bash +uv sync +python run.py fetch --split easy # or: --split main +python run.py list-tasks --split easy +python run.py dtu-check # verify the DTU round trip works +python run.py bake --agent amplifier-agent # build the golden image +python run.py run --agent amplifier-agent --task biostatisticians/task1 --split easy +``` + +## Commands + +``` +fetch download a split from Hugging Face into the local cache +list-tasks list tasks in a downloaded split, with rubric counts and weights +dtu-check launch a throwaway DTU and verify exec, push, pull, and destroy +bake build the shared base image and one agent's golden image +run execute an (agents x tasks) matrix, grading each trial +grade re-grade an already-completed run without re-running the agent +``` + +### Running a matrix + +`run` takes any number of agents and tasks and executes the cross product with +bounded concurrency. `--agent` and `--task` are repeatable and also accept +comma-separated values. + +```bash +# every agent against every task in the split +python run.py run --agent all --all-tasks --split main --max-parallel 4 + +# two agents, two tasks +python run.py run --agent amplifier-agent,opencode-vanilla \ + --task biostatisticians/task1 --task biostatisticians/task2 --split easy + +# see the matrix and a cost estimate without launching anything +python run.py run --agent all --all-tasks --split main --dry-run +``` + +There is no checkpoint-resume. Recovery is re-invocation: point `--run-id` at an +existing run directory and add `--skip-existing` to leave finished trials alone +and re-run everything else. + +```bash +python run.py run --agent all --all-tasks --split main \ + --run-id 20260817T153414Z --skip-existing +``` + +Useful flags: `--model`, `--bundle` (amplifier-foundation only), `--timeout`, +`--output-dir`, `--no-grade`, and the `--judge-*` family. + +### Cost and runtime + +Pilot on a single task before committing to a sweep. `--dry-run` prints the +matrix and a cost estimate without launching anything, and that estimate is the +honest one: it comes from `cost_usd` observed in prior runs' own `trial.json` +telemetry, not a pricing table. + +``` +$2.04 - $10.07 per trial, mean ~$6.00 src/jobbench/matrix.py +``` + +That range is agent tokens only. The judge's token usage is captured per rubric +in the grade report but is never priced, so it is not in the figure above and +not in the totals `run` prints. Judge load scales with rubric count (569 across +the main split) and with deliverable size, since every call carries the full +extracted text of every deliverable. + +Wall clock, per trial: + +``` +launch from golden image ~15s, versus provisioning from scratch +agent run bounded by --timeout, default 3600s +grade up to --judge-max-workers rubrics in parallel, + --judge-timeout-per-rubric (default 300s) each +``` + +The full `main` split is 65 tasks; all four agents against all of it is 260 +trials. Divide by `--max-parallel` (default 2) for a rough wall-clock estimate, +but treat the agent-run number as a ceiling rather than a mean -- how long a +trial actually takes varies by task and by agent, and this harness records no +aggregate of it. Baking is separate and out of band: a full toolchain install +can take tens of minutes (`DEFAULT_BAKE_TIMEOUT_S` is 1800s), but it happens +once per agent image, not once per trial. + +### One run per host at a time + +Do not start a second `run.py run` on a host that already has one in flight. +`run` sweeps orphaned DTUs before and after the matrix, and that sweep destroys +every `jb-`-prefixed instance the DTU CLI reports (see +`src/jobbench/orphans.py`). It has no way to tell a leaked container from a +peer run's live one, so a second invocation's pre-run sweep will kill the first +run's trials mid-flight. + +Pass `--no-orphan-sweep` to the second invocation when concurrent runs on one +host are genuinely needed. Leaked containers then have to be cleaned up by +hand. + +### Agents + +``` +amplifier-agent the amplifier-agent CLI +amplifier-foundation amplifier run, on a pinned bundle +opencode-vanilla stock OpenCode, direct to Anthropic +opencode-amplifier OpenCode fronting amplifier-agent +``` + +## How a trial runs + +One DTU per (agent, task), launched from a per-agent golden image so the +container is warm in about 15 seconds rather than provisioning from scratch. + +``` +launch -> seed -> run agent -> pull deliverables -> pull sessions -> destroy +``` + +The agent sees only `task_folder/`. Rubrics, task cards, and any +search-discoverable reference material stay on the host. It writes deliverables +to a dedicated output directory; that directory is what gets graded. + +## Output layout + +``` +runs// + run-manifest.json reproducibility record + /__task/ + trial.json status, exit code, timings, dtu id + launch_profile.yaml exact profile used + prompt.txt exact bytes sent to the agent + agent.log stdout + stderr + metrics.json tokens, cost, agent_run_s + deliverables/ what the agent produced + sessions/ raw agent-native trajectory + grade/_judge.json per-rubric verdicts and score + grade/judge.log judge stdout + stderr +``` + +Trial status is recorded independently of grading, so a crash, a timeout, and a +legitimate zero are distinguishable. `run` prints a per-trial line and a totals +line at the end, and each trial's score is merged back into its own +`trial.json`; a trial that ran but could not be scored carries `grade_error` +there rather than having its failure folded into `status`. There is no +aggregate rollup across trials -- scoring a whole sweep means reading the +per-trial `trial.json` files. + +## Grading + +Judge defaults to `gpt-5.6-terra` at medium reasoning effort over an +OpenAI-compatible endpoint (`OPENAI_BASE_URL`, `OPENAI_API_KEY`). One call per +rubric, carrying the full extracted text of every deliverable. Rubrics whose +text mentions plots or figures additionally get up to 8 images attached. + +The JobBench authors validated their rubrics against `grok-4.3`. Scores produced +with any other judge, including ours, are internally consistent and valid for +agent-vs-agent comparison, but are not comparable to the published leaderboard. + +## Baseline environment + +JobBench does not specify an execution environment for the agent under test. A +fixed toolchain is baked into `profiles/jobbench-base.bake.yaml`, identical for +every agent, and its image alias is recorded in each run manifest. An agent that +must install pandas before it can start work is not being measured on the same +footing as one that cannot. + +The toolchain covers every format the judge can extract text from, since a +deliverable the judge cannot read scores zero regardless of its quality. + +## Known issues + +Read this before trusting a number out of this harness. + +### tool results are lost on the opencode-amplifier arm + +Tool results are sometimes lost crossing the opencode bridge, so the model +repeatedly re-decides it has not read files it already read. The provider +injects a synthetic `[SYSTEM ERROR: Tool result missing from conversation +history]` message and only logs a `logger.warning` +(`amplifier-module-provider-anthropic/__init__.py:2014`); the model then +narrates that error back in its own words, burning wall-clock on a +degenerate loop. Measured on one task: 22 to 27 occurrences per run, and +roughly double the wall-clock of the other three agents, while the trial +still exits 0 and still produces plausible deliverables and a plausible +score. `opencode-vanilla` is the control -- same CLI, same model, talking +directly to Anthropic -- and never exhibits it, so the fault is in the +bridge, not the model. + +Root cause is outside this harness; nothing here works around it. Every +trial scans `agent.log` for the failure signatures +(`_detect_tool_result_loss` in `src/jobbench/trial.py`) and, when found, +appends an entry to `warnings` in that trial's `trial.json`: + +``` +{"kind": "tool_result_loss", "confidence": "direct" | "heuristic", "count": N, "detail": "..."} +``` + +`confidence` is `"direct"` when the literal signature string is found, and +the weaker `"heuristic"` when it is only inferred from the model repeatedly +narrating that it hasn't read something it already read (the literal string +is often never observable, since it lives in the message history the +provider sends the model, not in `agent.log` itself). + +`warnings` is never folded into `status` -- a trial can read `completed` +and still carry this warning. Check `warnings` explicitly before treating an +`opencode-amplifier` trial as clean. See +`profiles/agents/opencode-amplifier.bake.yaml` and +`src/jobbench/agents/opencode_amplifier.py` for the full account. + +### opencode-amplifier reports no cost or token telemetry, ever + +`OpencodeAmplifierAdapter` declares `session_dirs: tuple[str, ...] = ()` +alongside `metrics_source = "events"` (`src/jobbench/agents/ +opencode_amplifier.py`). No location under `/root` was found to hold an +amplifier-agent-style session tree when amplifier-agent is driven through +the opencode wrapper, and the adapter leaves the path empty rather than +guess one. This is deliberate, not a bug: every token and cost field for +this arm reports the exact string `"not_available"`, on every trial, by +construction. Cross-arm cost or token comparisons that include +`opencode-amplifier` cannot be done from this harness's own numbers; the run +summary says so via that same `not_available` string rather than a +fabricated zero. + +### judge cost is captured but never priced + +Already noted under "Cost and runtime" above, repeated here because it is a +trust caveat, not just a cost-estimation footnote: `_extract_usage` / +`_sum_usage` in `src/jobbench/judge.py` capture the judge's token usage into +a `usage` block on every rubric result and on the report, but nothing prices +it -- upstream JobBench discarded `response.usage` entirely, and this +harness only added token capture, not a rate card for the judge model. Judge +cost is in no total `run` or `grade` prints. To see it, read `usage` out of +`grade/_judge.json` yourself. Judge load scales with rubric count and +deliverable size, so it is not necessarily small next to agent cost. + +### scores are not comparable to the published leaderboard + +Already noted under "Grading" above. The JobBench authors validated their +rubrics against `grok-4.3`; this harness defaults to `gpt-5.6-terra`. Scores +produced here are internally consistent and valid for agent-vs-agent +comparison, but are not comparable to the public leaderboard, regardless of +which judge model you point this harness at. + +### opencode-vanilla's cost_usd is recomputed, not opencode's own figure + +For the `opencode-vanilla` arm (`metrics_source = "opencode_db"`), +`cost_usd` is recomputed from that session's token counts against this +harness's own reference rate card, not taken from opencode's self-reported +`cost` column (`src/jobbench/metrics.py`, the `_opencode_model_id` / +`compute_cost_from_tokens` block). The two are logged and can diverge, +since opencode's own figure is priced from a `models.dev` card that differs +on cache rates. If a session's model is not in this harness's rate card, +`cost_usd` for that session is `"not_available"` even though opencode itself +reported a number -- that number is left out because it is not comparable to +the amplifier arms' figures, not because it does not exist. diff --git a/.amplifier/evaluation/jobbench/THIRD-PARTY-NOTICES.md b/.amplifier/evaluation/jobbench/THIRD-PARTY-NOTICES.md new file mode 100644 index 00000000..f2776030 --- /dev/null +++ b/.amplifier/evaluation/jobbench/THIRD-PARTY-NOTICES.md @@ -0,0 +1,247 @@ +# Third-party notices + +This harness includes code derived from a third-party project licensed under +the Apache License, Version 2.0. That code, and the notices required by that +license, are identified below. + +Everything else in this harness is part of the parent repository and is +licensed MIT under the repository's top-level `LICENSE` (Copyright (c) +Microsoft Corporation). + +## JobBench (job-bench-eval) + +``` +Project: JobBench +Source: https://github.com/Job-Bench/job-bench-eval +License: Apache License, Version 2.0 +``` + +Derived files in this harness: + +``` +src/jobbench/judge.py substantially derived from upstream's judge. Modified + by Microsoft Corporation; the modifications are + enumerated in the ADAPTATIONS section of that file's + module docstring. +src/jobbench/prompt.py the prompt template only (the `_TEMPLATE` string, + lines 29-46). Reproduced verbatim from upstream's + eval/run_benchmark_codex_cli.sh (the `prompt_msg` + assignment, lines 350-367) and its two sibling + runners, across which it is byte-identical. Modified in the three + interpolated paths; the wording is unchanged. The + surrounding module is original work. +``` + +The JobBench dataset and rubrics are fetched at runtime from + and are not redistributed +here. See that dataset's own terms. + +## Apache License, Version 2.0 + +The full text of the license, as published at +: + +```text + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +``` diff --git a/.amplifier/evaluation/jobbench/profiles/agents/amplifier-agent.bake.yaml b/.amplifier/evaluation/jobbench/profiles/agents/amplifier-agent.bake.yaml new file mode 100644 index 00000000..be4a231f --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/agents/amplifier-agent.bake.yaml @@ -0,0 +1,33 @@ +# Agent-specific layer on top of the shared jobbench-base image. +# +# Installs ONLY the amplifier-agent CLI. Everything else an agent needs +# (pandas, LibreOffice, uv, node...) already lives in jobbench-base, so this +# profile stays small and fast to re-bake when the agent itself changes. +# +# Deliberately does NOT configure a provider, model, or API key. Secrets +# must never land in a published image layer, and the model is a per-run +# choice -- both get written into the container at trial launch time, not +# baked here. +# +# Not launched directly by trials. See src/jobbench/images.py. +name: jobbench-amplifier-agent +description: > + jobbench-base plus the amplifier-agent CLI, with no provider/model/secret + baked in. + +base: + image: local:jobbench-base + +passthrough: + allow_external: true + +provision: + setup_cmds: + - | + export PATH="$HOME/.local/bin:$PATH" + uv tool install --reinstall --force --from git+https://github.com/microsoft/amplifier-agent amplifier-agent + amplifier-agent-post-install || true + +readiness: + - name: amplifier-agent-installed + command: "amplifier-agent --version" diff --git a/.amplifier/evaluation/jobbench/profiles/agents/amplifier-foundation.bake.yaml b/.amplifier/evaluation/jobbench/profiles/agents/amplifier-foundation.bake.yaml new file mode 100644 index 00000000..738f022b --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/agents/amplifier-foundation.bake.yaml @@ -0,0 +1,40 @@ +# Agent-specific layer on top of the shared jobbench-base image. +# +# Installs the full Amplifier foundation CLI (`amplifier run`) and pre-warms +# the default bundle's module resolution -- `amplifier bundle show` downloads +# every module a bundle composes and needs no API key, so it moves ~30 module +# git clones out of the timed per-trial run and into the bake. If a trial +# overrides the bundle via --bundle, resolution just falls back to happening +# at run time for that trial; it does not fail the bake. +# +# Deliberately does NOT configure a provider, model, or API key. Secrets must +# never land in a published image layer, and both are per-run choices -- +# written into the container at trial launch time instead. See +# src/jobbench/agents/amplifier_foundation.py. +# +# Not launched directly by trials. See src/jobbench/images.py. +name: jobbench-amplifier-foundation +description: > + jobbench-base plus the amplifier CLI, with the default bundle's modules + pre-warmed and no provider/model/secret baked in. + +base: + image: local:jobbench-base + +passthrough: + allow_external: true + +provision: + setup_cmds: + - | + export PATH="$HOME/.local/bin:$PATH" + uv tool install git+https://github.com/microsoft/amplifier + amplifier --version + - | + export PATH="$HOME/.local/bin:$PATH" + amplifier bundle show 'git+https://github.com/microsoft/amplifier-foundation@main#subdirectory=bundles/anchors/bundle.md' >/dev/null 2>&1 || \ + echo "anchors pre-warm failed; modules will resolve at trial time" >&2 + +readiness: + - name: amplifier-installed + command: "amplifier --version" diff --git a/.amplifier/evaluation/jobbench/profiles/agents/opencode-amplifier.bake.yaml b/.amplifier/evaluation/jobbench/profiles/agents/opencode-amplifier.bake.yaml new file mode 100644 index 00000000..8b391694 --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/agents/opencode-amplifier.bake.yaml @@ -0,0 +1,56 @@ +# Agent-specific layer on top of the shared jobbench-base image. +# +# Installs amplifier-agent, stock OpenCode, and amplifier-app-opencode (the +# OpenCode frontend backed by amplifier-agent), plus jq (amplifier-opencode's +# launcher shells out to it). No provider/model/secret baked in. +# +# OpenCode is installed UNPINNED on purpose: the benchmark is meant to measure +# opencode as it actually ships today, so a pin here would quietly freeze this +# arm at an old build. The retry loop around the installer is for +# install-endpoint flakiness only, not for version drift. +# +# Known issue affecting this agent's results: tool results are lost across the +# opencode bridge, so the model repeatedly re-decides it has not read files it +# already read. Trials still exit 0 and produce deliverables, which makes the +# condition easy to miss; the harness flags affected runs via `warnings` in +# trial.json. Root cause is outside this harness. See +# src/jobbench/agents/opencode_amplifier.py. +# +# Not launched directly by trials. See src/jobbench/images.py. +name: jobbench-opencode-amplifier +description: > + jobbench-base plus amplifier-agent, OpenCode (latest at bake time), and + amplifier-app-opencode, with no provider/model/secret baked in. + +base: + image: local:jobbench-base + +passthrough: + allow_external: true + +provision: + setup_cmds: + - | + export PATH="$HOME/.local/bin:$PATH" + uv tool install --with 'httpx>=0.27,<1' git+https://github.com/microsoft/amplifier-agent + - | + for i in 1 2 3 4 5; do + curl -fsSL https://opencode.ai/install | bash && break + echo "opencode install attempt $i failed; retrying in $((i*10))s..." >&2 + sleep $((i*10)) + done + - printf 'export PATH="/root/.opencode/bin:$PATH"\n' > /etc/profile.d/jobbench-opencode-path.sh + - | + export PATH="$HOME/.local/bin:/root/.opencode/bin:$PATH" + uv tool install --from git+https://github.com/microsoft/amplifier-app-opencode amplifier-app-opencode + - | + apt-get update -qq + apt-get install -y --no-install-recommends jq + rm -rf /var/lib/apt/lists/* + - | + export PATH="$HOME/.local/bin:/root/.opencode/bin:$PATH" + amplifier-opencode --help >/dev/null + +readiness: + - name: amplifier-opencode-installed + command: "amplifier-opencode --help >/dev/null && echo ok" diff --git a/.amplifier/evaluation/jobbench/profiles/agents/opencode-vanilla.bake.yaml b/.amplifier/evaluation/jobbench/profiles/agents/opencode-vanilla.bake.yaml new file mode 100644 index 00000000..8b412e45 --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/agents/opencode-vanilla.bake.yaml @@ -0,0 +1,45 @@ +# Agent-specific layer on top of the shared jobbench-base image. +# +# Installs ONLY stock OpenCode, UNPINNED on purpose: the benchmark is meant to +# measure opencode as it actually ships today, so a pin here would quietly +# freeze the control arm at an old build. The installer is known to be flaky +# against its own CDN, hence the retry loop with backoff -- that loop is for +# install-endpoint flakiness only, not for version drift. +# +# A persistent PATH entry is written to /etc/profile.d rather than exported +# inline in provision.setup_cmds: the DTU CLI's own exec wraps every command +# in an outer login shell that sources /etc/profile.d/*.sh (see the same +# convention in amplifier-agent.bake.yaml), so this is what makes `opencode` +# resolve for readiness here AND for every later configure()/command() call +# without each of those repeating the export. +# +# Deliberately does NOT configure a model, provider, or API key -- those are +# per-run choices, written into opencode.json at trial time instead. See +# src/jobbench/agents/opencode_vanilla.py. +# +# Not launched directly by trials. See src/jobbench/images.py. +name: jobbench-opencode-vanilla +description: > + jobbench-base plus stock OpenCode (latest at bake time), with no + model/provider/secret baked in. + +base: + image: local:jobbench-base + +passthrough: + allow_external: true + +provision: + setup_cmds: + - | + for i in 1 2 3 4 5; do + curl -fsSL https://opencode.ai/install | bash && break + echo "opencode install attempt $i failed; retrying in $((i*10))s..." >&2 + sleep $((i*10)) + done + - printf 'export PATH="/root/.opencode/bin:$PATH"\n' > /etc/profile.d/jobbench-opencode-path.sh + - opencode --version + +readiness: + - name: opencode-installed + command: "opencode --version" diff --git a/.amplifier/evaluation/jobbench/profiles/jobbench-base.bake.yaml b/.amplifier/evaluation/jobbench/profiles/jobbench-base.bake.yaml new file mode 100644 index 00000000..b2bbbb20 --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/jobbench-base.bake.yaml @@ -0,0 +1,90 @@ +# Baseline environment every JobBench agent runs in identically. +# +# JobBench does not specify an execution environment for the agent under +# test, so we bake one: a generous document-processing toolchain (pandas, +# docx, xlsx, pdf, pptx, LibreOffice, matplotlib) so an agent is scored on +# its knowledge work, not on whether it remembered to `pip install pandas` +# first. This is the "-bake" half of the golden-image pattern -- built once +# via `python run.py bake --base-only`, then reused by every per-agent bake +# profile (`base.image: local:jobbench-base`) and, eventually, by task +# relaunches. +# +# Not launched directly by trials. See src/jobbench/images.py. +name: jobbench-base +description: > + Shared JobBench workbench toolchain (Python data/doc stack, LibreOffice, + Node.js, uv). Every agent image is built on top of this one so all agents + are measured against the same baseline environment. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + +provision: + setup_cmds: + # System packages. --no-install-recommends keeps LibreOffice's install + # from pulling in language packs and help docs we never use; headless + # conversion and `soffice --version` both work fine without them. + - | + apt-get update + apt-get install -y --no-install-recommends \ + python3 python3-pip python3-venv \ + git curl ca-certificates jq unzip zip \ + sqlite3 fonts-dejavu pandoc \ + libreoffice-writer libreoffice-calc libreoffice-impress + rm -rf /var/lib/apt/lists/* + + # uv, for the agent's own tool installs (agent bake profiles layer on + # top of this image and use it to install themselves). + - curl -LsSf https://astral.sh/uv/install.sh | sh + + # Node.js 22 LTS via NodeSource. Not needed by every agent, but baking + # it here once is cheaper than every agent image repeating the setup. + - | + curl -fsSL https://deb.nodesource.com/setup_22.x | bash - + apt-get install -y nodejs + rm -rf /var/lib/apt/lists/* + + # Python packages installed system-wide (not into a venv) so plain + # `python3` sees them without an agent having to activate anything. + # Ubuntu 24.04 marks the system interpreter PEP 668 + # "externally-managed"; --break-system-packages is the documented + # escape hatch for a container image that is not a general-purpose + # Debian system and will never run `apt install python3-*` again. + # + # --ignore-installed is required alongside it: Ubuntu preinstalls some + # of these transitively (e.g. python3-typing-extensions via + # python3-pip) as dpkg-owned packages with no pip RECORD file. Without + # --ignore-installed, pip tries to uninstall the dpkg-owned version + # before upgrading it and fails with "Cannot uninstall ...; RECORD + # file not found". --ignore-installed skips that uninstall step and + # just lays down the newer pip-managed version on top. + - | + pip3 install --break-system-packages --ignore-installed \ + pandas numpy scipy statsmodels lifelines \ + openpyxl xlsxwriter xlrd \ + python-docx mammoth \ + reportlab fpdf2 pypdf pdfplumber \ + python-pptx \ + matplotlib seaborn Pillow \ + nbformat jupyter-core ipykernel \ + beautifulsoup4 lxml requests httpx tabulate markdown + +readiness: + # One check that actually imports the full stack -- a package that + # installed but fails to import (a missing shared lib, an ABI mismatch) + # is caught here, not three tasks into the first real run. + - name: python-doc-stack + command: >- + python3 -c "import pandas, numpy, scipy, statsmodels, lifelines, + openpyxl, xlsxwriter, xlrd, docx, mammoth, reportlab, fpdf, pypdf, + pdfplumber, pptx, matplotlib, seaborn, PIL, nbformat, bs4, lxml, + requests, httpx, tabulate, markdown; print('ok')" + - name: uv-installed + command: "uv --version" + - name: node-installed + command: "node --version" + - name: libreoffice-installed + command: "soffice --version" diff --git a/.amplifier/evaluation/jobbench/profiles/smoke.yaml b/.amplifier/evaluation/jobbench/profiles/smoke.yaml new file mode 100644 index 00000000..0fa059c3 --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/smoke.yaml @@ -0,0 +1,18 @@ +name: jobbench-smoke +description: > + Minimal Ubuntu 24.04 DTU for exercising the harness's DTU layer end to end + (exec, file-push, file-pull) without any task-specific provisioning. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + +provision: + setup_cmds: + - mkdir -p /workspace + +readiness: + - name: workspace-exists + command: "test -d /workspace && echo OK" diff --git a/.amplifier/evaluation/jobbench/profiles/task.template.yaml b/.amplifier/evaluation/jobbench/profiles/task.template.yaml new file mode 100644 index 00000000..b34e269b --- /dev/null +++ b/.amplifier/evaluation/jobbench/profiles/task.template.yaml @@ -0,0 +1,33 @@ +# Per-trial launch profile TEMPLATE. Not launched directly -- trial.py reads +# this file, substitutes __AGENT_IMAGE__ for the real baked alias (e.g. +# jobbench-amplifier-agent), and writes the result to +# /launch_profile.yaml before calling DTU.launch on it. +# +# base.image uses a literal string placeholder rather than DTU's own +# ${VAR} launch-variable substitution: that substitution is only documented +# to reach provision.setup_cmds, not base.image, so relying on it here would +# be relying on undocumented behavior for the one field that picks the whole +# environment. +name: jobbench-task +description: > + Per-trial environment for one JobBench (agent, task) pair, launched fresh + from a pre-baked agent image with no provider/model/secret in the layer. + +base: + image: local:__AGENT_IMAGE__ + +passthrough: + allow_external: true # tasks may require live web search (main split) + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: anthropic_base_url + key_env: ANTHROPIC_BASE_URL + +provision: + setup_cmds: + - mkdir -p /workspace /workspace/output + +readiness: + - name: output-dir-exists + command: "test -d /workspace/output" diff --git a/.amplifier/evaluation/jobbench/pyproject.toml b/.amplifier/evaluation/jobbench/pyproject.toml new file mode 100644 index 00000000..ab80697b --- /dev/null +++ b/.amplifier/evaluation/jobbench/pyproject.toml @@ -0,0 +1,55 @@ +[project] +name = "jobbench-harness" +version = "0.1.0" +description = "Runs Amplifier agents against JobBench in DTU containers, with trajectory capture, token/cost metrics, and rubric grading." +requires-python = ">=3.11" +dependencies = [ + # Dataset fetch. + "huggingface-hub>=0.30", + # Profile composition and DTU launch profiles. + "pyyaml>=6.0", + # Judge: API client plus the document text extractors it needs. This set + # mirrors the upstream JobBench judge's dependencies -- a deliverable the + # judge cannot read scores zero regardless of its quality. + "openai>=1.0", + "pandas>=2.0", + "openpyxl>=3.1", + "xlrd>=2.0", + "python-pptx>=1.0", + "pdfplumber>=0.11", + "mammoth>=1.8", +] + +# An application driven via run.py, not a distributable package. run.py places +# src/ on sys.path itself. +[tool.uv] +package = false + +[dependency-groups] +dev = [ + "pytest>=8.0", + "pytest-asyncio>=0.23", +] + +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"] + +[tool.pyright] +include = ["src", "tests"] +extraPaths = ["src"] + +[tool.ruff] +line-length = 100 +src = ["src"] + +# These two carry upstream code with documented local adaptations. Keeping +# them close to their upstream shape is what lets a future re-sync be a real +# diff; formatting or "modernizing" them would bury the adaptations in noise. +# Excluded from format and lint rather than edited to satisfy either. +[tool.ruff.lint.per-file-ignores] +"src/jobbench/dtu.py" = ["ALL"] +"src/jobbench/judge.py" = ["ALL"] + +[tool.ruff.format] +exclude = ["src/jobbench/dtu.py", "src/jobbench/judge.py"] diff --git a/.amplifier/evaluation/jobbench/run.py b/.amplifier/evaluation/jobbench/run.py new file mode 100644 index 00000000..21d2e354 --- /dev/null +++ b/.amplifier/evaluation/jobbench/run.py @@ -0,0 +1,809 @@ +#!/usr/bin/env python3 +"""JobBench harness entry point. + +python run.py fetch --split easy +python run.py list-tasks --split easy +python run.py dtu-check +python run.py bake --agent amplifier-agent +python run.py run --agent amplifier-agent --task biostatisticians/task1 --split easy +python run.py run --agent all --all-tasks --split easy --max-parallel 4 --dry-run +python run.py grade runs/20260114T093012Z +""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import shlex +import sys +import tempfile +import uuid +from datetime import UTC, datetime +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parent / "src")) + +from jobbench import dataset, grading, images, matrix, orphans, scheduler, trial +from jobbench import dtu as dtu_mod +from jobbench.dataset import DatasetError, Task +from jobbench.dtu import DTUError +from jobbench.images import ImageError +from jobbench.matrix import MatrixError + +# Reasoning models take an effort knob, not a temperature. Fixed rather than +# an argparse option -- letting it vary silently would make scores across +# runs incomparable for no offsetting benefit (see src/jobbench/judge.py). +JUDGE_REASONING_EFFORT = "medium" + + +def cmd_fetch(args: argparse.Namespace) -> int: + dest = dataset.fetch(args.split, force=args.force) + tasks = dataset.discover(args.split) + total_rubrics = sum(t.rubric_count() for t in tasks) + total_weight = sum(t.max_score() for t in tasks) + print(f"split {args.split}") + print(f"location {dest}") + print(f"revision {dataset.revision(args.split)}") + print(f"tasks {len(tasks)} across {len({t.occupation for t in tasks})} occupations") + print(f"rubrics {total_rubrics} ({total_weight} total weight)") + return 0 + + +def cmd_list_tasks(args: argparse.Namespace) -> int: + tasks = dataset.discover(args.split) + if args.occupation: + tasks = [t for t in tasks if t.occupation == args.occupation] + if not tasks: + print(f"no tasks for occupation {args.occupation!r}", file=sys.stderr) + return 1 + + print(f"{'task':<52} {'rubrics':>7} {'weight':>7} {'inputs':>9} search") + for task in tasks: + size_kb = task.input_bytes() / 1024 + size = f"{size_kb / 1024:.1f}M" if size_kb >= 1024 else f"{size_kb:.0f}K" + print( + f"{task.selector:<52} {task.rubric_count():>7} {task.max_score():>7} " + f"{size:>9} {'yes' if task.has_search_files else 'no'}" + ) + print(f"\n{len(tasks)} tasks, {sum(t.max_score() for t in tasks)} total weight") + return 0 + + +def cmd_bake(args: argparse.Namespace) -> int: + return asyncio.run(_bake(args.agent, force=args.force, base_only=args.base_only)) + + +async def _bake(agent: str, *, force: bool, base_only: bool) -> int: + """Bake jobbench-base, then the agent image on top of it. + + Each stage is skipped (fast) when its image already exists and `force` + wasn't passed, so re-running this after the first successful bake is + close to instant. Progress and elapsed time print per stage since a + full bake (LibreOffice in particular) can take tens of minutes. + """ + print(f"baking {images.BASE_ALIAS} from {images.BASE_PROFILE}...") + try: + base_result = await images.ensure_base_image(force=force) + except ImageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + if base_result.baked: + print(f"{images.BASE_ALIAS}: baked in {base_result.elapsed_s:.0f}s") + else: + print(f"{images.BASE_ALIAS}: already exists, skipped ({base_result.elapsed_s:.1f}s)") + + if base_only: + return 0 + + alias = images.agent_alias(agent) + profile = images.agent_bake_profile(agent) + print(f"baking {alias} from {profile}...") + try: + agent_result = await images.ensure_agent_image(agent, force=force) + except ImageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + if agent_result.baked: + print(f"{alias}: baked in {agent_result.elapsed_s:.0f}s") + else: + print(f"{alias}: already exists, skipped ({agent_result.elapsed_s:.1f}s)") + + return 0 + + +def cmd_dtu_check(args: argparse.Namespace) -> int: + return asyncio.run(_dtu_check()) + + +async def _dtu_check() -> int: + """Live end-to-end round trip through the DTU layer. + + Not a unit test -- this actually launches a container. It exists to catch + the class of bug unit tests can't: a real CLI version whose JSON envelope + shape drifted, or a real file-push/file-pull path that silently does + nothing. Every step prints PASS/FAIL as it happens so a failure is + diagnosable from the log alone. + """ + + def report(name: str, ok: bool, detail: str = "") -> bool: + status = "PASS" if ok else "FAIL" + line = f"[{status}] {name}" + if detail: + line += f" -- {detail}" + print(line) + return ok + + if not dtu_mod.cli_available(): + report("cli-available", False, f"`{dtu_mod.CLI}` is not on PATH") + return 1 + report("cli-available", True) + + profile_path = Path(__file__).parent / "profiles" / "smoke.yaml" + name = f"jb-smoke-{uuid.uuid4().hex[:8]}" + print(f"launching {name} from {profile_path} (cold pulls can take several minutes)...") + try: + instance = await dtu_mod.DTU.launch(profile_path, name=name) + except DTUError as exc: + report("launch", False, str(exc)) + return 1 + report("launch", True, instance.id) + + ok = True + try: + result = await instance.exec_cmd(["bash", "-lc", "echo hello"]) + ok = report( + "exec-basic", + result.returncode == 0 and "hello" in result.stdout, + f"rc={result.returncode} stdout={result.stdout!r}", + ) + if not ok: + return 1 + + # CRITICAL: this is the load-bearing assertion for the whole harness. + # The DTU CLI's `exec` reports the inner command's exit code inside a + # JSON envelope on stdout and exits 0 itself; `_unwrap_exec_envelope` + # is what recovers the real code. If this comes back 0 instead of 7, + # the unwrap is not live and every fail-loud gate built on + # `CommandResult.returncode` is silently checking the wrong layer. + result = await instance.exec_cmd(["bash", "-lc", "exit 7"]) + ok = report( + "exec-envelope-unwrap-exit-7", + result.returncode == 7, + f"rc={result.returncode} (expected 7)" + if result.returncode == 7 + else ( + f"rc={result.returncode} -- BROKEN: inner exit code was not " + "recovered from the JSON envelope; the harness is silently " + "checking the CLI's own exit code instead" + ), + ) + if not ok: + return 1 + + with tempfile.TemporaryDirectory() as tmp: + tmp_path = Path(tmp) + pushed_dir = tmp_path / "push-me" + pushed_dir.mkdir() + (pushed_dir / "marker.txt").write_text("pushed\n", encoding="utf-8") + await instance.file_push(pushed_dir, "/workspace/") + result = await instance.exec_cmd(["bash", "-lc", "cat /workspace/push-me/marker.txt"]) + ok = report( + "file-push-roundtrip", + result.returncode == 0 and "pushed" in result.stdout, + f"rc={result.returncode} stdout={result.stdout!r}", + ) + if not ok: + return 1 + + pull_content = "pulled-content\n" + write_cmd = f"printf %s {shlex.quote(pull_content)} > /workspace/pull-me.txt" + result = await instance.exec_cmd(["bash", "-lc", write_cmd]) + if not report( + "write-file-in-container", + result.returncode == 0, + f"rc={result.returncode} stderr={result.stderr!r}", + ): + return 1 + + pulled_path = tmp_path / "pulled.txt" + await instance.file_pull("/workspace/pull-me.txt", pulled_path) + pulled = pulled_path.read_text(encoding="utf-8") if pulled_path.is_file() else None + ok = report( + "file-pull-roundtrip", + pulled == pull_content, + f"content={pulled!r} (expected {pull_content!r})", + ) + if not ok: + return 1 + + print("all dtu-check steps passed") + return 0 + finally: + print(f"destroying {instance.id}...") + await instance.destroy() + + +def _utc_stamp() -> str: + """Filesystem-safe UTC timestamp for a run directory, e.g. 20260114T093012Z.""" + return datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ") + + +def _print_dry_run( + agent_list: list[str], + tasks: list[Task], + pairs: list[matrix.Pair], + *, + split: str, + max_parallel: int, +) -> None: + """Everything `--dry-run` promises: the matrix, the trial count, and a + cost estimate. Launches nothing -- no image check, no orphan sweep. + """ + print(f"split {split} (revision {dataset.revision(split)})") + print(f"agents {len(agent_list)} {agent_list}") + print(f"tasks {len(tasks)}") + print(f"trials {len(pairs)} ({len(agent_list)} agents x {len(tasks)} tasks)") + print(f"max_parallel {max_parallel}") + print() + print("matrix:") + for pair in pairs: + print(f" {pair.agent:<24} {pair.task.selector}") + print() + low, mean, high = matrix.estimate_cost(len(pairs)) + print( + "cost estimate (derived from prior-run cost_usd observations -- an ESTIMATE, not a quote):" + ) + print(f" ${low:,.2f} - ${high:,.2f} (mean ~${mean:,.2f})") + print() + print("dry run -- nothing launched") + + +def _fmt_num(value: float | str | None) -> str: + if isinstance(value, float): + return f"{value:.1f}" + if value is None: + return "-" + return str(value) + + +def _print_summary( + outcomes: list[scheduler.PairOutcome], *, started_at: str, finished_at: str +) -> None: + """One line per trial (this run's own bookkeeping, straight from each + trial.json -- not the cross-task aggregator), plus totals. + """ + print() + print("results:") + for outcome in outcomes: + score = ( + f"{outcome.total_score}/{outcome.max_score}" if outcome.total_score is not None else "-" + ) + warn_flag = "WARN" if outcome.has_warnings else "" + print( + f" [{outcome.pair.label}] status={outcome.status:<14} " + f"agent_run_s={_fmt_num(outcome.agent_run_s):<8} " + f"cost_usd={_fmt_num(outcome.cost_usd):<8} score={score:<9} {warn_flag}" + ) + + completed = sum(1 for o in outcomes if not o.skipped and o.status == "completed") + skipped = sum(1 for o in outcomes if o.skipped) + failed = sum(1 for o in outcomes if not o.skipped and o.status != "completed") + + # The per-trial lines above print "-" for a missing cost; the total has to + # be just as honest. A whole arm can report no cost telemetry BY + # CONSTRUCTION (opencode-amplifier pulls no telemetry, so every cost field + # is the string "not_available"), which would otherwise make the headline + # cost silently understate the run by one full arm with nothing saying so. + costed = [o.cost_usd for o in outcomes if isinstance(o.cost_usd, int | float)] + total_cost = sum(costed) + uncosted = len(outcomes) - len(costed) + cost_note = ( + f" ({uncosted} of {len(outcomes)} trials reported no cost telemetry)" if uncosted else "" + ) + + elapsed_s = ( + datetime.fromisoformat(finished_at) - datetime.fromisoformat(started_at) + ).total_seconds() + + print() + print( + f"totals {completed} completed, {failed} failed, {skipped} skipped " + f"elapsed={elapsed_s:.0f}s cost=${total_cost:,.2f}{cost_note}" + ) + + +# Every key `_record_judge_attribution` owns. Rewritten (or removed) wholesale +# on each grading pass so a failed re-grade can never leave the PREVIOUS +# judge's run-level claim standing over scores it did not produce. +_JUDGE_ATTRIBUTION_KEYS = ( + "judge_model", + "judge_reasoning_effort", + "grading_complete", + "attempted_judge_model", + "attempted_judge_reasoning_effort", + "ungraded_trials", +) + + +def _record_judge_attribution(manifest: dict, *, judge_model: str, ungraded: int) -> None: + """Record which judge graded this run, honestly. + + A top-level `judge_model` is a claim that this judge produced EVERY score + in the run, so it is written only when `ungraded` is 0 -- i.e. every trial + in the matrix got a score from this judge on this pass. Otherwise the + judge is recorded as `attempted_judge_model` alongside `grading_complete: + false` and the count of trials it produced no score for, so a partial + grading pass is visible rather than indistinguishable from a clean one. + + Per-score provenance is not this function's job: each trial.json carries + its own `judge_model` next to its score (see grading.grade_and_record). + """ + for key in _JUDGE_ATTRIBUTION_KEYS: + manifest.pop(key, None) + manifest["grading_complete"] = ungraded == 0 + if ungraded == 0: + manifest["judge_model"] = judge_model + manifest["judge_reasoning_effort"] = JUDGE_REASONING_EFFORT + else: + manifest["attempted_judge_model"] = judge_model + manifest["attempted_judge_reasoning_effort"] = JUDGE_REASONING_EFFORT + manifest["ungraded_trials"] = ungraded + + +def cmd_run(args: argparse.Namespace) -> int: + return asyncio.run(_run(args)) + + +async def _run(args: argparse.Namespace) -> int: + """Run an (agent, task) matrix -- one pair by default, many under + --agent/--task expansion -- landing each trial's artifacts under + --output-dir exactly as a single-pair run always has. + + Everything one trial itself does (launch, seed, execute, pull, destroy) + lives in jobbench.trial; concurrency, skip-existing, and per-pair + bookkeeping live in jobbench.scheduler. This function is the CLI-facing + shell: resolve and validate the WHOLE matrix before anything launches, + check every agent's image is baked, lay out the run directory, sweep + orphaned DTUs before and after, and print progress/summary. + """ + try: + agent_list = matrix.resolve_agent_names(args.agent) + tasks = matrix.resolve_tasks(split=args.split, raw=args.task, all_tasks=args.all_tasks) + except MatrixError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + # --bundle is an amplifier-foundation-specific concept (which bundle.md + # `amplifier run` composes). Rejecting it for every other agent here, + # rather than letting it fall through to a TypeError from the adapter's + # own __init__, keeps the error message about the flag, not about + # Python's constructor mismatch. Checked against the WHOLE agent set + # up front, same as the agent/task validation above. + if args.bundle is not None: + non_foundation = [a for a in agent_list if a != "amplifier-foundation"] + if non_foundation: + print( + f"error: --bundle is only supported by amplifier-foundation, not {non_foundation}", + file=sys.stderr, + ) + return 2 + + def agent_kwargs_for(agent: str) -> dict[str, str]: + if agent == "amplifier-foundation" and args.bundle is not None: + return {"bundle": args.bundle} + return {} + + pairs = matrix.build_matrix(agent_list, tasks) + + if args.dry_run: + _print_dry_run(agent_list, tasks, pairs, split=args.split, max_parallel=args.max_parallel) + return 0 + + # Every image must be baked before the first trial launches -- a missing + # image for the 3rd of 4 agents must fail instantly, not after the first + # two agents' tasks have already run for hours. + missing = [ + (agent, images.agent_alias(agent)) + for agent in agent_list + if not await images.image_exists(images.agent_alias(agent)) + ] + if missing: + for agent, alias in missing: + print( + f"error: image {alias!r} is not baked; run " + f"`python run.py bake --agent {agent}` first", + file=sys.stderr, + ) + return 1 + + output_dir = Path(args.output_dir) + run_id = args.run_id or _utc_stamp() + run_root = output_dir / run_id + run_root.mkdir(parents=True, exist_ok=True) + + print(f"run {run_id}" + (" (reusing existing run dir)" if args.run_id else "")) + print(f"agents {agent_list}") + print(f"split {args.split} (revision {dataset.revision(args.split)})") + print(f"matrix {len(pairs)} trial(s) ({len(agent_list)} agents x {len(tasks)} tasks)") + print(f"max_parallel {args.max_parallel}") + print(f"model {args.model}") + print(f"timeout {args.timeout:.0f}s") + print(f"output dir {run_root}") + if args.skip_existing: + print("skip-existing on") + print() + + started_at = datetime.now(UTC).isoformat() + + # Sweep BEFORE the matrix starts -- none of OUR trials is running yet, so + # any jb-prefixed instance the CLI reports looks like a leak from a prior + # hard-killed run. It cannot tell that apart from a container a peer + # harness process on this host is using, which is what --no-orphan-sweep + # is for. See jobbench.orphans. + if args.orphan_sweep: + reaped_before = await orphans.sweep_orphans() + if reaped_before: + print( + f"orphan sweep (pre-run): destroyed {len(reaped_before)} " + f"leaked DTU(s): {reaped_before}" + ) + print() + else: + print("orphan sweep disabled (--no-orphan-sweep); leaked DTUs must be reaped by hand") + print() + + judge_api_base, judge_api_key = (None, None) + if args.grade: + judge_api_base, judge_api_key = grading.resolve_credentials( + args.judge_api_base, args.judge_api_key + ) + + outcomes = await scheduler.run_matrix( + pairs, + run_root, + model=args.model, + timeout_s=args.timeout, + max_parallel=args.max_parallel, + agent_kwargs_for=agent_kwargs_for, + skip_existing=args.skip_existing, + grade=args.grade, + judge_model=args.judge_model, + judge_api_base=judge_api_base, + judge_api_key=judge_api_key, + judge_max_workers=args.judge_max_workers, + judge_timeout_per_rubric=args.judge_timeout_per_rubric, + ) + + finished_at = datetime.now(UTC).isoformat() + + # Sweep AFTER the matrix finishes -- every one of our own trials has + # already run its own destroy() by this point, so anything still + # jb-prefixed here is a leak from a trial that never reached `finally` + # (or, if a peer harness process is running on this host, one of ITS live + # containers -- hence --no-orphan-sweep). + if args.orphan_sweep: + reaped_after = await orphans.sweep_orphans() + if reaped_after: + print() + print( + f"orphan sweep (post-run): destroyed {len(reaped_after)} " + f"leaked DTU(s): {reaped_after}" + ) + + manifest = { + "run_id": run_id, + "agents": agent_list, + "tasks": [t.selector for t in tasks], + "matrix_size": len(pairs), + "max_parallel": args.max_parallel, + "model": args.model, + "split": args.split, + "dataset_revision": dataset.revision(args.split), + "image_aliases": {a: images.agent_alias(a) for a in agent_list}, + "timeout_s": args.timeout, + "network_policy": {"allow_external": True}, + "skip_existing": args.skip_existing, + "started_at": started_at, + "finished_at": finished_at, + } + if len(agent_list) == 1: + # Back-compat with `cmd_grade`, which keys off a single "agent" + # string -- a single-agent manifest (any number of tasks) stays + # re-gradable through the existing `grade` subcommand unchanged. + manifest["agent"] = agent_list[0] + if "amplifier-foundation" in agent_list: + # Record what actually ran, not just what was overridden -- a run + # left at the default is otherwise invisible in the manifest, and + # the default is a moving `@main` ref that can change between runs. + from jobbench.agents.amplifier_foundation import DEFAULT_BUNDLE + + manifest["bundle"] = args.bundle or DEFAULT_BUNDLE + if args.grade: + # A skipped pair was NOT graded on this pass -- it keeps whatever + # score (and judge) a previous pass gave it, which may not be this + # judge -- so it counts as ungraded here just like a grading failure + # does. Either one means this judge did not produce every score in + # the run, and the manifest must not claim otherwise. + ungraded = sum(1 for o in outcomes if o.skipped or not o.graded_ok) + _record_judge_attribution(manifest, judge_model=args.judge_model, ungraded=ungraded) + (run_root / "run-manifest.json").write_text( + json.dumps(manifest, indent=2) + "\n", encoding="utf-8" + ) + + _print_summary(outcomes, started_at=started_at, finished_at=finished_at) + + # A skipped pair contributes neither a pass nor a fail; only pairs that + # actually ran THIS invocation determine the exit code. + ran = [o for o in outcomes if not o.skipped] + all_ok = all(o.status == "completed" and o.graded_ok for o in ran) + return 0 if all_ok else 1 + + +def cmd_grade(args: argparse.Namespace) -> int: + """Grade every trial in an already-completed run, without re-running the agent. + + Reads run-manifest.json to find the run's agent/split/tasks, then grades + each trial directory in place -- the same code path `run --grade` uses, + so a re-graded run and a graded-at-run-time run are scored identically. + + Only supports single-agent run directories (the shape `run` has always + written for one agent, regardless of task count). A multi-agent matrix + run is already graded per-trial as part of `run` itself (see + jobbench.scheduler) -- there is nothing left for this command to do for + those runs, so it fails loudly rather than guessing which agent's trials + to re-grade. + """ + run_dir = Path(args.run_dir) + manifest_path = run_dir / "run-manifest.json" + if not manifest_path.is_file(): + print(f"error: no run-manifest.json under {run_dir}", file=sys.stderr) + return 2 + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + + if "agent" not in manifest: + print( + f"error: {run_dir} is a multi-agent matrix run (agents={manifest.get('agents')}); " + "`grade` only supports single-agent run directories -- each trial in a matrix run " + "is already graded during `run` itself", + file=sys.stderr, + ) + return 2 + + agent = manifest["agent"] + split = manifest.get("split", "main") + api_base, api_key = grading.resolve_credentials(args.api_base, args.api_key) + + print(f"run {run_dir}") + print(f"agent {agent}") + print(f"judge model {args.judge_model} (reasoning_effort={JUDGE_REASONING_EFFORT})") + print() + + ungraded = 0 + all_ok = True + for task_selector in manifest.get("tasks", []): + task = dataset.resolve(split, [task_selector])[0] + trial_dir = run_dir / agent / task.id + if not trial_dir.is_dir(): + print( + f"warning: no trial directory for {task_selector} at {trial_dir}", file=sys.stderr + ) + ungraded += 1 + all_ok = False + continue + ok = grading.grade_and_record( + trial_dir, + task, + agent=agent, + judge_model=args.judge_model, + api_base=api_base, + api_key=api_key, + max_workers=args.max_workers, + timeout_per_rubric=args.timeout_per_rubric, + ) + ungraded += 0 if ok else 1 + all_ok = all_ok and ok + + # Only claim this judge graded the run when it actually graded every + # trial. A trial whose grading failed now holds no score at all (see + # grading.grade_and_record), so an unconditional claim here would attribute + # the REMAINING trials' scores -- some possibly from a previous judge -- to + # this one. + _record_judge_attribution(manifest, judge_model=args.judge_model, ungraded=ungraded) + manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") + + return 0 if all_ok else 1 + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(prog="run.py", description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + + def add_split(p: argparse.ArgumentParser) -> None: + p.add_argument( + "--split", + default="main", + choices=sorted(dataset.SPLITS), + help="which split to operate on (default: main)", + ) + + p_fetch = sub.add_parser("fetch", help="download a split from Hugging Face") + add_split(p_fetch) + p_fetch.add_argument("--force", action="store_true", help="re-download even if present") + p_fetch.set_defaults(func=cmd_fetch) + + p_list = sub.add_parser("list-tasks", help="list tasks in a downloaded split") + add_split(p_list) + p_list.add_argument("--occupation", help="restrict to one occupation") + p_list.set_defaults(func=cmd_list_tasks) + + p_dtu_check = sub.add_parser( + "dtu-check", help="live end-to-end DTU round trip (launches a real container)" + ) + p_dtu_check.set_defaults(func=cmd_dtu_check) + + p_bake = sub.add_parser( + "bake", help="bake the golden Incus images (jobbench-base, then an agent)" + ) + p_bake.add_argument("--agent", required=True, help="agent name, e.g. amplifier-agent") + p_bake.add_argument("--force", action="store_true", help="re-bake even if the image exists") + p_bake.add_argument( + "--base-only", action="store_true", help="bake jobbench-base only, skip the agent image" + ) + p_bake.set_defaults(func=cmd_bake) + + p_run = sub.add_parser( + "run", help="run an (agent, task) matrix in fresh DTUs, bounded by --max-parallel" + ) + p_run.add_argument( + "--agent", + action="append", + metavar="NAME", + required=True, + help=( + "agent name, e.g. amplifier-agent (repeatable: --agent a --agent b; comma-separated " + "ok: --agent a,b); 'all' expands to every registered agent" + ), + ) + p_run.add_argument( + "--task", + action="append", + metavar="SELECTOR", + help=( + "task selector, e.g. biostatisticians/task1 (repeatable, comma-separated ok); " + "required unless --all-tasks is given" + ), + ) + p_run.add_argument( + "--all-tasks", + action="store_true", + help="run every task in --split instead of listing --task selectors", + ) + add_split(p_run) + p_run.add_argument( + "--model", default="claude-sonnet-5", help="model to configure the agent with" + ) + p_run.add_argument( + "--bundle", + default=None, + help=( + "override the bundle amplifier-foundation runs (default: its own anchors " + "default); rejected for every other agent, which has no bundle concept" + ), + ) + p_run.add_argument( + "--timeout", + type=float, + default=trial.DEFAULT_TIMEOUT_S, + help="agent wall-clock timeout, seconds", + ) + p_run.add_argument("--output-dir", default="runs", help="root directory for run artifacts") + p_run.add_argument( + "--max-parallel", + type=int, + default=2, + help="max concurrent trials (default: 2; each trial launches a full container)", + ) + p_run.add_argument( + "--run-id", + default=None, + help=( + "reuse/extend an existing run directory under --output-dir (default: a fresh " + "UTC timestamp) -- the way to top up a partial sweep with --skip-existing" + ), + ) + p_run.add_argument( + "--skip-existing", + action="store_true", + help="skip (agent, task) pairs whose trial.json already has status=completed", + ) + p_run.add_argument( + "--dry-run", + action="store_true", + help="print the matrix, trial count, and a cost estimate, then exit -- launches nothing", + ) + p_run.add_argument( + "--no-orphan-sweep", + dest="orphan_sweep", + action="store_false", + default=True, + help=( + "skip the pre-run and post-run sweeps of leaked jb- DTUs; use this when another " + "harness process is running on the same host, since the sweep cannot tell that " + "process's live containers from leaks and would destroy them (leaked DTUs then " + "have to be reaped by hand)" + ), + ) + _add_judge_args(p_run, flag_prefix="judge-", dest_prefix="judge_") + p_run.add_argument( + "--grade", + action=argparse.BooleanOptionalAction, + default=True, + help="grade each trial's deliverables against its task rubric after it finishes (default: on)", + ) + p_run.set_defaults(func=cmd_run) + + p_grade = sub.add_parser( + "grade", help="grade every trial in an already-completed run, without re-running the agent" + ) + p_grade.add_argument("run_dir", help="run directory, e.g. runs/20260114T093012Z") + _add_judge_args(p_grade, flag_prefix="", dest_prefix="") + p_grade.set_defaults(func=cmd_grade) + + return parser + + +def _add_judge_args(p: argparse.ArgumentParser, *, flag_prefix: str, dest_prefix: str) -> None: + """Judge configuration flags shared by `run --grade` and `grade`. + + `run` namespaces these under `--judge-*` / `args.judge_*` so they read + clearly next to the agent's own `--model`; `grade`, where judging is the + only thing happening, uses the bare names. + """ + p.add_argument( + f"--{flag_prefix}model", + # Always `judge_model`, even for `run` where the agent's own `--model` + # already owns the bare `model` dest -- this is never that model. + dest="judge_model", + default=grading.DEFAULT_JUDGE_MODEL, + help=f"judge model (default: {grading.DEFAULT_JUDGE_MODEL})", + ) + p.add_argument( + f"--{flag_prefix}api-base", + dest=f"{dest_prefix}api_base", + default=None, + help="judge API base URL (default: $OPENAI_BASE_URL)", + ) + p.add_argument( + f"--{flag_prefix}api-key", + dest=f"{dest_prefix}api_key", + default=None, + help="judge API key (default: $OPENAI_API_KEY)", + ) + p.add_argument( + f"--{flag_prefix}max-workers", + dest=f"{dest_prefix}max_workers", + type=int, + default=grading.DEFAULT_MAX_WORKERS, + help=f"parallel rubric judge calls (default: {grading.DEFAULT_MAX_WORKERS})", + ) + p.add_argument( + f"--{flag_prefix}timeout-per-rubric", + dest=f"{dest_prefix}timeout_per_rubric", + type=int, + default=grading.DEFAULT_TIMEOUT_PER_RUBRIC, + help=f"per-rubric judge call timeout, seconds (default: {grading.DEFAULT_TIMEOUT_PER_RUBRIC})", + ) + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + try: + return int(args.func(args)) + except DatasetError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/.amplifier/evaluation/jobbench/src/jobbench/__init__.py b/.amplifier/evaluation/jobbench/src/jobbench/__init__.py new file mode 100644 index 00000000..564d2dd5 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/__init__.py @@ -0,0 +1 @@ +"""JobBench harness: run Amplifier agents against JobBench in DTU containers.""" diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/__init__.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/__init__.py new file mode 100644 index 00000000..464e330d --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/__init__.py @@ -0,0 +1,19 @@ +"""Agent-under-test adapters, registered by import. + +Each adapter module self-registers via `@register` at import time (see +base.py). Importing this package eagerly imports every known adapter module +so `get`/`names` see the full set without callers needing to know which +modules exist. +""" + +from __future__ import annotations + +from jobbench.agents import ( # noqa: F401 + amplifier_agent, + amplifier_foundation, + opencode_amplifier, + opencode_vanilla, +) +from jobbench.agents.base import Adapter, AdapterError, get, names + +__all__ = ["Adapter", "AdapterError", "get", "names"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_agent.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_agent.py new file mode 100644 index 00000000..612d52b7 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_agent.py @@ -0,0 +1,91 @@ +"""The amplifier-agent adapter. + +Drives amplifier-agent's own CLI non-interactively inside a DTU, the same +contract deep-swe's amplifier-agent runner uses: a host-config JSON pinning +approval mode and provider/model, a session id, and the prompt delivered via +`$(cat ...)` rather than string interpolation. + +The bake profile (profiles/agents/amplifier-agent.bake.yaml) installs the CLI +with no provider, model, or secret baked in -- those are per-run choices, so +they get written here, at trial time, instead. +""" + +from __future__ import annotations + +import json +import shlex +import uuid + +from jobbench import images +from jobbench.agents.base import Adapter, AdapterError, register +from jobbench.dtu import DTU + +DEFAULT_MODEL = "claude-sonnet-5" +HOST_CONFIG_PATH = "/root/host-config.json" + +# Defense-in-depth PATH write, matching the precedent in +# agents/amplifier-agent-local/install.yaml: the DTU CLI's own login shell +# already sources /etc/profile.d/dtu-env.sh (which includes /root/.local/bin, +# where `uv tool install` places amplifier-agent), but the agent under test +# should not depend on that engine behavior to find its own binary. +_PATH_PROFILE_SCRIPT = "/etc/profile.d/jobbench-amplifier-agent.sh" + + +@register +class AmplifierAgentAdapter(Adapter): + name = "amplifier-agent" + image_alias = images.agent_alias("amplifier-agent") + session_dirs = ("/root/.amplifier-agent/state/workspaces",) + metrics_source = "events" + + def __init__(self) -> None: + # Generated once per adapter instance (trial.py calls agents.get() + # fresh per trial) so `command()` can embed it without taking + # arguments -- the contract has no other place to source it from. + self._session_id = uuid.uuid4().hex[:12] + + async def configure(self, dtu: DTU, *, model: str) -> None: + """Write the per-trial host-config and guarantee PATH. + + No API key is written here -- ANTHROPIC_API_KEY reaches the + container through the launch profile's `passthrough.services`, so + it never touches disk in plaintext under our control. + """ + config = { + "approval": {"mode": "yes"}, + "provider": {"module": "anthropic", "config": {"default_model": model}}, + } + payload = json.dumps(config) + script = ( + f"printf %s {shlex.quote(payload)} > {HOST_CONFIG_PATH} && " + f"printf 'export PATH=\"/root/.local/bin:$PATH\"\\n' > {_PATH_PROFILE_SCRIPT}" + ) + result = await dtu.exec_cmd(["bash", "-c", script]) + if result.returncode != 0: + raise AdapterError( + f"amplifier-agent configure failed (exit {result.returncode}): " + f"{result.stderr.strip() or result.stdout.strip()}" + ) + + def command(self) -> list[str]: + """argv equivalent of: + + cd /workspace && amplifier-agent run -y --config /root/host-config.json \\ + --session-id --output json "$(cat /workspace/prompt.txt)" + + Run through `bash -c` (not `-lc`) because it needs shell constructs + (`&&`, `$(...)`) in one command; the DTU CLI's `exec` already wraps + every command in an outer login shell, so this is not a double wrap. + The prompt is substituted via `$(cat ...)` inside that single quoted + script, never interpolated into the Python string -- task prompts + contain quotes and newlines that would otherwise corrupt the argv. + """ + script = ( + "cd /workspace && amplifier-agent run -y " + f"--config {HOST_CONFIG_PATH} --session-id {self._session_id} " + '--output json "$(cat /workspace/prompt.txt)"' + ) + return ["bash", "-c", script] + + +__all__ = ["DEFAULT_MODEL", "AmplifierAgentAdapter"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_foundation.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_foundation.py new file mode 100644 index 00000000..b76f15fb --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/amplifier_foundation.py @@ -0,0 +1,126 @@ +"""The amplifier-foundation adapter (full `amplifier run` CLI, a pinned bundle). + +Ports deep-swe's AmplifierFoundationAgent +(../deep-swe/src/deepswe_agents/amplifier_foundation.py) to jobbench's Adapter +contract. Two differences from that reference, both load-bearing: + +1. deep-swe's workdir is /app; ours is /workspace (see jobbench.prompt). +2. deep-swe writes settings.yaml through pier's own env-dict exec, so the + secret rides in a Python-side env mapping that never touches argv. jobbench's + DTU.exec_cmd has no such hook -- it only shells out to the DTU CLI locally, + with no channel to inject env into the remote container process. The same + unquoted-heredoc trick is used instead, but the ${...} references expand + against the CONTAINER's own environment (already populated by the launch + profile's passthrough.services), not a dict we control. Either way the + secret value itself never appears in our argv or logs -- only the literal + `${ANTHROPIC_API_KEY}` reference does. + +The bake profile (profiles/agents/amplifier-foundation.bake.yaml) installs the +CLI and pre-warms the default bundle's module resolution, with no provider, +model, or secret baked in -- those are per-run choices, written here instead. +""" + +from __future__ import annotations + +from jobbench import images +from jobbench.agents.base import Adapter, AdapterError, register +from jobbench.dtu import DTU + +SETTINGS_PATH = "$HOME/.amplifier/settings.yaml" + +# anchors is deep-swe's coding-oriented default and may not be the right +# bundle for JobBench's knowledge-work tasks. Overridable two ways: the +# adapter constructor kwarg below (`agents.get("amplifier-foundation", +# bundle=...)`) and `run.py run --bundle`, which resolves to this kwarg. +DEFAULT_BUNDLE = ( + "git+https://github.com/microsoft/amplifier-foundation@main" + "#subdirectory=bundles/anchors/bundle.md" +) + +# Unquoted heredoc: ${...} expands INSIDE the container against its own +# environment, so the secret value never crosses into our Python process, +# argv, or logs -- only the literal reference does. +# +# provider-anthropic reads base_url from CONFIG ONLY; it has no runtime env +# fallback. Omitting this key would silently send this arm to +# api.anthropic.com while every other arm hits the configured proxy, i.e. +# benchmarking a different endpoint. The `:-https://api.anthropic.com` default +# only fires if a launch profile forgot to pass ANTHROPIC_BASE_URL through. +# +# No routing.matrix key here (re-introduces role-based fan-out to a different +# model) and no bundle: key (the run command pins the bundle explicitly, so a +# stray bundle: entry here would never be read anyway). +_SETTINGS_TEMPLATE = """config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key: ${ANTHROPIC_API_KEY} + base_url: ${ANTHROPIC_BASE_URL:-https://api.anthropic.com} + default_model: __MODEL__ + enable_1m_context: 'true' + enable_prompt_caching: 'true' + priority: 1 +""" + +_HEREDOC_MARKER = "JOBBENCH_SETTINGS_EOF" + + +@register +class AmplifierFoundationAdapter(Adapter): + name = "amplifier-foundation" + image_alias = images.agent_alias("amplifier-foundation") + + # `amplifier run` writes one session tree per project slug (the cwd with + # separators replaced). Collecting the whole projects/ parent rather than + # computing one slug survives a slug surprise and matches deep-swe's own + # choice. A bundle whose session composes more than one logging hook (the + # default anchors bundle does: hooks-logging + context-intelligence + # logging) writes events.jsonl twice; that is de-duplicated by + # metrics.parse_events on response identity, not by which file we collect. + session_dirs = ("/root/.amplifier/projects",) + metrics_source = "events" + + def __init__(self, *, bundle: str = DEFAULT_BUNDLE) -> None: + self._bundle = bundle + + async def configure(self, dtu: DTU, *, model: str) -> None: + """Write settings.yaml at TRIAL time, never at bake time. + + A bake-time write becomes an image layer; baking the API key there + would put the secret in the image. The model is also a per-run + choice, so it belongs here too, not in the bake profile. + """ + settings = _SETTINGS_TEMPLATE.replace("__MODEL__", model) + script = ( + 'mkdir -p "$HOME/.amplifier" && ' + f'cat > "{SETTINGS_PATH}" <<{_HEREDOC_MARKER}\n' + f"{settings}" + f"{_HEREDOC_MARKER}" + ) + result = await dtu.exec_cmd(["bash", "-c", script]) + if result.returncode != 0: + raise AdapterError( + f"amplifier-foundation configure failed (exit {result.returncode}): " + f"{result.stderr.strip() or result.stdout.strip()}" + ) + + def command(self) -> list[str]: + """argv equivalent of: + + cd /workspace && amplifier run --bundle '' --mode single \\ + --output-format json "$(cat /workspace/prompt.txt)" + + No --model flag: it requires --provider alongside it, and the model + is already pinned by default_model in settings.yaml. + """ + script = ( + "cd /workspace && " + f"amplifier run --bundle '{self._bundle}' " + "--mode single --output-format json " + '"$(cat /workspace/prompt.txt)"' + ) + return ["bash", "-c", script] + + +__all__ = ["DEFAULT_BUNDLE", "AmplifierFoundationAdapter"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/base.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/base.py new file mode 100644 index 00000000..ab6b630a --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/base.py @@ -0,0 +1,117 @@ +"""Agent-under-test adapter contract for JobBench trials. + +trial.py owns the DTU lifecycle (launch, seed, run, pull, destroy) and is +agent-agnostic. Everything specific to one agent's CLI -- how it takes a +config, how it wants the prompt handed to it, where its session state lives +-- is isolated behind this Adapter contract so a new agent (opencode, +claude-code, codex) is a new module in this package, not a change to +trial.py. + +`session_dirs` and `metrics_source` are part of the contract now even though +nothing consumes them yet (that lands with telemetry/grading in a later +phase). Declaring the shape up front means adding those phases won't require +every adapter's public surface to change. +""" + +from __future__ import annotations + +from abc import ABC, abstractmethod +from typing import Any + +from jobbench.dtu import DTU + + +class AdapterError(RuntimeError): + """Unknown agent name, or an adapter misbehaved during configure/command.""" + + +class Adapter(ABC): + """One agent-under-test, as trial.py needs to see it. + + A concrete adapter overrides the four properties as plain class + attributes (this satisfies the abstract-property contract without + needing a getter method) and implements `configure`/`command`. + """ + + @property + @abstractmethod + def name(self) -> str: + """Registry key, e.g. ``amplifier-agent``. Also the agent-under-test + half of ``images.agent_alias(name)``.""" + + @property + @abstractmethod + def image_alias(self) -> str: + """Published Incus image alias this adapter launches from.""" + + @property + @abstractmethod + def session_dirs(self) -> tuple[str, ...]: + """Container paths holding this agent's session/trajectory state. + Unused in this phase; reserved for the telemetry-pull phase.""" + + @property + @abstractmethod + def metrics_source(self) -> str: + """How a later phase should extract metrics: ``"events"`` for the + Amplifier events.jsonl convention, ``"opencode_db"`` for OpenCode's + SQLite session store, etc. Unused in this phase.""" + + @abstractmethod + async def configure(self, dtu: DTU, *, model: str) -> None: + """Write whatever per-trial config this agent needs into a live DTU. + + Called once, after launch and before seeding. Must not write + secrets to disk -- API keys arrive via the launch profile's + `passthrough.services`, never through this method. + """ + + @abstractmethod + def command(self) -> list[str]: + """argv to invoke the agent, run from /workspace inside the DTU. + + Takes no arguments: anything the command needs beyond the CLI's own + state (e.g. a per-trial session id) must be resolved by `configure` + or the adapter's own `__init__` and captured on the instance. + """ + + +_REGISTRY: dict[str, type[Adapter]] = {} + + +def register(cls: type[Adapter]) -> type[Adapter]: + """Class decorator: add an adapter to the registry under `cls().name`. + + Instantiates once at import time purely to read the `name` property -- + cheap, since adapter `__init__` does no I/O. + """ + key = cls().name + _REGISTRY[key] = cls + return cls + + +def get(name: str, **kwargs: Any) -> Adapter: + """Look up an adapter by name and return a fresh instance. + + Fresh per call: an adapter instance carries per-trial state (e.g. a + generated session id), so callers must not share one instance across + trials. + + `kwargs` are forwarded to the adapter's own `__init__` -- e.g. + `agents.get("amplifier-foundation", bundle=...)`. Every current adapter's + `__init__` takes no required arguments, so a caller that passes nothing + gets the same `cls()` this function always produced; only a caller that + deliberately overrides an agent-specific constructor kwarg needs this. + """ + try: + cls = _REGISTRY[name] + except KeyError: + raise AdapterError(f"unknown agent {name!r}; expected one of {names()}") from None + return cls(**kwargs) + + +def names() -> list[str]: + return sorted(_REGISTRY) + + +__all__ = ["Adapter", "AdapterError", "get", "names", "register"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_amplifier.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_amplifier.py new file mode 100644 index 00000000..8bbcc677 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_amplifier.py @@ -0,0 +1,91 @@ +"""OpenCode frontend backed by amplifier-agent (amplifier-app-opencode). + +Ports deep-swe's OpencodeAmplifierAgent +(../deep-swe/src/deepswe_agents/opencode_amplifier.py) to jobbench's Adapter +contract, with no stubbing and no retries beyond what deep-swe already does. + +KNOWN ISSUE affecting this arm's results. Tool results are lost somewhere in +the opencode round trip, so the model repeatedly re-decides it has not read +files it already read. The provider injects a synthetic +"[SYSTEM ERROR: Tool result missing from conversation history]" message +(amplifier-module-provider-anthropic/__init__.py:2014) and the model narrates +it back in its own words. Measured on one task: 22 to 27 occurrences per run, +and roughly double the wall-clock of the other three agents, while still +exiting 0 with plausible deliverables and a plausible score. The control that +isolates it is opencode-vanilla, which is the same CLI and model talking +directly to Anthropic and never exhibits it. + +Root cause is outside this harness, so nothing here works around it. Affected +runs are flagged via `warnings` in trial.json (see trial._detect_tool_result_loss) +so the contaminated figures cannot be mistaken for a clean measurement. + +configure() does no I/O beyond recording the model: neither deep-swe's +reference nor this port writes any settings file for this agent, since +amplifier-agent (wrapped by amplifier-opencode) reads ANTHROPIC_API_KEY and +ANTHROPIC_BASE_URL straight from the container environment the launch +profile's passthrough.services already populated -- the same assumption +jobbench's own amplifier-agent adapter makes. + +session_dirs is deliberately empty. Nothing under /root was found to hold an +amplifier-agent-style session tree when running through the opencode +wrapper -- see the module-level note below for what was checked. +""" + +from __future__ import annotations + +from jobbench import images +from jobbench.agents.base import Adapter, register +from jobbench.dtu import DTU + +# amplifier-agent imports httpx at import time, on the path of every CLI +# command. It used to arrive transitively via `mcp`; that pull no longer +# exists, so it is pinned here, matching deep-swe's own pin. +AMPLIFIER_AGENT_WITH = "httpx>=0.27,<1" + + +@register +class OpencodeAmplifierAdapter(Adapter): + name = "opencode-amplifier" + image_alias = images.agent_alias("opencode-amplifier") + + # No known location under /root persists an amplifier-agent-style session + # tree (events.jsonl or equivalent) when amplifier-agent is driven through + # the amplifier-opencode wrapper rather than its own CLI. Checked and + # ruled out: /root/.amplifier-agent/state/workspaces (empty -- that path + # is amplifier-agent's OWN CLI session store, not touched by the + # amplifier-opencode launcher), /root/.local/share/opencode (opencode's + # own SQLite usage store, which records opencode's own token accounting, + # not amplifier-agent's -- and this agent's failure mode below means + # opencode itself likely never reaches a billable turn either). + # Left empty rather than guessed: an unpulled telemetry path is honestly + # not_available; a wrong guess would silently pull nothing every time and + # look identical to "no telemetry exists" without ever being caught. + session_dirs: tuple[str, ...] = () + metrics_source = "events" + + def __init__(self) -> None: + # Set by configure(); command() has no other place to source the + # model from, per the Adapter contract. + self._model: str | None = None + + async def configure(self, dtu: DTU, *, model: str) -> None: + del dtu # No per-trial config file needed; see module docstring. + self._model = model + + def command(self) -> list[str]: + """argv equivalent of: + + cd /workspace && amplifier-opencode launch -- run --auto \\ + --model amplifier/ "$(cat /workspace/prompt.txt)" + """ + if self._model is None: + raise RuntimeError("opencode-amplifier command() called before configure()") + script = ( + "cd /workspace && " + f"amplifier-opencode launch -- run --auto --model amplifier/{self._model} " + '"$(cat /workspace/prompt.txt)"' + ) + return ["bash", "-c", script] + + +__all__ = ["AMPLIFIER_AGENT_WITH", "OpencodeAmplifierAdapter"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_vanilla.py b/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_vanilla.py new file mode 100644 index 00000000..8bc09919 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/agents/opencode_vanilla.py @@ -0,0 +1,148 @@ +"""Stock OpenCode talking straight to Anthropic. The control arm. + +Ports deep-swe's OpencodeVanillaAgent +(../deep-swe/src/deepswe_agents/opencode_vanilla.py) to jobbench's Adapter +contract. One structural difference: deep-swe normalizes ANTHROPIC_BASE_URL by +overriding the Python-side env dict pier hands to `environment.exec`. jobbench's +DTU.exec_cmd has no such hook -- there is no channel to inject env into the +remote container process from here. The normalization is done instead inline +in the command's own shell script, re-exporting ANTHROPIC_BASE_URL for that +one invocation from whatever value the launch profile's passthrough.services +already put in the container's environment. + +The `cost` block written into opencode.json is best-effort only: opencode +ignores the `cost.cache` override, so the published dollar figure actually +comes from `metrics.parse_opencode_db`, which recomputes cost from the +recorded token counts against MODEL_RATES_PER_M. See that function's +docstring for why. +""" + +from __future__ import annotations + +import json +from typing import Any + +from jobbench import images +from jobbench.agents.base import Adapter, AdapterError, register +from jobbench.dtu import DTU +from jobbench.metrics import MODEL_RATES_PER_M + +OPENCODE_CONFIG_PATH = "$HOME/.config/opencode/opencode.json" +_HEREDOC_MARKER = "JOBBENCH_OPENCODE_EOF" + + +def _model_entry(model: str) -> dict[str, Any]: + entry: dict[str, Any] = {"name": model} + rates = MODEL_RATES_PER_M.get(model) + if rates: + entry["cost"] = { + "input": rates["input"], + "output": rates["output"], + "cache": {"read": rates["cache_read"], "write": rates["cache_write"]}, + } + return entry + + +def _opencode_config(model: str) -> str: + return json.dumps( + { + "$schema": "https://opencode.ai/config.json", + "model": f"anthropic/{model}", + # Pin the SMALL model to the benchmark model too. opencode uses a + # separate "small" model for its session-title agent, and its + # default family priority ends at claude-haiku -- a model this + # endpoint may not serve. That request fails with a bare + # `AI_APICallError: Not Found` and kills the process (exit 1) + # before any task work happens; this was deep-swe's single most + # common failure mode for this arm. + "small_model": f"anthropic/{model}", + "provider": { + "anthropic": { + "npm": "@ai-sdk/anthropic", + "models": {model: _model_entry(model)}, + } + }, + } + ) + + +# Normalizes ANTHROPIC_BASE_URL for opencode's ai-sdk provider before the run. +# +# The two clients disagree on what "base URL" means: +# * the Anthropic SDK (amplifier-agent, amplifier-foundation) wants the host +# root and appends /v1 itself -> https://api.anthropic.com +# * ai-sdk's @ai-sdk/anthropic (opencode) treats it as the full API root and +# appends only /messages -> needs https://api.anthropic.com/v1 +# +# Forwarding the passthrough value unchanged makes opencode request +# https://api.anthropic.com/messages, which 404s -- reported as a bare +# `Error: Not Found` with no mention of a URL, which looks like a model or +# auth problem instead of what it is. +_BASE_URL_NORMALIZE = ( + 'base="${ANTHROPIC_BASE_URL%/}"; ' + 'case "$base" in */v1) ;; *) base="$base/v1" ;; esac; ' + 'export ANTHROPIC_BASE_URL="$base"' +) + + +@register +class OpencodeVanillaAdapter(Adapter): + name = "opencode-vanilla" + image_alias = images.agent_alias("opencode-vanilla") + + # opencode runs SQLite in WAL mode, so the newest writes -- including, in + # practice, the entire final session -- live in the opencode.db-wal + # sidecar. Collecting the whole data dir (not just the db file) is what + # keeps the sidecar co-located for sqlite3 to replay; pulling opencode.db + # alone yields a stale database that silently under-reports. + session_dirs = ("/root/.local/share/opencode",) + metrics_source = "opencode_db" + + def __init__(self) -> None: + # Set by configure(); command() has no other place to source the + # model from, per the Adapter contract. + self._model: str | None = None + + async def configure(self, dtu: DTU, *, model: str) -> None: + """Write opencode.json at TRIAL time: model, small_model, cost table. + + No secret here -- opencode reads ANTHROPIC_API_KEY straight from the + container environment the launch profile's passthrough.services + already populated. + """ + self._model = model + config_json = _opencode_config(model) + script = ( + 'mkdir -p "$HOME/.config/opencode" && ' + f"cat > \"{OPENCODE_CONFIG_PATH}\" <<'{_HEREDOC_MARKER}'\n" + f"{config_json}\n" + f"{_HEREDOC_MARKER}" + ) + result = await dtu.exec_cmd(["bash", "-c", script]) + if result.returncode != 0: + raise AdapterError( + f"opencode-vanilla configure failed (exit {result.returncode}): " + f"{result.stderr.strip() or result.stdout.strip()}" + ) + + def command(self) -> list[str]: + """argv equivalent of: + + cd /workspace && opencode run --model anthropic/ --auto \\ + "$(cat /workspace/prompt.txt)" + + preceded by the ANTHROPIC_BASE_URL normalization above, scoped to this + one exec (there is no persistent env to poison for a later step). + """ + if self._model is None: + raise AdapterError("opencode-vanilla command() called before configure()") + script = ( + f"{_BASE_URL_NORMALIZE}; " + "cd /workspace && " + f"opencode run --model anthropic/{self._model} --auto " + '"$(cat /workspace/prompt.txt)"' + ) + return ["bash", "-c", script] + + +__all__ = ["OpencodeVanillaAdapter"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/dataset.py b/.amplifier/evaluation/jobbench/src/jobbench/dataset.py new file mode 100644 index 00000000..75f7a937 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/dataset.py @@ -0,0 +1,284 @@ +"""Fetch and describe JobBench tasks. + +The dataset lives on Hugging Face at ``JobBench/job-bench`` and is downloaded +into a local cache at setup time. The cache is keyed by the resolved dataset +revision so a run manifest can name the exact data it scored against. + +Upstream's ``setup.sh`` renames the two split directories on the way down -- +HF ``dataset/`` becomes local ``main/``, HF ``dataset_easy/`` becomes local +``easy/``. We keep that convention so paths are recognizable to anyone who has +used the reference runners. + +Layout of one task, as published: + + //task/ + task_folder/ the ONLY thing the agent may see + TASK_INSTRUCTIONS.txt required; the prompt + ... source files the task operates on + files_required_to_search/ main split only; withheld from the agent, + which is expected to find equivalents on + the open web + RUBRICS.json grading key; withheld + task_card.md human-readable brief; withheld +""" + +from __future__ import annotations + +import json +import os +import shutil +from dataclasses import dataclass +from pathlib import Path + +REPO_ID = "JobBench/job-bench" +REPO_TYPE = "dataset" + +# Local split name -> path prefix within the HF repo. +SPLITS: dict[str, str] = {"main": "dataset", "easy": "dataset_easy"} + +INSTRUCTIONS_NAME = "TASK_INSTRUCTIONS.txt" +RUBRICS_NAME = "RUBRICS.json" +TASK_FOLDER_NAME = "task_folder" +SEARCH_FILES_NAME = "files_required_to_search" + +_REVISION_STAMP = ".jobbench-revision" + + +class DatasetError(RuntimeError): + """Dataset is missing, incomplete, or malformed.""" + + +def cache_root() -> Path: + """Where downloaded splits live. + + Defaults to ``dataset-cache/`` beside this harness (gitignored). Override + with ``JOBBENCH_CACHE_DIR`` to share one download across checkouts. + """ + override = os.environ.get("JOBBENCH_CACHE_DIR") + if override: + return Path(override).expanduser().resolve() + return (Path(__file__).resolve().parents[2] / "dataset-cache").resolve() + + +def split_root(split: str) -> Path: + _validate_split(split) + return cache_root() / split + + +def _validate_split(split: str) -> None: + if split not in SPLITS: + raise DatasetError(f"unknown split {split!r}; expected one of {sorted(SPLITS)}") + + +# -------------------------------------------------------------------------- +# Fetch +# -------------------------------------------------------------------------- + + +def fetch(split: str, *, force: bool = False) -> Path: + """Download one split into the cache. Idempotent unless ``force``. + + Downloads to a staging directory and moves the split into place only after + the transfer succeeds, so an interrupted fetch can never leave a + half-populated tree that later looks complete to ``discover``. + """ + _validate_split(split) + dest = split_root(split) + + if dest.exists() and any(dest.iterdir()) and not force: + return dest + + try: + from huggingface_hub import snapshot_download + except ImportError as exc: # pragma: no cover - environment problem + raise DatasetError("huggingface-hub is not installed; run `uv sync`") from exc + + prefix = SPLITS[split] + staging = cache_root() / f".staging-{split}" + if staging.exists(): + shutil.rmtree(staging) + staging.mkdir(parents=True, exist_ok=True) + + local = snapshot_download( + repo_id=REPO_ID, + repo_type=REPO_TYPE, + allow_patterns=[f"{prefix}/**"], + local_dir=str(staging), + ) + + produced = Path(local) / prefix + if not produced.is_dir(): + raise DatasetError(f"download did not produce {prefix}/ under {local}") + + if dest.exists(): + shutil.rmtree(dest) + dest.parent.mkdir(parents=True, exist_ok=True) + shutil.move(str(produced), str(dest)) + + revision = _resolve_revision() + (dest / _REVISION_STAMP).write_text(revision, encoding="utf-8") + + shutil.rmtree(staging, ignore_errors=True) + return dest + + +def _resolve_revision() -> str: + """Best-effort commit sha of the dataset repo, for the run manifest.""" + try: + from huggingface_hub import HfApi + + return HfApi().repo_info(REPO_ID, repo_type=REPO_TYPE).sha or "unknown" + except Exception: # noqa: BLE001 - provenance is best effort, never fatal + return "unknown" + + +def revision(split: str) -> str: + stamp = split_root(split) / _REVISION_STAMP + if stamp.is_file(): + return stamp.read_text(encoding="utf-8").strip() or "unknown" + return "unknown" + + +# -------------------------------------------------------------------------- +# Task model +# -------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class Task: + """One JobBench task, as it exists on the host.""" + + split: str + occupation: str + task_num: int + root: Path + + @property + def id(self) -> str: + """Stable identifier, e.g. ``biostatisticians__task1``.""" + return f"{self.occupation}__task{self.task_num}" + + @property + def slug(self) -> str: + """Filesystem-safe directory name for this task's trial output.""" + return self.id + + @property + def selector(self) -> str: + """How a user names this task on the CLI, e.g. ``biostatisticians/task1``.""" + return f"{self.occupation}/task{self.task_num}" + + @property + def task_folder(self) -> Path: + return self.root / TASK_FOLDER_NAME + + @property + def instructions_path(self) -> Path: + return self.task_folder / INSTRUCTIONS_NAME + + @property + def rubrics_path(self) -> Path: + return self.root / RUBRICS_NAME + + @property + def has_search_files(self) -> bool: + """True when the task expects the agent to find withheld material online.""" + return (self.root / SEARCH_FILES_NAME).is_dir() + + def instructions(self) -> str: + if not self.instructions_path.is_file(): + raise DatasetError(f"{self.id}: missing {INSTRUCTIONS_NAME}") + return self.instructions_path.read_text(encoding="utf-8") + + def rubrics(self) -> list[dict]: + if not self.rubrics_path.is_file(): + raise DatasetError(f"{self.id}: missing {RUBRICS_NAME}") + data = json.loads(self.rubrics_path.read_text(encoding="utf-8")) + # Upstream accepts either key; `rubrics` wins when both are present. + rubrics = data.get("rubrics") or data.get("evaluation_rubrics") or [] + if not rubrics: + raise DatasetError(f"{self.id}: {RUBRICS_NAME} declares no rubrics") + return rubrics + + def rubric_count(self) -> int: + return len(self.rubrics()) + + def max_score(self) -> int: + return sum(int(r.get("weight", 0)) for r in self.rubrics()) + + def input_bytes(self) -> int: + """Total size of what gets seeded into the container.""" + return sum(p.stat().st_size for p in self.task_folder.rglob("*") if p.is_file()) + + +# -------------------------------------------------------------------------- +# Discovery +# -------------------------------------------------------------------------- + + +def discover(split: str) -> list[Task]: + """All tasks in a split, sorted by occupation then task number. + + A directory only counts as a task when it actually carries the two things + every downstream stage needs: a prompt for the agent and a rubric for the + judge. Anything else is a malformed download, not a task we silently skip. + """ + root = split_root(split) + if not root.is_dir(): + raise DatasetError(f"split {split!r} is not downloaded; run `run.py fetch --split {split}`") + + tasks: list[Task] = [] + for occupation_dir in sorted(p for p in root.iterdir() if p.is_dir()): + for task_dir in sorted(p for p in occupation_dir.iterdir() if p.is_dir()): + num = _task_number(task_dir.name) + if num is None: + continue + task = Task( + split=split, + occupation=occupation_dir.name, + task_num=num, + root=task_dir, + ) + if not task.instructions_path.is_file(): + raise DatasetError( + f"{task.selector}: missing {TASK_FOLDER_NAME}/{INSTRUCTIONS_NAME}" + ) + if not task.rubrics_path.is_file(): + raise DatasetError(f"{task.selector}: missing {RUBRICS_NAME}") + tasks.append(task) + + if not tasks: + raise DatasetError(f"split {split!r} contains no tasks under {root}") + return tasks + + +def _task_number(name: str) -> int | None: + """``task12`` -> 12. Anything else -> None. + + Stricter than the reference runner's ``task[0-9]*`` glob, which also matches + names like ``task1_backup``. The judge's own walker uses ``task[0-9]+``, so + matching the judge keeps discovery and grading agreed on what a task is. + """ + if not name.startswith("task"): + return None + suffix = name[len("task") :] + return int(suffix) if suffix.isdigit() else None + + +def resolve(split: str, selectors: list[str] | None) -> list[Task]: + """Pick tasks by ``occupation/taskN`` selector, or all when none given.""" + tasks = discover(split) + if not selectors: + return tasks + + by_selector = {t.selector: t for t in tasks} + by_id = {t.id: t for t in tasks} + chosen: list[Task] = [] + for sel in selectors: + task = by_selector.get(sel) or by_id.get(sel) + if task is None: + raise DatasetError( + f"no task {sel!r} in split {split!r}; expected e.g. {tasks[0].selector!r}" + ) + chosen.append(task) + return chosen diff --git a/.amplifier/evaluation/jobbench/src/jobbench/dtu.py b/.amplifier/evaluation/jobbench/src/jobbench/dtu.py new file mode 100644 index 00000000..3984895f --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/dtu.py @@ -0,0 +1,434 @@ +# Copied verbatim from amplifier-bundle-evaluation's +# src/amplifier_evaluation/harness/dtu.py. Stdlib-only, no changes needed for +# JobBench. Keep in sync with upstream; do not fork the logic here. + +"""Thin async wrapper over the `amplifier-digital-twin` CLI. + +The harness shells out to the published CLI rather than importing the engine +directly. This keeps the dependency surface tiny and lets us swap in a +different DTU backend (or mock) by replacing this one file. + +All methods are async and use `asyncio.create_subprocess_exec` so they don't +block the event loop while many trials run concurrently. +""" + +from __future__ import annotations + +import asyncio +import json +import logging +import os +import shlex +import shutil +import uuid +from dataclasses import dataclass +from pathlib import Path + +logger = logging.getLogger(__name__) + + +CLI = "amplifier-digital-twin" + + +def _file_push_args(instance_id: str, src: Path, destination: str) -> list[str]: + """Build the `file-push` argv for one source path. + + Current DTU CLI versions auto-detect directory sources and push them + recursively regardless of this flag (DTU PR #18). We still pass + `--recursive` for directories as compatibility with older CLI versions + that predate auto-detect. The flag must be OMITTED for plain files: + with `--recursive` the CLI treats the destination as a parent directory + (`dest/`) instead of an exact file path, which would break + single-file pushes. + """ + args = [CLI, "file-push"] + if src.is_dir(): + args.append("--recursive") + args.extend([instance_id, str(src), destination]) + return args + + +def _pushed_dir_root(src: Path, destination: str) -> str: + """Remote path where a directory push lands. + + The CLI preserves the source directory *name* under the destination + (`cp -r` convention): pushing `data/` to `/workspace/` creates + `/workspace/data/`. + """ + return f"{destination.rstrip('/')}/{src.name}" + + +def _dir_check_script(src: Path, remote_root: str) -> str: + """Shell script verifying a pushed directory actually landed in the DTU. + + Checks the remote root exists and, when the source has entries, that + the remote root is non-empty. Guards against the CLI reporting success + while delivering nothing (silently-empty mounts corrupt grading). + """ + quoted = shlex.quote(remote_root) + script = f"test -d {quoted}" + if any(src.iterdir()): + script += f' && [ -n "$(ls -A {quoted})" ]' + return script + + +class DTUError(RuntimeError): + """Raised when a DTU CLI invocation fails.""" + + def __init__( + self, message: str, *, returncode: int | None = None, stderr: str = "" + ): + super().__init__(message) + self.returncode = returncode + self.stderr = stderr + + +@dataclass +class CommandResult: + """Outcome of one `dtu.exec_cmd()` call.""" + + returncode: int + stdout: str + stderr: str + elapsed_s: float + + +async def _run( + args: list[str], + *, + timeout: float | None = None, + env: dict[str, str] | None = None, +) -> tuple[int, str, str]: + """Run a CLI command, return (returncode, stdout, stderr).""" + proc_env = os.environ.copy() + if env: + proc_env.update(env) + logger.debug("dtu exec: %s", " ".join(args)) + proc = await asyncio.create_subprocess_exec( + *args, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + env=proc_env, + ) + try: + stdout_b, stderr_b = await asyncio.wait_for(proc.communicate(), timeout=timeout) + except asyncio.TimeoutError: + proc.kill() + await proc.wait() + raise DTUError( + f"DTU command timed out after {timeout}s: {' '.join(args)}", + returncode=None, + ) from None + return ( + proc.returncode or 0, + stdout_b.decode("utf-8", errors="replace"), + stderr_b.decode("utf-8", errors="replace"), + ) + + +def cli_available() -> bool: + """Quick check that the DTU CLI is on PATH.""" + return shutil.which(CLI) is not None + + +_ENVELOPE_KEYS = {"command", "exit_code", "stdout", "stderr"} + + +def _envelope_from_payload(payload: dict) -> dict | None: + """Return the exec-result envelope inside `payload`, or None. + + The strict FLAT shape — at least {command, exit_code, stdout, stderr} + with an int exit_code — is checked first. Only when that gate fails do + we accept the nested variant some CLI versions emit, where those same + fields live under an "output" key. Ordering matters: the flat gate is + the primary protocol, so a flat envelope that happens to also carry an + "output" sub-object is never shadowed by it, and the json-lookalike + passthrough semantics for flat command output are unchanged. + """ + if _ENVELOPE_KEYS <= set(payload) and isinstance(payload.get("exit_code"), int): + return payload + nested = payload.get("output") + if ( + isinstance(nested, dict) + and _ENVELOPE_KEYS <= set(nested) + and isinstance(nested.get("exit_code"), int) + ): + return nested + return None + + +def _unwrap_exec_envelope(rc: int, stdout: str, stderr: str) -> tuple[int, str, str]: + """Unwrap the DTU CLI's `exec` JSON result envelope, if present. + + The DTU CLI's `exec` in JSON mode (the default, and the mode this + harness uses) reports the INNER command's result as a JSON envelope on + stdout — {"id", "command", "exit_code", "stdout", "stderr"} — and exits + 0 itself; only the opt-in `--stream` mode propagates the inner exit + code. Without unwrapping, `CommandResult.returncode` is the CLI + process's exit code, so every caller that checks it (notably + `install_agent`'s fail-loud setup_cmds check, install.py:171) tests the + WRONG LAYER: an inner command can fail with exit 1 and the trial + proceeds. This is not hypothetical — observed live in an earlier + evaluation run, all six trials recorded a FAILED in-DTU code-identity + gate ("exit_code": 1 in the envelope, `--- exit 0 ---` outer marker) and + ran to full metered completion anyway. + + Unwrap only when the shape matches: outer success, and either the whole + stdout or (fallback) its last line is one JSON object carrying the + envelope — the flat {command, exit_code, stdout, stderr} shape with an + int exit_code, or, only after that flat gate fails, the same fields + nested under an "output" key (a variant some CLI versions emit; see + `_envelope_from_payload`). The last-line scan tolerates an envelope + preceded by other output on the same stream (e.g. a wrapper banner). + + Everything else passes through untouched: outer CLI failures (nonzero + rc, timeouts, container gone) silently, and — with a loud + `logger.warning`, so an envelope-shape change in the CLI shows up in + logs instead of silently mis-layering every result — outer-success + stdout that is not a recognizable envelope (`--stream`-style plain + output, JSON command output lacking the envelope keys, and + mock/alternative backends that don't wrap). Caveat: the CLI multiplexes + data and control on one stdout, so an inner command whose own output + exactly matches the envelope shape (the four keys with an int + exit_code) WILL be unwrapped — that ambiguity is inherent to the + envelope protocol and cannot be resolved on this side. + """ + if rc != 0: + return rc, stdout, stderr + s = stdout.strip() + # The envelope is normally the entire stdout; fall back to the last + # line to tolerate an envelope preceded by other output. + candidates = [s] + if "\n" in s: + last_line = s.rsplit("\n", 1)[-1].strip() + if last_line: + candidates.append(last_line) + for candidate in candidates: + if not (candidate.startswith("{") and candidate.endswith("}")): + continue + try: + payload = json.loads(candidate) + except json.JSONDecodeError: + continue + if not isinstance(payload, dict): + continue + envelope = _envelope_from_payload(payload) + if envelope is None: + continue + inner_stderr = str(envelope.get("stderr") or "") + if stderr.strip(): + inner_stderr = f"{inner_stderr}\n{stderr}" if inner_stderr else stderr + return envelope["exit_code"], str(envelope.get("stdout") or ""), inner_stderr + logger.warning( + "dtu exec: CLI stdout is not a recognizable JSON envelope; passing " + "through raw output with outer rc=%s (envelope shape may have " + "changed). First 200 chars: %r", + rc, + s[:200], + ) + return rc, stdout, stderr + + +@dataclass +class DTU: + """A handle to one running Digital Twin Universe instance.""" + + id: str + profile_path: str + + # ---- lifecycle ---------------------------------------------------------- + + @classmethod + async def launch( + cls, + profile_path: Path | str, + *, + name: str | None = None, + variables: dict[str, str] | None = None, + launch_timeout_s: float = 600.0, + ) -> "DTU": + """Launch a new DTU instance from a profile. + + `name` is set deterministically when omitted so callers always know + the resulting id up front. After launch we parse the CLI's JSON output + to validate it actually came up. + """ + if not cli_available(): + raise DTUError(f"`{CLI}` is not on PATH") + + if name is None: + name = f"dtu-{uuid.uuid4().hex[:8]}" + + args = [CLI, "launch", "--name", name] + if variables: + for k, v in variables.items(): + args.extend(["--var", f"{k}={v}"]) + args.append(str(profile_path)) + + rc, stdout, stderr = await _run(args, timeout=launch_timeout_s) + if rc != 0: + raise DTUError( + f"DTU launch failed (exit {rc}): {stderr.strip() or stdout.strip()}", + returncode=rc, + stderr=stderr, + ) + + # Parse the CLI's last stdout line as JSON to confirm the launch and + # extract the instance id. We rely on `--name` to control the id, so + # the fallback is safe, but log loudly when we use it: that means the + # CLI changed its output shape and our parser is out of date. + instance_id = name + parsed_id: str | None = None + last_line = stdout.strip().splitlines()[-1] if stdout.strip() else "" + try: + payload = json.loads(last_line) + if isinstance(payload, dict): + for key in ("id", "container_id", "name"): + if payload.get(key): + parsed_id = str(payload[key]) + break + except json.JSONDecodeError: + pass + + if parsed_id is None: + logger.warning( + "dtu launch: could not extract id from CLI output; " + "falling back to --name=%s. Last stdout line: %r", + name, + last_line[:200], + ) + else: + instance_id = parsed_id + + logger.info("dtu launched: %s (profile=%s)", instance_id, profile_path) + return cls(id=instance_id, profile_path=str(profile_path)) + + async def destroy(self, *, timeout_s: float = 120.0) -> None: + """Stop and delete the DTU. Idempotent: missing instance is a no-op.""" + rc, _stdout, stderr = await _run([CLI, "destroy", self.id], timeout=timeout_s) + if rc != 0: + # Don't raise on cleanup failures - log and move on. + logger.warning( + "dtu destroy %s returned %s: %s", self.id, rc, stderr.strip() + ) + + # ---- operations --------------------------------------------------------- + + async def exec_cmd( + self, + command: list[str], + *, + timeout_s: float | None = 600.0, + stream_to_logfile: Path | None = None, + ) -> CommandResult: + """Run a command inside the DTU. Returns full stdout/stderr. + + `command` is the raw argv; the CLI separates it with `--`. + """ + import time + + args = [CLI, "exec"] + if timeout_s is None: + args.extend(["--timeout", "none"]) + else: + args.extend(["--timeout", str(int(timeout_s))]) + args.append(self.id) + args.append("--") + args.extend(command) + + start = time.monotonic() + # We use the CLI's own timeout for the inner command; add slack for the + # outer wait so the CLI can report properly. + outer = (timeout_s + 60.0) if timeout_s is not None else None + rc, stdout, stderr = await _run(args, timeout=outer) + rc, stdout, stderr = _unwrap_exec_envelope(rc, stdout, stderr) + elapsed = time.monotonic() - start + + if stream_to_logfile is not None: + try: + stream_to_logfile.parent.mkdir(parents=True, exist_ok=True) + with stream_to_logfile.open("a", encoding="utf-8") as f: + f.write(f"$ {' '.join(command)}\n") + f.write(stdout) + if stderr: + f.write("\n--- stderr ---\n") + f.write(stderr) + f.write(f"\n--- exit {rc} ({elapsed:.1f}s) ---\n\n") + except OSError as exc: + logger.warning("could not append to %s: %s", stream_to_logfile, exc) + + return CommandResult( + returncode=rc, stdout=stdout, stderr=stderr, elapsed_s=elapsed + ) + + async def file_push( + self, + source: Path | str, + destination: str, + *, + timeout_s: float = 300.0, + ) -> None: + """Push a single host path (file or directory) into the DTU. + + Directory sources are pushed with `--recursive`. Current DTU CLI + versions auto-detect directories and push them recursively either + way (DTU PR #18); the flag is kept as compatibility with older CLI + versions that predate auto-detect. Per the CLI's `cp -r` convention + the directory *name* is preserved: pushing `data/` to `/workspace/` + creates `/workspace/data/`. + + After a directory push the destination is verified inside the DTU. + A push that "succeeds" but delivers nothing raises DTUError instead + of proceeding silently -- empty mounts corrupt everything downstream + (e.g. graders scoring an empty workspace). + """ + src = Path(source) + if not src.exists(): + raise DTUError(f"file-push source missing: {src}") + rc, _stdout, stderr = await _run( + _file_push_args(self.id, src, destination), + timeout=timeout_s, + ) + if rc != 0: + raise DTUError( + f"file-push failed: {src} -> {self.id}:{destination} " + f"(exit {rc}): {stderr.strip()}", + returncode=rc, + stderr=stderr, + ) + if src.is_dir(): + remote_root = _pushed_dir_root(src, destination) + check = await self.exec_cmd( + ["sh", "-c", _dir_check_script(src, remote_root)], + timeout_s=60.0, + ) + if check.returncode != 0: + raise DTUError( + f"file-push reported success but {self.id}:{remote_root} " + f"is missing or empty (source: {src}); the directory was " + f"not delivered into the DTU", + returncode=check.returncode, + stderr=check.stderr, + ) + + async def file_pull( + self, + source: str, + destination: Path | str, + *, + timeout_s: float = 300.0, + ) -> None: + """Pull a path out of the DTU to a host destination.""" + dest = Path(destination) + dest.parent.mkdir(parents=True, exist_ok=True) + rc, _stdout, stderr = await _run( + [CLI, "file-pull", self.id, source, str(dest)], + timeout=timeout_s, + ) + if rc != 0: + raise DTUError( + f"file-pull failed: {self.id}:{source} -> {dest} " + f"(exit {rc}): {stderr.strip()}", + returncode=rc, + stderr=stderr, + ) diff --git a/.amplifier/evaluation/jobbench/src/jobbench/grading.py b/.amplifier/evaluation/jobbench/src/jobbench/grading.py new file mode 100644 index 00000000..b11a8366 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/grading.py @@ -0,0 +1,219 @@ +"""Grade one trial's deliverables against its task's rubric. + +Thin wrapper around the JobBench judge (judge.py). Invokes it +DIRECTLY as a module subprocess, never through upstream's `run_judge.sh` -- +that script silently SKIPS an empty output directory and writes no result +file at all, making a crashed task indistinguishable from one that was never +run. Calling judge.py directly avoids that: its own `--output-dir` handling +already produces a real all-fail report with "No output files found" for an +empty deliverables directory, which is exactly the honest signal a crashed +or no-deliverables trial should grade to. +""" + +from __future__ import annotations + +import json +import os +import re +import subprocess +import sys +from collections.abc import Callable +from pathlib import Path + +from jobbench.dataset import Task + +JUDGE_MODULE = Path(__file__).resolve().parent / "judge.py" + +DEFAULT_JUDGE_MODEL = "gpt-5.6-terra" +DEFAULT_MAX_WORKERS = 10 +DEFAULT_TIMEOUT_PER_RUBRIC = 300 + + +class GradingError(RuntimeError): + """The judge subprocess could not produce a details report.""" + + +def _safe_name(name: str) -> str: + """Filesystem-safe fragment for a judge-model-derived filename. + + Judge model ids can carry characters (`/`, `:`) that are fine in an API + request but not in a path component. + """ + safe = re.sub(r"[^A-Za-z0-9_.-]+", "-", name).strip("-") + return safe or "judge" + + +def grade( + trial_dir: Path, + task: Task, + *, + agent: str, + judge_model: str = DEFAULT_JUDGE_MODEL, + api_base: str | None = None, + api_key: str | None = None, + max_workers: int = DEFAULT_MAX_WORKERS, + timeout_per_rubric: int = DEFAULT_TIMEOUT_PER_RUBRIC, +) -> dict: + """Run the judge on one trial's deliverables. + + Returns the parsed details report (see judge.py's + `build_details_report`): `total_score`, `max_score`, `passed_count`, + `total_count`, a `usage` block, and the full per-rubric `rubrics` list. + + Judge stdout/stderr land at `/grade/judge.log`; the details + report itself at `/grade/_judge.json`. + + Raises GradingError if the judge subprocess exits non-zero or does not + write a details file -- that is a genuine grading failure, distinct from + a legitimate all-fail report (which judge.py writes and this function + returns normally). + """ + grade_dir = trial_dir / "grade" + grade_dir.mkdir(parents=True, exist_ok=True) + details_path = grade_dir / f"{_safe_name(judge_model)}_judge.json" + log_path = grade_dir / "judge.log" + + args = [ + sys.executable, + str(JUDGE_MODULE), + "--output-dir", + str(trial_dir / "deliverables"), + "--rubrics-file", + str(task.rubrics_path), + "--details-file", + str(details_path), + "--evaluated-model", + agent, + "--judge-model", + judge_model, + "--max-workers", + str(max_workers), + "--timeout-per-rubric", + str(timeout_per_rubric), + ] + if api_base: + args.extend(["--api-base", api_base]) + if api_key: + args.extend(["--api-key", api_key]) + + with log_path.open("w", encoding="utf-8") as log: + result = subprocess.run(args, stdout=log, stderr=subprocess.STDOUT, check=False) + + if result.returncode != 0: + raise GradingError(f"judge exited {result.returncode} for {trial_dir}; see {log_path}") + if not details_path.is_file(): + raise GradingError( + f"judge exited 0 but wrote no details file at {details_path}; see {log_path}" + ) + + return json.loads(details_path.read_text(encoding="utf-8")) + + +def grade_and_record( + trial_dir: Path, + task: Task, + *, + agent: str, + judge_model: str, + api_base: str | None, + api_key: str | None, + max_workers: int, + timeout_per_rubric: int, + on_stage: Callable[[str], None] | None = None, +) -> bool: + """Grade one trial and merge the score into its trial.json. + + trial.json's `status` (did it run) and the grade fields (what did it + score) stay separate keys -- a crashed trial and a trial that legitimately + scored zero must remain distinguishable from each other. Returns True when + grading itself ran to completion (any score, including zero); False when + grading could not run at all (judge crashed, wrote no report) -- that + failure is recorded as `grade_error`, not folded into `status`. + + A score and the judge that produced it are written together: on success + `judge_model` is stamped alongside the score; on failure the score fields + AND `judge_model` are cleared, so no reader can ever pair a score with a + judge that did not produce it. + + `on_stage`, if given, receives progress/result lines instead of a direct + print -- the hook a concurrent matrix run uses to prefix every line with + its (agent, task) pair. Defaults to plain stdout/stderr for the + single-trial CLI path, unchanged from before this took a hook. + """ + + def emit(msg: str, *, err: bool = False) -> None: + if on_stage is not None: + on_stage(msg) + else: + print(msg, file=sys.stderr if err else sys.stdout) + + trial_json_path = trial_dir / "trial.json" + trial_data = ( + json.loads(trial_json_path.read_text(encoding="utf-8")) if trial_json_path.is_file() else {} + ) + try: + report = grade( + trial_dir, + task, + agent=agent, + judge_model=judge_model, + api_base=api_base, + api_key=api_key, + max_workers=max_workers, + timeout_per_rubric=timeout_per_rubric, + ) + except GradingError as exc: + # Clear every grade field, not just set grade_error. This trial.json + # may already hold a score from an EARLIER grading pass with a + # different judge; leaving it behind would let a reader pick up that + # stale number while the run now claims to have been graded by this + # judge. A missing score is honest; a score attributed to a judge that + # did not produce it is fabricated provenance, which is strictly worse. + trial_data["total_score"] = None + trial_data["max_score"] = None + trial_data["passed_count"] = None + trial_data["total_count"] = None + trial_data["judge_model"] = None + trial_data["grade_error"] = str(exc) + trial_json_path.write_text(json.dumps(trial_data, indent=2) + "\n", encoding="utf-8") + emit(f"error: grading failed for {trial_dir}: {exc}", err=True) + return False + + trial_data["total_score"] = report.get("total_score") + trial_data["max_score"] = report.get("max_score") + trial_data["passed_count"] = report.get("passed_count") + trial_data["total_count"] = report.get("total_count") + # Every score carries its own provenance, next to the score itself: a + # reader never has to consult the run manifest to know which judge + # produced this number. + trial_data["judge_model"] = judge_model + trial_data["grade_error"] = None + trial_json_path.write_text(json.dumps(trial_data, indent=2) + "\n", encoding="utf-8") + emit( + f"score {report.get('total_score')}/{report.get('max_score')} " + f"passed {report.get('passed_count')}/{report.get('total_count')} ({task.selector})" + ) + return True + + +def resolve_credentials( + api_base_arg: str | None, api_key_arg: str | None +) -> tuple[str | None, str | None]: + """CLI arg wins; else the OpenAI-style env vars this harness's env sets. + + Kept separate from `grade` itself so callers (run.py) can resolve once + and log what was used without `grade` reaching into os.environ itself. + """ + api_base = api_base_arg or os.environ.get("OPENAI_BASE_URL") + api_key = api_key_arg or os.environ.get("OPENAI_API_KEY") + return api_base, api_key + + +__all__ = [ + "DEFAULT_JUDGE_MODEL", + "DEFAULT_MAX_WORKERS", + "DEFAULT_TIMEOUT_PER_RUBRIC", + "GradingError", + "grade", + "resolve_credentials", +] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/images.py b/.amplifier/evaluation/jobbench/src/jobbench/images.py new file mode 100644 index 00000000..2ca0de95 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/images.py @@ -0,0 +1,183 @@ +"""Golden Incus images for the JobBench harness. + +260 trials (65 tasks x 4 agents) each launching a fresh DTU makes +cold-provisioning the full toolchain on every launch untenable -- LibreOffice +alone takes minutes to install. So we bake once and relaunch from a published +Incus image instead. This is the convention documented in +amplifier-bundle-evaluation's ``context/harness/golden_image_caching.md``; +this module is JobBench's instance of it. + +Two images, one dependency chain: + + jobbench-base apt + pip toolchain every agent gets + jobbench- jobbench-base + that agent's CLI, no secrets + +Baking is idempotent: ``ensure_image`` skips the `incus launch` + `publish` +round trip entirely when the alias already exists, unless ``force`` is set. +""" + +from __future__ import annotations + +import asyncio +import logging +import time +from dataclasses import dataclass +from pathlib import Path + +from jobbench.dtu import DTU, DTUError + +logger = logging.getLogger(__name__) + +PROFILES_DIR = Path(__file__).resolve().parents[2] / "profiles" + +BASE_ALIAS = "jobbench-base" +BASE_PROFILE = PROFILES_DIR / "jobbench-base.bake.yaml" + +# Bake profiles live at profiles/agents/.bake.yaml; images are +# published as jobbench-. +AGENT_PROFILES_DIR = PROFILES_DIR / "agents" + +# Baking involves a full package/toolchain install (apt, pip, uv, LibreOffice, +# or an agent CLI) which can legitimately take tens of minutes. The caller is +# expected to run this out of band from any per-trial timeout budget. +DEFAULT_BAKE_TIMEOUT_S = 1800.0 + + +class ImageError(RuntimeError): + """A bake, publish, or lookup step failed.""" + + +@dataclass +class BakeResult: + """Outcome of one `ensure_image` call, for progress reporting.""" + + alias: str + baked: bool # False when the image already existed and we skipped baking + elapsed_s: float + + +def agent_alias(agent: str) -> str: + """Published image alias for an agent, e.g. ``jobbench-amplifier-agent``.""" + return f"jobbench-{agent}" + + +def agent_bake_profile(agent: str) -> Path: + """Path to an agent's bake profile.""" + return AGENT_PROFILES_DIR / f"{agent}.bake.yaml" + + +async def _run(*args: str) -> None: + """Run an `incus` subcommand, raising on non-zero exit. + + Used for the plumbing steps (`stop`, `publish`, `delete`) around a DTU + launch, where `incus` output isn't needed -- only success/failure. + """ + proc = await asyncio.create_subprocess_exec( + *args, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + ) + stdout_b, stderr_b = await proc.communicate() + if proc.returncode != 0: + stderr = stderr_b.decode("utf-8", errors="replace").strip() + stdout = stdout_b.decode("utf-8", errors="replace").strip() + raise ImageError(f"command failed: {' '.join(args)}: {stderr or stdout}") + + +async def image_exists(alias: str) -> bool: + """True if a local Incus image with this alias is already published.""" + proc = await asyncio.create_subprocess_exec( + "incus", + "image", + "info", + alias, + stdout=asyncio.subprocess.DEVNULL, + stderr=asyncio.subprocess.DEVNULL, + ) + return await proc.wait() == 0 + + +async def ensure_image( + bake_profile: Path | str, + alias: str, + *, + force: bool = False, + launch_timeout_s: float = DEFAULT_BAKE_TIMEOUT_S, +) -> BakeResult: + """Bake `alias` from `bake_profile` if it doesn't already exist. + + Skips the bake entirely when the image is already published, unless + `force` -- the common path for every run after the first. On failure at + any step the build container is always deleted in `finally` so a broken + bake never leaks a container that then blocks a retry (Incus refuses to + reuse a name that's still in use). + """ + start = time.monotonic() + + if not force and await image_exists(alias): + return BakeResult(alias=alias, baked=False, elapsed_s=time.monotonic() - start) + + bake_profile = Path(bake_profile) + if not bake_profile.is_file(): + raise ImageError(f"bake profile not found: {bake_profile}") + + if force and await image_exists(alias): + await _run("incus", "image", "delete", alias) + + build_name = f"{alias}-build" + dtu = await DTU.launch(bake_profile, name=build_name, launch_timeout_s=launch_timeout_s) + try: + await _run("incus", "stop", dtu.id) + try: + await _run("incus", "publish", dtu.id, "--alias", alias) + except ImageError as exc: + raise ImageError( + f"incus publish failed for {alias} (built from {bake_profile}): {exc}" + ) from exc + finally: + # Always clean up the build container, success or failure, so a + # failed bake never leaks and blocks the next attempt. + try: + await _run("incus", "delete", "--force", dtu.id) + except ImageError as exc: + logger.warning("could not delete build container %s: %s", dtu.id, exc) + + if not await image_exists(alias): + raise ImageError(f"publish reported success but image {alias!r} is not present") + + return BakeResult(alias=alias, baked=True, elapsed_s=time.monotonic() - start) + + +async def ensure_base_image(*, force: bool = False) -> BakeResult: + """Bake jobbench-base, the toolchain shared by every agent.""" + return await ensure_image(BASE_PROFILE, BASE_ALIAS, force=force) + + +async def ensure_agent_image(agent: str, *, force: bool = False) -> BakeResult: + """Bake jobbench- on top of jobbench-base. + + Callers must ensure jobbench-base already exists (see + `ensure_base_image`) -- the agent bake profile references it as + `base.image: local:jobbench-base` and will fail outright if it's + missing. + """ + profile = agent_bake_profile(agent) + if not profile.is_file(): + raise ImageError(f"no bake profile for agent {agent!r}; expected {profile}") + return await ensure_image(profile, agent_alias(agent), force=force) + + +__all__ = [ + "AGENT_PROFILES_DIR", + "BASE_ALIAS", + "BASE_PROFILE", + "BakeResult", + "DTUError", + "ImageError", + "agent_alias", + "agent_bake_profile", + "ensure_agent_image", + "ensure_base_image", + "ensure_image", + "image_exists", +] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/judge.py b/.amplifier/evaluation/jobbench/src/jobbench/judge.py new file mode 100644 index 00000000..d8f1fc6e --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/judge.py @@ -0,0 +1,1069 @@ +#!/usr/bin/env python3 +# Portions of this file are derived from github.com/Job-Bench/job-bench-eval, +# licensed under the Apache License, Version 2.0. +# Modifications by Microsoft Corporation are documented in the ADAPTATIONS +# section of the module docstring below. +# See ../../THIRD-PARTY-NOTICES.md for the full license text. +"""JobBench LLM-as-Judge. + +Based on the JobBench judge, adapted to this harness's needs. + +Reads model output files, evaluates them against task rubrics, and writes a +scalar reward file plus an optional detailed JSON report. The API client is +OpenAI-compatible by construction, so it can be pointed at OpenAI or any +compatible proxy via JUDGE_API_BASE / JUDGE_API_KEY / JUDGE_MODEL. + +ADAPTATIONS +----------- +All three are confined to `judge_rubric`'s `client.chat.completions.create(...)` +call and the result shapes that flow from it. Everything else is upstream's. + +1. `temperature=0.0` removed, `reasoning_effort="medium"` added. Reasoning + models reject any temperature other than 1, so upstream's hardcoded value + fails outright against one. `reasoning_effort` is the equivalent control. + +2. `max_completion_tokens` lowered from 200000 to 128000. gpt-5.6-terra + rejects upstream's value: + 400 ... "max_tokens is too large: 200000. This model supports at most + 128000 completion tokens" + Every rubric failed after exhausting retries until this matched the model's + real cap. Left as a constant rather than a knob because it is a hard + ceiling of the judge model, not a tuning parameter. Re-check it if the + judge model changes. + +3. Token usage capture. Upstream reads only the message content and discards + `response.usage`, so grading has no cost accounting at all. `_extract_usage` + pulls prompt/completion/total/reasoning tokens off one response, `_sum_usage` + combines them across retries and rubrics, and every rubric result plus the + report carries a `usage` block. Both return `NOT_AVAILABLE` rather than a + fabricated zero when nothing was captured, matching the discipline used + everywhere else in this harness: reporting a real judge run as free is worse + than reporting nothing. +""" + +from __future__ import annotations + +import argparse +import base64 +import binascii +import fcntl +import hashlib +import json +import os +import re +import tempfile +import time +import traceback +from contextlib import contextmanager +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timezone +from pathlib import Path +from zipfile import BadZipFile, ZipFile + + +MAX_CHARS_PER_FILE = 200_000 +SQLITE_EXTS = {"db", "sqlite", "sqlite3"} +SQLITE_ROWS_PER_TABLE = 500 +DEFAULT_JUDGE_API_BASE = "https://api.x.ai/v1" + +VISION_IMAGE_EXTS = {"png", "jpg", "jpeg", "gif", "webp"} +MAX_VISION_IMAGES = 8 +VISUAL_RUBRIC_PATTERN = re.compile( + r"\b(plot|figure|visualization|visualisation|visualize|visualise|" + r"heatmap|histogram|scatter ?plot|biplot|diagram|q[- ]?q)\b", + re.IGNORECASE, +) +VISION_MIME = {"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg", "gif": "image/gif", "webp": "image/webp"} + +# Never a fabricated 0/zero-dict -- an absent usage block means the judge +# cannot price or count that call, not that it was free. Mirrors the +# not_available convention used by jobbench.metrics for the agent-under-test +# side of the same honesty rule. +NOT_AVAILABLE = "not_available" + + +def _extract_usage(response) -> dict | None: + """Token usage for one judge API call, or None if the response carried none. + + Upstream discards `response.usage` entirely. We keep it so the report can + show what grading itself cost. Absence must propagate as None, never as a + fabricated {"prompt_tokens": 0, ...} -- see NOT_AVAILABLE above. + """ + usage = getattr(response, "usage", None) + if usage is None: + return None + result: dict = { + "prompt_tokens": getattr(usage, "prompt_tokens", None), + "completion_tokens": getattr(usage, "completion_tokens", None), + "total_tokens": getattr(usage, "total_tokens", None), + } + details = getattr(usage, "completion_tokens_details", None) + reasoning_tokens = getattr(details, "reasoning_tokens", None) if details is not None else None + if reasoning_tokens is not None: + result["reasoning_tokens"] = reasoning_tokens + if all(value is None for value in result.values()): + return None + return result + + +def _sum_usage(usage_records: list[dict]) -> dict | str: + """Combine per-attempt usage into one totals block for a rubric or report. + + Returns NOT_AVAILABLE (never a zeroed dict) when no attempt produced a + usage block -- e.g. every API call raised before a response came back. + """ + if not usage_records: + return NOT_AVAILABLE + totals = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "api_calls": 0} + reasoning_total = 0 + saw_reasoning = False + for record in usage_records: + for key in ("prompt_tokens", "completion_tokens", "total_tokens"): + value = record.get(key) + if isinstance(value, (int, float)): + totals[key] += value + calls = record.get("api_calls", 1) + totals["api_calls"] += calls if isinstance(calls, (int, float)) else 1 + if "reasoning_tokens" in record: + saw_reasoning = True + value = record.get("reasoning_tokens") + if isinstance(value, (int, float)): + reasoning_total += value + if saw_reasoning: + totals["reasoning_tokens"] = reasoning_total + return totals + + +def _read_text(path: Path) -> str: + try: + return path.read_text(encoding="utf-8", errors="replace") + except Exception as exc: + return f"[ERROR: Failed to read text file: {path.name}: {exc}]" + + +def convert_file_to_text(path: Path) -> str: + ext = path.suffix.lower().lstrip(".") + + if ext in ( + "txt", "md", "csv", "py", "json", "sh", "log", "xml", "html", + "css", "js", "ts", "yaml", "yml", "ini", "cfg", "conf", "sql", + "rules", "geojson", + ): + return _read_text(path) + + if ext in ("xlsx", "xls"): + try: + import pandas as pd + + xl = pd.ExcelFile(str(path)) + parts = [] + for sheet in xl.sheet_names: + df = pd.read_excel(xl, sheet_name=sheet) + parts.append(f"=== Sheet: {sheet} ===\n{df.to_csv(index=False)}") + return "\n".join(parts) + except ImportError: + return f"[ERROR: pandas/openpyxl not available for {path.name}]" + except Exception as exc: + return f"[ERROR: Failed to read Excel {path.name}: {exc}]" + + if ext == "docx": + try: + import mammoth + + def embedded_image_placeholder(image): + return { + "src": "embedded-image", + "alt": f"Embedded image: {image.content_type}", + } + + with open(str(path), "rb") as f: + result = mammoth.convert_to_markdown( + f, + convert_image=mammoth.images.img_element(embedded_image_placeholder), + ) + return result.value + except ImportError: + return f"[ERROR: mammoth not available for {path.name}]" + except Exception as exc: + return f"[ERROR: Failed to read DOCX {path.name}: {exc}]" + + if ext == "pdf": + try: + import pdfplumber + + with pdfplumber.open(str(path)) as pdf: + parts = [] + for i, page in enumerate(pdf.pages): + text = page.extract_text(layout=True) or "" + parts.append(f"=== Page {i + 1} ===\n{text}") + return "\n".join(parts) + except ImportError: + return f"[ERROR: pdfplumber not available for {path.name}]" + except Exception as exc: + return f"[ERROR: Failed to read PDF {path.name}: {exc}]" + + if ext in ("db", "sqlite", "sqlite3"): + import sqlite3 as sqlite + + try: + con = sqlite.connect(str(path)) + cur = con.cursor() + schema = con.execute("SELECT sql FROM sqlite_master WHERE sql IS NOT NULL").fetchall() + parts = ["=== Schema ==="] + parts.extend(row[0] for row in schema if row[0]) + tables = [row[0] for row in con.execute("SELECT name FROM sqlite_master WHERE type='table'").fetchall()] + for table in tables: + parts.append(f"\n=== Table: {table} ===") + try: + total_rows = con.execute(f'SELECT COUNT(*) FROM "{table}"').fetchone()[0] + rows = cur.execute(f'SELECT * FROM "{table}" LIMIT {SQLITE_ROWS_PER_TABLE}').fetchall() + cols = [d[0] for d in cur.description] + parts.append(f"-- total_rows: {total_rows}; shown: {len(rows)} (LIMIT {SQLITE_ROWS_PER_TABLE})") + parts.append(",".join(cols)) + for row in rows: + parts.append(",".join("" if value is None else str(value) for value in row)) + except Exception as exc: + parts.append(f"[ERROR reading table {table}: {exc}]") + con.close() + return "\n".join(parts) + except Exception as exc: + return f"[ERROR: Failed to read SQLite {path.name}: {exc}]" + + if ext == "pptx": + try: + from pptx import Presentation + + prs = Presentation(str(path)) + parts = [] + for idx, slide in enumerate(prs.slides): + parts.append(f"=== Slide {idx + 1} ===") + for shape in slide.shapes: + if hasattr(shape, "text") and shape.text: + parts.append(shape.text) + return "\n".join(parts) + except ImportError: + return f"[ERROR: python-pptx not available for {path.name}]" + except Exception as exc: + return f"[ERROR: Failed to read PowerPoint {path.name}: {exc}]" + + if ext == "ipynb": + try: + nb = json.loads(path.read_text(encoding="utf-8")) + parts = [] + for cell in nb.get("cells", []): + parts.append(f"=== {cell['cell_type']} ===") + parts.append("".join(cell.get("source", []))) + for output in cell.get("outputs", []): + if "text" in output: + parts.append("".join(output["text"])) + return "\n".join(parts) + except Exception as exc: + return f"[ERROR: Failed to read notebook {path.name}: {exc}]" + + if ext in ("png", "jpg", "jpeg", "gif", "svg", "bmp"): + return f"[Image file: {path.name} — cannot extract text content]" + + return f"[Binary or unsupported file type: {ext} — {path.name}]" + + +def extract_all_file_contents(output_dir: Path) -> str: + parts = [] + for file_path in sorted(output_dir.rglob("*")): + if not file_path.is_file(): + continue + content = convert_file_to_text(file_path) + ext = file_path.suffix.lower().lstrip(".") + if ext not in SQLITE_EXTS and len(content) > MAX_CHARS_PER_FILE: + content = content[:MAX_CHARS_PER_FILE] + f"\n... [Content truncated at {MAX_CHARS_PER_FILE} characters]" + parts.append(f"=== FILE: {file_path.name} ===\n{content}\n") + return "\n".join(parts) + + +def rubric_needs_vision(rubric: dict) -> bool: + text = rubric.get("rubric", "") or "" + criterion = rubric.get("criterion", []) + if isinstance(criterion, list): + text = text + " " + " ".join(criterion) + elif isinstance(criterion, str): + text = text + " " + criterion + return bool(VISUAL_RUBRIC_PATTERN.search(text)) + + +def collect_image_paths(output_dir: Path, cap: int | None = MAX_VISION_IMAGES) -> list[Path]: + if not output_dir.exists(): + return [] + images = [ + p + for p in sorted(output_dir.rglob("*")) + if p.is_file() and p.suffix.lower().lstrip(".") in VISION_IMAGE_EXTS + ] + return images if cap is None else images[:cap] + + +def image_to_data_url(path: Path) -> str | None: + ext = path.suffix.lower().lstrip(".") + mime = VISION_MIME.get(ext) + if mime is None: + return None + try: + encoded = base64.b64encode(path.read_bytes()).decode("ascii") + except Exception: + return None + return f"data:{mime};base64,{encoded}" + + +def collect_image_attachments( + output_dir: Path, + cap: int = MAX_VISION_IMAGES, +) -> list[tuple[str, str]]: + """Collect standalone and embedded images as deduplicated data URLs.""" + if not output_dir.exists() or cap <= 0: + return [] + + attachments: list[tuple[str, str]] = [] + seen_hashes: set[str] = set() + + def add_attachment(name: str, mime: str, image_bytes: bytes) -> bool: + digest = hashlib.sha256(image_bytes).hexdigest() + if digest in seen_hashes: + return False + seen_hashes.add(digest) + encoded = base64.b64encode(image_bytes).decode("ascii") + attachments.append((name, f"data:{mime};base64,{encoded}")) + return len(attachments) >= cap + + for path in collect_image_paths(output_dir, cap=None): + ext = path.suffix.lower().lstrip(".") + mime = VISION_MIME.get(ext) + if mime is None: + continue + try: + image_bytes = path.read_bytes() + except OSError: + continue + if add_attachment(path.relative_to(output_dir).as_posix(), mime, image_bytes): + return attachments + + for docx_path in sorted(output_dir.rglob("*.docx")): + try: + with ZipFile(docx_path) as archive: + media_names = [ + name + for name in sorted(archive.namelist()) + if name.startswith("word/media/") + and Path(name).suffix.lower().lstrip(".") in VISION_IMAGE_EXTS + ] + for media_name in media_names: + ext = Path(media_name).suffix.lower().lstrip(".") + mime = VISION_MIME.get(ext) + if mime is None: + continue + try: + image_bytes = archive.read(media_name) + except (KeyError, OSError): + continue + display_name = f"{docx_path.relative_to(output_dir).as_posix()}:{media_name}" + if add_attachment(display_name, mime, image_bytes): + return attachments + except (BadZipFile, OSError): + continue + + for notebook_path in sorted(output_dir.rglob("*.ipynb")): + try: + notebook = json.loads(notebook_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + continue + for cell_index, cell in enumerate(notebook.get("cells", [])): + for output_index, output in enumerate(cell.get("outputs", [])): + data = output.get("data", {}) + if not isinstance(data, dict): + continue + for mime in ("image/png", "image/jpeg", "image/gif", "image/webp"): + encoded = data.get(mime) + if isinstance(encoded, list): + encoded = "".join(str(part) for part in encoded) + if not isinstance(encoded, str): + continue + try: + image_bytes = base64.b64decode("".join(encoded.split()), validate=True) + except (ValueError, binascii.Error): + continue + display_name = ( + f"{notebook_path.relative_to(output_dir).as_posix()}:" + f"cell-{cell_index}-output-{output_index}:{mime}" + ) + if add_attachment(display_name, mime, image_bytes): + return attachments + + return attachments + + +def normalize_criteria(rubric: dict) -> list[str]: + criterion_raw = rubric.get("criterion", []) + if isinstance(criterion_raw, str): + return [criterion_raw] + return list(criterion_raw) + + +def build_failed_rubric_result( + rubric_index: int, + rubric: dict, + overall_reasoning: str, + criteria_reasoning: str | None = None, +) -> dict: + criteria = normalize_criteria(rubric) + per_criterion_reasoning = criteria_reasoning or overall_reasoning + return { + "index": rubric_index, + "rubric": rubric.get("rubric", ""), + "weight": rubric.get("weight", 0), + "result": { + "passed": False, + "score": 0, + "criteria_count": len(criteria), + "criteria_passed": 0, + "criteria_results": [ + { + "index": idx, + "criterion": criterion, + "passed": False, + "reasoning": per_criterion_reasoning, + "evidence": "", + } + for idx, criterion in enumerate(criteria) + ], + "overall_reasoning": overall_reasoning, + }, + "usage": NOT_AVAILABLE, + } + + +def build_scorecard(results: list[dict]) -> dict[str, float | int]: + total_score = sum(result["result"]["score"] for result in results) + max_score = sum(result["weight"] for result in results) + passed_count = sum(1 for result in results if result["result"]["passed"]) + total_count = len(results) + normalized = round(total_score / max_score, 4) if max_score > 0 else 0.0 + pass_rate = round(passed_count / total_count, 4) if total_count > 0 else 0.0 + + scorecard: dict[str, float | int] = { + "total_score": total_score, + "max_score": max_score, + "normalized_score": normalized, + "pass_rate": pass_rate, + "passed_count": passed_count, + "total_count": total_count, + } + for result in results: + idx = result["index"] + scorecard[f"rubric_{idx}_passed"] = 1 if result["result"]["passed"] else 0 + scorecard[f"rubric_{idx}_score"] = result["result"]["score"] + return scorecard + + +def build_reward(scorecard: dict[str, float | int]) -> dict[str, float]: + return {"reward": float(scorecard.get("normalized_score", 0.0))} + + +def _aggregate_report_usage(results: list[dict]) -> dict | str: + """Sum per-rubric usage into one report-level totals block. + + NOT_AVAILABLE (never a zeroed dict) when no rubric call ever returned + usage -- e.g. every rubric hit build_failed_rubric_result before any API + call was made (no output files, unreadable rubrics). + """ + per_rubric = [ + result["usage"] + for result in results + if isinstance(result.get("usage"), dict) + ] + return _sum_usage(per_rubric) + + +def build_details_report( + evaluated_model: str, + judge_model: str, + results: list[dict], + total_count: int | None = None, +) -> dict: + total_score = sum(result["result"]["score"] for result in results) + max_score = sum(result["weight"] for result in results) + passed_count = sum(1 for result in results if result["result"]["passed"]) + effective_total_count = total_count if total_count is not None else len(results) + pass_rate_value = int((passed_count / effective_total_count) * 100) if effective_total_count > 0 else 0 + + return { + "evaluated_model": evaluated_model, + "judge_model": judge_model, + "timestamp": datetime.now(timezone.utc).isoformat(timespec="seconds"), + "total_score": total_score, + "max_score": max_score, + "pass_rate": f"{pass_rate_value}%", + "passed_count": passed_count, + "total_count": effective_total_count, + "usage": _aggregate_report_usage(results), + "rubrics": results, + } + + +def write_json(path: Path, payload: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with tempfile.NamedTemporaryFile( + "w", + encoding="utf-8", + delete=False, + dir=path.parent, + prefix=f".{path.name}.", + suffix=".tmp", + ) as handle: + json.dump(payload, handle, ensure_ascii=False, indent=2) + handle.write("\n") + tmp_path = Path(handle.name) + os.replace(tmp_path, path) + + +def write_outputs( + result_file: Path | None, + reward: dict[str, float], + details_file: Path | None, + details: dict | None, +) -> None: + if result_file is not None: + write_json(result_file, reward) + if details_file is not None: + write_json(details_file, details or {}) + + +@contextmanager +def file_lock(lock_file: Path): + lock_file.parent.mkdir(parents=True, exist_ok=True) + with lock_file.open("a+", encoding="utf-8") as handle: + fcntl.flock(handle.fileno(), fcntl.LOCK_EX) + try: + yield + finally: + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + + +def load_existing_details(details_file: Path | None) -> dict | None: + if details_file is None or not details_file.exists(): + return None + try: + payload = json.loads(details_file.read_text(encoding="utf-8")) + except Exception: + return None + if isinstance(payload, dict): + return payload + return None + + +def load_existing_rubric_results(details_file: Path | None) -> dict[int, dict]: + payload = load_existing_details(details_file) + if not payload: + return {} + + results: dict[int, dict] = {} + for rubric_result in payload.get("rubrics", []): + if not isinstance(rubric_result, dict): + continue + index = rubric_result.get("index") + if isinstance(index, int): + results[index] = rubric_result + return results + + +def append_rubric_result( + details_file: Path, + lock_file: Path, + evaluated_model: str, + judge_model: str, + total_count: int, + rubric_result: dict, +) -> None: + with file_lock(lock_file): + current_payload = load_existing_details(details_file) or {} + current_rubrics = [ + item + for item in current_payload.get("rubrics", []) + if isinstance(item, dict) and item.get("index") != rubric_result.get("index") + ] + current_rubrics.append(rubric_result) + current_rubrics.sort(key=lambda item: item.get("index", -1)) + updated_payload = build_details_report( + evaluated_model, + judge_model, + current_rubrics, + total_count=total_count, + ) + write_json(details_file, updated_payload) + + +def first_non_empty(*values: str | None) -> str: + for value in values: + if value: + return value + return "" + + +def resolve_api_config( + judge_model: str, + api_base_arg: str | None, + api_key_arg: str | None, +) -> tuple[str, str]: + if not judge_model: + raise ValueError("No judge model provided. Set --judge-model or JUDGE_MODEL.") + + api_base = first_non_empty(api_base_arg, os.environ.get("JUDGE_API_BASE"), DEFAULT_JUDGE_API_BASE) + api_key = first_non_empty(api_key_arg, os.environ.get("JUDGE_API_KEY")) + return api_base, api_key + + +def get_openai_client(api_base: str, api_key: str): + from openai import OpenAI + + if not api_key: + raise ValueError("No judge API key provided. Set JUDGE_API_KEY.") + + return OpenAI(base_url=api_base, api_key=api_key) + + +def parse_judge_json(content: str) -> tuple[dict, str]: + try: + return json.loads(content), "direct_json" + except json.JSONDecodeError: + pass + + fence = re.search(r"```(?:json)?\s*\n(.*?)\n\s*```", content, re.DOTALL) + if fence: + try: + return json.loads(fence.group(1).strip()), "markdown_fence" + except json.JSONDecodeError: + pass + + first = content.find("{") + last = content.rfind("}") + if first != -1 and last > first: + try: + return json.loads(content[first:last + 1]), "first_last_brace" + except json.JSONDecodeError: + pass + + for candidate in reversed(re.findall(r"\{.*?\"criteria_results\"\s*:\s*\[.*?\].*?\}", content, re.DOTALL)): + try: + return json.loads(candidate), "regex_extract" + except json.JSONDecodeError: + continue + + raise ValueError(f"Could not extract JSON from response: {content[:500]}") + + +def judge_rubric( + rubric_index: int, + rubric: dict, + file_contents: str, + judge_model: str, + api_base: str, + api_key: str, + timeout_sec: int = 300, + max_retries: int = 3, + image_attachments: list[tuple[str, str]] | None = None, +) -> tuple[dict, dict]: + rubric_text = rubric.get("rubric", "") + weight = rubric.get("weight", 0) + criteria = normalize_criteria(rubric) + criterion_count = len(criteria) + criteria_list_text = "\n".join(f"Criterion {idx}: {criterion}" for idx, criterion in enumerate(criteria)) + + attached_images: list[tuple[str, str]] = [] + if image_attachments and rubric_needs_vision(rubric): + attached_images.extend(image_attachments) + vision_used = bool(attached_images) + + prompt = f"""You are an evaluation judge. Your task is to evaluate ALL criteria for a single rubric. + +## Rubric Description +{rubric_text} + +## Criteria to Evaluate (Judge ALL of them) +{criteria_list_text} + +## Output Files Content +The following are the contents of all output files to evaluate: + +{file_contents} + +## Evaluation Rules +- Evaluate EACH criterion listed above independently +- For each criterion: determine if it PASSES or FAILS +- Semantic matching is acceptable (you don't need exact wording match) +- Binary judgment for each criterion: PASS or FAIL only +- The rubric passes ONLY if ALL criteria pass + +## Output Format +Return your judgment as a JSON object with EXACTLY this structure (no markdown, no extra text): +{{ + "criteria_results": [ + {{"index": 0, "passed": true/false, "reasoning": "...", "evidence": "..."}}, + {{"index": 1, "passed": true/false, "reasoning": "...", "evidence": "..."}} + ], + "rubric_passed": true/false, + "overall_reasoning": "Summary of why the rubric passed or failed" +}} + +IMPORTANT: +- criteria_results array must have exactly {criterion_count} items (one for each criterion) +- rubric_passed should be true ONLY if ALL criteria passed +- Include specific evidence from the output files{' and the attached images' if vision_used else ''} +""" + + if vision_used: + user_content: list[dict] = [{"type": "text", "text": prompt}] + user_content.append({ + "type": "text", + "text": f"\n## Attached Images ({len(attached_images)} file{'s' if len(attached_images) != 1 else ''})", + }) + for i, (fname, url) in enumerate(attached_images, start=1): + user_content.append({"type": "text", "text": f"Image {i}: {fname}"}) + user_content.append({"type": "image_url", "image_url": {"url": url}}) + else: + user_content = prompt + + last_error = None + raw_response = "" + parse_status = "failed" + usage_records: list[dict] = [] + for attempt in range(max_retries): + try: + client = get_openai_client(api_base, api_key) + response = client.chat.completions.create( + model=judge_model, + messages=[ + { + "role": "system", + "content": "You are an evaluation judge. You must return valid JSON only, with no markdown formatting or extra text.", + }, + {"role": "user", "content": user_content}, + ], + # Upstream's 200000 exceeds gpt-5.6-terra's actual cap (API + # rejects with "max_tokens is too large ... at most 128000 + # completion tokens"); lowered to fit. See ADAPTATIONS above. + max_completion_tokens=128000, + # Reasoning models reject temperature != 1; reasoning_effort is + # their equivalent knob. See ADAPTATIONS above. + reasoning_effort="medium", + timeout=timeout_sec, + ) + call_usage = _extract_usage(response) + if call_usage is not None: + usage_records.append(call_usage) + raw_response = response.choices[0].message.content.strip() + parsed, parse_status = parse_judge_json(raw_response) + + model_criteria = parsed.get("criteria_results", []) + rubric_passed = bool(parsed.get("rubric_passed", False)) + overall_reasoning = parsed.get("overall_reasoning", "") + + enriched = [] + for idx, criterion in enumerate(criteria): + item = model_criteria[idx] if idx < len(model_criteria) else {} + enriched.append( + { + "index": idx, + "criterion": criterion, + "passed": bool(item.get("passed", False)), + "reasoning": item.get("reasoning", ""), + "evidence": item.get("evidence", ""), + } + ) + + score = weight if rubric_passed else 0 + criteria_passed = sum(1 for item in enriched if item["passed"]) + result = { + "index": rubric_index, + "rubric": rubric_text, + "weight": weight, + "result": { + "passed": rubric_passed, + "score": score, + "criteria_count": criterion_count, + "criteria_passed": criteria_passed, + "criteria_results": enriched, + "overall_reasoning": overall_reasoning, + }, + "usage": _sum_usage(usage_records), + } + debug = { + "api_base": api_base, + "parse_status": parse_status, + "api_exit_code": 0, + "criterion_count": criterion_count, + "criteria_list_text": criteria_list_text, + "rubric_text": rubric_text, + "raw_response": raw_response, + "vision_used": vision_used, + "attached_images": [name for name, _ in attached_images], + } + return result, debug + except Exception as exc: + last_error = exc + if attempt < max_retries - 1: + time.sleep(2) + continue + + default_criteria = [ + { + "index": idx, + "criterion": criterion, + "passed": False, + "reasoning": f"Failed to get judge response: {last_error}", + "evidence": "", + } + for idx, criterion in enumerate(criteria) + ] + result = { + "index": rubric_index, + "rubric": rubric_text, + "weight": weight, + "result": { + "passed": False, + "score": 0, + "criteria_count": criterion_count, + "criteria_passed": 0, + "criteria_results": default_criteria, + "overall_reasoning": f"Failed after {max_retries} attempts: {last_error}", + }, + "usage": _sum_usage(usage_records), + } + debug = { + "api_base": api_base, + "parse_status": parse_status, + "api_exit_code": 1 if raw_response else 2, + "criterion_count": criterion_count, + "criteria_list_text": criteria_list_text, + "rubric_text": rubric_text, + "raw_response": raw_response, + "error": str(last_error) if last_error is not None else "", + "vision_used": vision_used, + "attached_images": [name for name, _ in attached_images], + } + return result, debug + + +def write_detail_log( + detail_log_dir: Path | None, + detail_log_prefix: str, + rubric_index: int, + judge_model: str, + debug: dict, + final_result: dict, +) -> None: + if detail_log_dir is None or not detail_log_prefix: + return + + detail_log_dir.mkdir(parents=True, exist_ok=True) + detail_log_file = detail_log_dir / f"{detail_log_prefix}_rubric_{rubric_index}.log" + sections = [ + "========================================", + "Rubric Judge Detail Log", + "========================================", + f"Timestamp: {datetime.now(timezone.utc).isoformat(timespec='seconds')}", + f"Unique Key: {detail_log_prefix}", + f"Rubric Index: {rubric_index}", + f"Judge Model: {judge_model}", + f"API Base: {debug.get('api_base', '')}", + f"Parse Status: {debug.get('parse_status', 'failed')}", + f"API Exit Code: {debug.get('api_exit_code', '')}", + f"Criteria Count: {debug.get('criterion_count', '')}", + f"Vision Used: {debug.get('vision_used', False)}", + f"Attached Images: {', '.join(debug.get('attached_images', [])) or '(none)'}", + ] + if debug.get("error"): + sections.append(f"Error: {debug['error']}") + + sections.extend( + [ + "", + "========================================", + "RUBRIC TEXT", + "========================================", + debug.get("rubric_text", ""), + "", + "========================================", + "CRITERIA", + "========================================", + debug.get("criteria_list_text", ""), + "", + "========================================", + "RAW API RESPONSE", + "========================================", + debug.get("raw_response", ""), + "", + "========================================", + "FINAL RESULT", + "========================================", + json.dumps(final_result, ensure_ascii=False, indent=2), + "", + "======================================== END ========================================", + "", + ] + ) + detail_log_file.write_text("\n".join(sections), encoding="utf-8") + + +def main() -> None: + parser = argparse.ArgumentParser(description="JobBench LLM judge") + parser.add_argument("--output-dir", required=True, help="Directory with model output files") + parser.add_argument("--rubrics-file", required=True, help="Path to RUBRICS.json") + parser.add_argument("--result-file", default=None, help="Optional path for reward JSON") + parser.add_argument("--details-file", default=None, help="Where to write detailed results JSON") + parser.add_argument("--judge-model", default=os.environ.get("JUDGE_MODEL", "")) + parser.add_argument("--api-base", default=None, help="OpenAI-compatible API base URL") + parser.add_argument("--api-key", default=None, help="OpenAI-compatible API key") + parser.add_argument("--max-workers", type=int, default=10) + parser.add_argument("--max-retries", type=int, default=3) + parser.add_argument("--timeout-per-rubric", type=int, default=300) + parser.add_argument("--evaluated-model", default="", help="Name of the model output being judged") + parser.add_argument("--lock-file", default=None, help="Optional lock file path for incremental details writes") + parser.add_argument("--detail-log-dir", default=None, help="Optional directory for per-rubric detail logs") + parser.add_argument("--detail-log-prefix", default="", help="Prefix for per-rubric detail logs") + args = parser.parse_args() + + output_dir = Path(args.output_dir) + rubrics_file = Path(args.rubrics_file) + result_file = Path(args.result_file) if args.result_file else None + details_file = Path(args.details_file) if args.details_file else None + lock_file = Path(args.lock_file) if args.lock_file else None + detail_log_dir = Path(args.detail_log_dir) if args.detail_log_dir else None + if lock_file is None and details_file is not None: + lock_file = details_file.with_name(f".{details_file.stem}.lock") + + try: + if not rubrics_file.exists(): + write_outputs( + result_file, + build_reward(build_scorecard([])), + details_file, + {"error": "rubrics not found", "rubrics_file": str(rubrics_file)}, + ) + return + + rubrics_data = json.loads(rubrics_file.read_text(encoding="utf-8")) + rubrics = rubrics_data.get("rubrics") or rubrics_data.get("evaluation_rubrics") or [] + if not rubrics: + write_outputs( + result_file, + build_reward(build_scorecard([])), + details_file, + {"error": "no rubrics", "rubrics_file": str(rubrics_file)}, + ) + return + + total_rubric_count = len(rubrics) + evaluated_model = args.evaluated_model or output_dir.name + has_output_files = output_dir.exists() and any(path.is_file() for path in output_dir.rglob("*")) + if not has_output_files: + results = [ + build_failed_rubric_result(idx, rubric, "No output files found in the model output directory.") + for idx, rubric in enumerate(rubrics) + ] + else: + file_contents = extract_all_file_contents(output_dir) + if not file_contents.strip(): + results = [ + build_failed_rubric_result(idx, rubric, "Output files were unreadable or empty after conversion.") + for idx, rubric in enumerate(rubrics) + ] + else: + existing_results = load_existing_rubric_results(details_file) + api_base, api_key = resolve_api_config(args.judge_model, args.api_base, args.api_key) + image_attachments = collect_image_attachments(output_dir) + + results: list[dict | None] = [existing_results.get(idx) for idx in range(len(rubrics))] + with ThreadPoolExecutor(max_workers=args.max_workers) as executor: + futures = { + executor.submit( + judge_rubric, + idx, + rubric, + file_contents, + args.judge_model, + api_base, + api_key, + args.timeout_per_rubric, + args.max_retries, + image_attachments, + ): idx + for idx, rubric in enumerate(rubrics) + if results[idx] is None + } + for future in as_completed(futures): + idx = futures[future] + try: + results[idx], debug = future.result() + except Exception as exc: + results[idx] = build_failed_rubric_result( + idx, + rubrics[idx], + f"Judge raised an exception: {exc}", + ) + debug = { + "api_base": api_base, + "parse_status": "failed", + "api_exit_code": 2, + "criterion_count": len(normalize_criteria(rubrics[idx])), + "criteria_list_text": "\n".join( + f"Criterion {criterion_idx}: {criterion}" + for criterion_idx, criterion in enumerate(normalize_criteria(rubrics[idx])) + ), + "rubric_text": rubrics[idx].get("rubric", ""), + "raw_response": "", + "error": str(exc), + "vision_used": False, + "attached_images": [], + } + if details_file is not None and lock_file is not None and results[idx] is not None: + append_rubric_result( + details_file, + lock_file, + evaluated_model, + args.judge_model, + total_rubric_count, + results[idx], + ) + write_detail_log( + detail_log_dir, + args.detail_log_prefix, + idx, + args.judge_model, + debug, + results[idx], + ) + + results = [result for result in results if result is not None] + + scorecard = build_scorecard(results) + reward = build_reward(scorecard) + details = build_details_report( + evaluated_model, + args.judge_model, + results, + total_count=total_rubric_count, + ) + write_outputs(result_file, reward, details_file, details) + except Exception as exc: + fallback_reward = build_reward(build_scorecard([])) + write_outputs( + result_file, + fallback_reward, + details_file, + { + "error": str(exc), + "traceback": traceback.format_exc(), + "evaluated_model": args.evaluated_model or output_dir.name, + "judge_model": args.judge_model, + }, + ) + + +if __name__ == "__main__": + main() diff --git a/.amplifier/evaluation/jobbench/src/jobbench/matrix.py b/.amplifier/evaluation/jobbench/src/jobbench/matrix.py new file mode 100644 index 00000000..e5aae98f --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/matrix.py @@ -0,0 +1,128 @@ +"""Build and validate the (agent, task) matrix for `run.py run`. + +A sweep is `--agent` values x `--task` values (or every task in a split, via +`--all-tasks`). This module owns turning raw, repeatable/comma-separated CLI +input into a validated list of `Pair`s -- BEFORE anything is launched, so a +typo in the third agent name fails instantly instead of four hours into a +sweep. All of dataset selector validation is delegated to `dataset.resolve`, +which already fails loudly on an unknown task selector; this module adds the +equivalent validation for agent names (the registry has no such check today) +and the `all` expansion for both axes. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from jobbench import agents, dataset +from jobbench.dataset import Task + +# Observed cost_usd range across prior JobBench runs (trial.json's own +# telemetry). Not a pricing table -- purely a planning aid for --dry-run, so +# a user can tell "$8 sweep" from "$1,600 sweep" before launching anything. +OBSERVED_COST_MIN_USD = 2.04 +OBSERVED_COST_MAX_USD = 10.07 +OBSERVED_COST_MEAN_USD = 6.0 + + +class MatrixError(RuntimeError): + """Invalid --agent/--task selection. Raised before any trial launches.""" + + +@dataclass(frozen=True) +class Pair: + """One cell of the (agent, task) matrix.""" + + agent: str + task: Task + + @property + def label(self) -> str: + """`agent task-selector`, the prefix every progress line uses.""" + return f"{self.agent} {self.task.selector}" + + +def _split_csv(values: list[str]) -> list[str]: + """Flatten repeated `--flag a --flag b` and comma-separated `--flag a,b` + into one ordered, deduped list. Both habits are common enough (and cheap + enough to support) that neither should be a surprise to the user. + """ + seen: dict[str, None] = {} + for raw in values: + for part in raw.split(","): + part = part.strip() + if part: + seen.setdefault(part, None) + return list(seen) + + +def resolve_agent_names(raw: list[str] | None) -> list[str]: + """Expand `--agent` values into a validated list of registered agent names. + + `all` (anywhere in the values) expands to every name currently in the + adapter registry (`agents.names()`) -- never a hardcoded list, so a newly + registered adapter is included automatically. Every other name is checked + against the registry up front; an unknown name raises before the matrix + is even built, let alone launched. + """ + if not raw: + raise MatrixError("--agent is required (repeatable, comma-separated, or 'all')") + values = _split_csv(raw) + if "all" in values: + return agents.names() + known = set(agents.names()) + unknown = [v for v in values if v not in known] + if unknown: + raise MatrixError( + f"unknown agent(s) {unknown}; expected one of {agents.names()} (or 'all')" + ) + return values + + +def resolve_tasks(*, split: str, raw: list[str] | None, all_tasks: bool) -> list[Task]: + """Expand `--task`/`--all-tasks` into a validated list of tasks. + + `--all-tasks` wins over `--task` when both are given, matching how `all` + wins for `--agent`. Selector validation itself is `dataset.resolve`'s + job -- it already raises `DatasetError` naming the exact bad selector. + """ + if all_tasks: + return dataset.discover(split) + if not raw: + raise MatrixError("--task is required (repeatable, comma-separated) or use --all-tasks") + return dataset.resolve(split, _split_csv(raw)) + + +def build_matrix(agent_list: list[str], tasks: list[Task]) -> list[Pair]: + """Agent-major cross product: every task for the first agent, then the + next agent, and so on. Order only matters for how --dry-run and the + summary read; concurrency and skip-existing behave the same regardless. + """ + return [Pair(agent=agent, task=task) for agent in agent_list for task in tasks] + + +def estimate_cost(n_pairs: int) -> tuple[float, float, float]: + """(low, mean, high) USD estimate for `n_pairs` trials. + + Derived from observed trial.json cost_usd values across prior runs -- + an ESTIMATE for sweep planning, not a quote. Actual cost varies with + task, agent, and model. + """ + return ( + n_pairs * OBSERVED_COST_MIN_USD, + n_pairs * OBSERVED_COST_MEAN_USD, + n_pairs * OBSERVED_COST_MAX_USD, + ) + + +__all__ = [ + "OBSERVED_COST_MAX_USD", + "OBSERVED_COST_MEAN_USD", + "OBSERVED_COST_MIN_USD", + "MatrixError", + "Pair", + "build_matrix", + "estimate_cost", + "resolve_agent_names", + "resolve_tasks", +] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/metrics.py b/.amplifier/evaluation/jobbench/src/jobbench/metrics.py new file mode 100644 index 00000000..506338b7 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/metrics.py @@ -0,0 +1,813 @@ +# Vendored from ../deep-swe/src/deepswe_agents/metrics.py. Stdlib-only. +# Kept byte-identical to that copy so a fix in either place ports cleanly; +# do not fork the logic here. +# +# The token-accounting normalization documented in the module docstring below +# was developed here and ported back upstream, so both harnesses now share it. +# Its short version: the two token sources disagree on what "input" means, so +# `input_tokens` is normalized to fresh-only in BOTH branches and +# `total_tokens` is the sum of four disjoint fields. + +"""Parse extracted session logs and emit one normalized metrics.json. + +Forked from an earlier internal evaluation harness and since diverged; this +copy is the source of truth for deep-swe. + +Two token/cost sources are supported, and both produce the same record shape: + +- Amplifier `events.jsonl` (one JSON object per line), in either of the two + envelope shapes the runtimes emit (`input_tokens`/`timestamp` or the bare + `input`/`ts` names). A single call is written to disk once per logging hook + the session composes, so `parse_events` de-duplicates by response identity. +- Vanilla opencode, which has NO events.jsonl and records per-session usage in + the SQLite `opencode.db`. Only the session whose `directory` matches this + harness's workdir (`/app`) is counted, excluding the install-time warm-up + session, and cost is recomputed from `MODEL_RATES_PER_M` (see + `parse_opencode_db` for why). + +`not_available` discipline (never fabricate): every normalized field is either +a real number or the exact string `"not_available"` -- never a silent 0. + +TOKEN ACCOUNTING. Every token field means exactly one thing, in both branches, +and the four are disjoint so they add up: + + input_tokens fresh input only, never previously cached + cache_read_tokens input served from cache + cache_write_tokens input written into cache + output_tokens generated output + total_tokens the sum of all four: every token actually processed + +Reaching that required normalizing the two sources, which do NOT agree on what +"input" means: + +- opencode's `session.tokens_input` column is already fresh-only. Verified on a + real run: `tokens_input=322` alongside `tokens_cache_read=19,299,708`. +- The amplifier stacks fold cache_read INTO their reported `input_tokens` (but + not cache_write), so `parse_events` subtracts it back out. Verified across + 114 events of a real run: 0/114 had input < cache_read, while 114/114 had + input < cache_read + cache_write -- e.g. `input=872` with `cache_read=0` and + `cache_write=12354`, which only a fresh-plus-cache_read reading explains. + +Why it matters: while `total_tokens` was `input + output`, an opencode trial +reported 95,147 against an amplifier trial's 1,218,757 on the same run -- an +apparent 12x gap that INVERTED the true ordering, since opencode had actually +processed ~19.6M tokens to amplifier's ~10.2M. The old figure silently dropped +opencode's entire 19.3M cache-read volume. + +`cost_usd` is unaffected by any of this: it is priced from the raw per-source +counts against `MODEL_RATES_PER_M`, never derived from `total_tokens`. +""" + +from __future__ import annotations + +import datetime +import hashlib +import json +import re +import sqlite3 +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +# The normalized, agent-agnostic efficiency schema. Every field must appear in +# metrics.json as a number or the NOT_AVAILABLE marker. +NOT_AVAILABLE = "not_available" + +METRIC_FIELDS = ( + "cost_usd", + "input_tokens", + "output_tokens", + "cache_read", + "cache_write", + "total_tokens", + "llm_responses", + "agent_wallclock_s", + "total_wallclock_s", +) + + +def _parse_iso(tstr: str) -> float | None: + """Parse an ISO-8601 string to epoch seconds, trimming sub-microsecond fraction. + + Handles both the amplifier-agent `timestamp` (nanoseconds, e.g. + "2026-07-07T17:22:40.591486782+00:00") and the Python amplifier `ts` (e.g. + "2026-02-05T22:33:33.323+00:00"). datetime.fromisoformat only accepts up to + microseconds, so the fraction is trimmed to '.' + 6 digits. + """ + m = re.match(r"^(.*T\d{2}:\d{2}:\d{2})(\.\d+)?(.*)$", tstr) + if not m: + return None + base, frac, tz = m.groups() + frac = frac[:7] if frac else "" # '.' + up to 6 digits + try: + return datetime.datetime.fromisoformat(f"{base}{frac}{tz}").timestamp() + except ValueError: + return None + + +def _event_epoch(obj: dict) -> float | None: + """Return an event's time as epoch seconds. + + Reads whichever time field is present -- amplifier-agent uses `timestamp`, + the Python amplifier hooks-logging format uses `ts`. Both emit ISO-8601. + """ + for key in ("ts", "timestamp"): + val = obj.get(key) + if isinstance(val, str): + epoch = _parse_iso(val) + if epoch is not None: + return epoch + return None + + +def _pick(usage: dict, *keys: str) -> object: + """Return the first present key from `usage`, else None. + + Providers disagree on token field names: the amplifier-agent stack emits + `input_tokens`/`cache_read_tokens`; the Python Anthropic provider emits + `input`/`cache_read`. Try the `_tokens` name first, then the bare name. + """ + for key in keys: + if key in usage: + return usage[key] + return None + + +def _to_int(value: object) -> int: + """Coerce a usage field to int; missing/malformed -> 0.""" + try: + return int(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return 0 + + +def _to_float(value: object) -> float: + """Coerce a cost field (often a string like '0.01867525') to float; else 0.0.""" + try: + return float(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return 0.0 + + +def _ms_to_epoch(value: object) -> float | None: + """Coerce an opencode timestamp (epoch milliseconds) to epoch seconds.""" + if isinstance(value, bool) or not isinstance(value, (int, float)): + return None + return float(value) / 1000.0 + + +@dataclass +class _Usage: + """Token/cost/timestamp accumulator shared by both parser branches. + + The two parsers read genuinely different sources (SQLite rows vs JSONL + lines) but accumulate the same figures and must return the same dict shape, + because `_finalize` treats the two branches uniformly. That common part + lives here once; the reading bodies stay separate. + """ + + input_tokens: int = 0 + output_tokens: int = 0 + cache_read_tokens: int = 0 + cache_write_tokens: int = 0 + cost_usd: float = 0.0 + saw_cost: bool = False + llm_responses: int = 0 + files_read: int = 0 + min_ts: float | None = None + max_ts: float | None = None + #: Records where reported input was somehow smaller than cache_read, i.e. + #: the source's convention is not what `parse_events` assumes. Surfaced in + #: `notes` rather than swallowed, because the resulting figure is wrong. + negative_fresh_input: int = 0 + + def observe_time(self, epoch: float | None) -> None: + """Widen the earliest-to-latest span with one timestamp; None is a no-op.""" + if epoch is None: + return + self.min_ts = epoch if self.min_ts is None else min(self.min_ts, epoch) + self.max_ts = epoch if self.max_ts is None else max(self.max_ts, epoch) + + def as_dict(self) -> dict[str, Any]: + """The keys `_finalize` reads, identical for both branches. + + `agent_wallclock_s` is 0.0 when no timestamp was seen -- callers must + consult `had_timestamps` to tell that apart from a genuine 0-length run. + """ + lo, hi = self.min_ts, self.max_ts + had_timestamps = lo is not None and hi is not None + return { + "input_tokens": self.input_tokens, + "output_tokens": self.output_tokens, + "cache_read_tokens": self.cache_read_tokens, + "cache_write_tokens": self.cache_write_tokens, + # Every token the model actually processed. `input_tokens` is + # fresh-only in both branches by construction, so cache_read and + # cache_write are additive here and nothing is double-counted. + "total_tokens": ( + self.input_tokens + + self.cache_read_tokens + + self.cache_write_tokens + + self.output_tokens + ), + "negative_fresh_input": self.negative_fresh_input, + "cost_usd": self.cost_usd, + "cost_from_events": self.saw_cost, + "llm_responses": self.llm_responses, + "files_read": self.files_read, + "had_timestamps": had_timestamps, + "agent_wallclock_s": (hi - lo) if lo is not None and hi is not None else 0.0, + } + + +# --------------------------------------------------------------------------- +# Reference rate card +# --------------------------------------------------------------------------- + +#: USD per 1M tokens, keyed by model id. THE single rate card for this harness. +#: +#: Mirrors `_RATES` in amplifier-module-provider-anthropic/_cost.py, which is +#: what stamps `cost_usd` into the amplifier arms' events.jsonl. Arms are only +#: comparable if every dollar figure comes from the same card, so this is also +#: what the opencode arm's cost is RECOMPUTED with -- see +#: `compute_cost_from_tokens` and the WHY in `parse_opencode_db`. +#: +#: `opencode_vanilla.py` imports this to populate the model's `cost` block in +#: opencode.json. One definition, one home. +MODEL_RATES_PER_M: dict[str, dict[str, float]] = { + "claude-sonnet-5": {"input": 3.00, "output": 15.00, "cache_read": 0.30, "cache_write": 3.75}, + "claude-sonnet-4-5": {"input": 3.00, "output": 15.00, "cache_read": 0.30, "cache_write": 3.75}, + "claude-opus-5": {"input": 5.00, "output": 25.00, "cache_read": 0.50, "cache_write": 6.25}, +} + + +def compute_cost_from_tokens( + model: str | None, + *, + input_tokens: int = 0, + output_tokens: int = 0, + cache_read_tokens: int = 0, + cache_write_tokens: int = 0, +) -> float | None: + """USD for one call/session from token counts and the reference card. + + Returns None for an unrecognised model -- semantically distinct from 0.0 + (a genuinely free call). Callers must propagate that as `not_available` + rather than as a free run. + """ + rates = MODEL_RATES_PER_M.get(model or "") + if rates is None: + return None + return ( + input_tokens * rates["input"] + + output_tokens * rates["output"] + + cache_read_tokens * rates["cache_read"] + + cache_write_tokens * rates["cache_write"] + ) / 1_000_000.0 + + +def _opencode_model_id(raw: Any) -> str | None: + """Extract the bare model id from opencode's `session.model` column. + + Stored as a JSON object, e.g. + ``{"id":"claude-sonnet-5","providerID":"anthropic","variant":"default"}``, + which SQLite hands back as the raw JSON text. + """ + if isinstance(raw, str): + try: + raw = json.loads(raw) + except (json.JSONDecodeError, ValueError): + return None + if isinstance(raw, dict): + mid = raw.get("id") + return mid if isinstance(mid, str) and mid else None + return None + + +def _opencode_sessions(db_path: str, workspace_dir: str) -> tuple[list[dict] | None, int, int]: + """Read per-session usage rows from one opencode SQLite db (`opencode.db`). + + Returns (sessions, assistant_message_count, total_session_count), or + (None, 0, 0) when the file is not a usable opencode db (unreadable, or + missing the expected `session` columns). The db is opened READ-ONLY + (`mode=ro`); the WAL sidecar (`opencode.db-wal`) must be co-located for the + newest writes to be visible. + + ONLY sessions whose `directory == workspace_dir` are returned. There is + deliberately NO "if nothing matched, return everything" fallback. + + That fallback was worse than useless. `directory` is an absolute path, so a + caller passing the wrong workspace matched nothing and silently summed + EVERY session in the database -- including unrelated ones -- publishing a + plausible-looking number that was wrong by orders of magnitude. An empty + result surfaces as `not_available`, which is honest and fixable; a + fabricated total is neither. + """ + try: + con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) + except sqlite3.Error: + return None, 0, 0 + try: + con.row_factory = sqlite3.Row + cols = {row[1] for row in con.execute("PRAGMA table_info(session)")} + if not {"tokens_input", "tokens_output", "cost", "directory"}.issubset(cols): + return None, 0, 0 + sessions = [dict(r) for r in con.execute("SELECT * FROM session")] + matched = [s for s in sessions if s.get("directory") == workspace_dir] + + ids = {s.get("id") for s in matched} + assistant = 0 + try: + for sid, data in con.execute("SELECT session_id, data FROM message"): + if sid not in ids: + continue + try: + md = json.loads(data) + except (json.JSONDecodeError, ValueError, TypeError): + continue + if isinstance(md, dict) and md.get("role") == "assistant": + assistant += 1 + except sqlite3.Error: + assistant = 0 + return matched, assistant, len(sessions) + except sqlite3.Error: + return None, 0, 0 + finally: + con.close() + + +def parse_opencode_db(db_paths: list[str], workspace_dir: str) -> dict[str, Any]: + """Sum token/cost usage for a vanilla opencode run from its SQLite db(s). + + Vanilla opencode has NO amplifier events.jsonl; the `session` table in + `opencode.db` already aggregates per-session usage (tokens_input/output/ + cache_read/cache_write, cost) with a `directory` and epoch-ms + time_created/time_updated. llm_responses is the count of assistant rows in + the `message` table for the task session. + + Returns the SAME dict shape as `parse_events` (see `_Usage.as_dict`) plus + `extra_notes`, so `_finalize` can treat both branches uniformly. + `cost_from_events` is True when a real cost figure was produced, so a + genuine $0 is not mistaken for a free run. + + COST IS RECOMPUTED, NOT READ. The `session.cost` column is deliberately + ignored for the published figure and reported only as an audit note. + + Why: opencode prices calls from a rate card it fetches from models.dev, and + that card is NOT the one this harness bills against -- it diverges on the + cache rates, and opencode ignores the explicit `cost.cache` override baked + into opencode.json. Comparing arms priced from two different cards is a + silent error: both numbers look plausible and neither is flagged. The token + counts are ground truth from the agent; only the multiplication belongs to + the harness, so cost is recomputed from `MODEL_RATES_PER_M`. + + Args: + db_paths: Paths to extracted `opencode.db` files (the WAL sidecar must + be co-located for completeness). + workspace_dir: Session directory identifying the task run, used to + exclude the install-time warm-up session. Required: a wrong value + silently matches nothing, so there is no safe default. + """ + usage = _Usage() + extra_notes: list[str] = [] + + for db_path in db_paths: + sessions, assistant, total_sessions = _opencode_sessions(db_path, workspace_dir) + if sessions is None: + continue + if not sessions: + # The db is a valid opencode db but holds no session for this + # workspace. Report it; do NOT substitute the other sessions. + extra_notes.append( + f"{Path(db_path).name} held {total_sessions} session(s), none with " + f"directory == {workspace_dir!r}; contributed nothing. If the agent " + f"really ran, the workspace_dir passed to the parser is wrong." + ) + continue + usage.files_read += 1 + # Prefer the true assistant-turn count; fall back to session count so a + # run with clear usage is never reported as 0 responses. + usage.llm_responses += assistant or len(sessions) + for s in sessions: + # opencode's `tokens_input` column is already FRESH-ONLY (it + # excludes both cache figures), which is the convention this module + # normalizes to, so it is accumulated as-is. The amplifier branch + # has to strip cache_read out to reach the same meaning; see + # `parse_events`. + s_in = _to_int(s.get("tokens_input")) + s_out = _to_int(s.get("tokens_output")) + s_cr = _to_int(s.get("tokens_cache_read")) + s_cw = _to_int(s.get("tokens_cache_write")) + usage.input_tokens += s_in + usage.output_tokens += s_out + usage.cache_read_tokens += s_cr + usage.cache_write_tokens += s_cw + + model = _opencode_model_id(s.get("model")) + recomputed = compute_cost_from_tokens( + model, + input_tokens=s_in, + output_tokens=s_out, + cache_read_tokens=s_cr, + cache_write_tokens=s_cw, + ) + reported = _to_float(s.get("cost")) + if recomputed is None: + # Unknown model => no rate card => no honest dollar figure. + # NOT 0.0: the run was not free, we just cannot price it. + extra_notes.append( + f"Model {model!r} is not in the reference rate card, so cost_usd is " + f"not_available for this session. opencode self-reported " + f"${reported:.6f}, which is priced from ITS card and is not " + f"comparable to the amplifier arms." + ) + else: + usage.saw_cost = True + usage.cost_usd += recomputed + # Always record the divergence. If the two ever agree this note + # is the evidence; when they disagree it is the explanation. + delta = recomputed - reported + pct = (delta / recomputed * 100.0) if recomputed else 0.0 + extra_notes.append( + f"cost_usd RECOMPUTED from token counts at the reference rate card " + f"for {model}: ${recomputed:.6f}. opencode self-reported " + f"${reported:.6f} (delta ${delta:+.6f}, {pct:+.1f}%); its figure is " + f"priced from a models.dev card that differs on cache rates and is " + f"NOT used." + ) + for key in ("time_created", "time_updated"): + usage.observe_time(_ms_to_epoch(s.get(key))) + + return {**usage.as_dict(), "extra_notes": extra_notes} + + +def _response_identity(obj: dict) -> str | None: + """Return a stable identity for an `llm:response` event, or None. + + WHY THIS EXISTS. A single logical LLM call is written to disk more than once + whenever a session composes more than one logging hook. The published + `anchors` bundle does exactly this: it includes BOTH + `foundation:behaviors/logging` (hooks-logging -> `/events.jsonl`) + and `context-intelligence:behaviors/context-intelligence-logging` + (hook-context-intelligence -> `/context-intelligence/events.jsonl`). + Both files are legitimate telemetry and both are pulled by extraction, so + summing across them counted every call, token and dollar TWICE. + + The two copies are not detectable by comparing bytes: the loggers use + different envelope shapes (`ts` vs `timestamp`, metadata at the top level vs + nested under `data`), so the files share zero identical lines. They are only + recognisable by what they DESCRIBE. Hence identity, not equality: + + strong the provider's own response id (`data.raw.id`). Globally unique + per call, present whenever raw payload capture is on. + fallback session id + event timestamp + usage, hashed. Used when raw + capture is off. The timestamp is sub-microsecond and is byte + identical across both loggers (verified on real run artifacts), + so it discriminates distinct calls reliably. + None neither is available -- see the caller. We do NOT guess. + + Returning None on a weak signal is deliberate. Over-de-duplicating would + silently UNDERSTATE cost, which is a worse failure than the overcount this + function exists to fix: an inflated number invites scrutiny, a deflated one + does not. + """ + # Bind before narrowing: `x.get(k) if isinstance(x.get(k), dict) else {}` + # calls get() twice, and the isinstance on the first call does not narrow + # the second, so the result stays `Any | None`. + raw_data = obj.get("data") + data: dict[str, Any] = raw_data if isinstance(raw_data, dict) else {} + + raw_payload = data.get("raw") + if isinstance(raw_payload, dict): + rid = raw_payload.get("id") + if isinstance(rid, str) and rid: + return f"id:{rid}" + + # Fallback fingerprint. Require a timestamp: without one, two genuinely + # distinct calls that happened to use the same token counts would collide. + ts = data.get("timestamp") or obj.get("ts") or obj.get("timestamp") + if not isinstance(ts, (str, int, float)) or isinstance(ts, bool): + return None + + session_id = data.get("session_id") or obj.get("session_id") + raw_usage = data.get("usage") + usage: dict[str, Any] = raw_usage if isinstance(raw_usage, dict) else {} + fingerprint = json.dumps([session_id, str(ts), usage], sort_keys=True, default=str) + return "fp:" + hashlib.sha1(fingerprint.encode()).hexdigest() + + +def parse_events(events_paths: list[str]) -> dict[str, Any]: + """Sum token/cost usage and compute wallclock from Amplifier events.jsonl. + + Dual-shape aware (see the module docstring). Malformed lines and unreadable + files are skipped defensively; a run is never silently dropped because one + line failed to parse. + + Args: + events_paths: Paths to events.jsonl files (one JSON object per line). + + Returns: + The keys listed in `_Usage.as_dict`, plus `duplicate_responses` (count + of duplicate `llm:response` events dropped) and + `unidentified_responses` (count that carried no usable identity and + were therefore counted without de-duplication). + """ + usage = _Usage() + duplicate_responses = 0 + unidentified_responses = 0 + seen_responses: set[str] = set() + + for path in events_paths: + try: + with open(path, encoding="utf-8") as f: + lines = f.readlines() + except OSError: + continue + usage.files_read += 1 + + for line in lines: + line = line.strip() + if not line: + continue + try: + obj = json.loads(line) + except (json.JSONDecodeError, ValueError): + continue + if not isinstance(obj, dict): + continue + + # Track wallclock across ALL events that carry a usable time field. + usage.observe_time(_event_epoch(obj)) + + if obj.get("event") != "llm:response": + continue + + # De-duplicate by identity. The same call reaches this loop once per + # logging hook the session composed (see _response_identity), and + # summing every copy is what inflated cost, tokens and call counts. + identity = _response_identity(obj) + if identity is None: + # No usable identity: count it rather than risk collapsing two + # genuinely distinct calls. Surfaced in the notes. + unidentified_responses += 1 + elif identity in seen_responses: + duplicate_responses += 1 + continue + else: + seen_responses.add(identity) + + data = obj.get("data") + event_usage = data.get("usage") if isinstance(data, dict) else None + if not isinstance(event_usage, dict): + # Still count the response even if usage is absent. + usage.llm_responses += 1 + continue + + usage.llm_responses += 1 + # Field names differ by runtime/provider: amplifier-agent emits the + # `_tokens`-suffixed names, the Python Anthropic provider emits the + # bare names. Accept either. + reported_in = _to_int(_pick(event_usage, "input_tokens", "input")) + ev_cache_read = _to_int(_pick(event_usage, "cache_read_tokens", "cache_read")) + ev_cache_write = _to_int(_pick(event_usage, "cache_write_tokens", "cache_write")) + # The amplifier stacks report an `input_tokens` that ALREADY + # contains cache_read (but not cache_write), while opencode reports + # a fresh-only figure. Strip cache_read here so `input_tokens` means + # exactly one thing -- genuinely-new, never-cached input -- no + # matter which source produced the record. Measured on a real run: + # 114/114 events had input >= cache_read and input < cache_read + + # cache_write, e.g. input=872 with cache_read=0, cache_write=12354. + fresh_in = reported_in - ev_cache_read + if fresh_in < 0: + # The invariant above broke, so the assumption no longer holds + # for this source. Clamp rather than emit a negative token + # count, and say so: a silently wrong number is the failure + # this whole normalization exists to prevent. + usage.negative_fresh_input += 1 + fresh_in = 0 + usage.input_tokens += fresh_in + usage.output_tokens += _to_int(_pick(event_usage, "output_tokens", "output")) + usage.cache_read_tokens += ev_cache_read + usage.cache_write_tokens += ev_cache_write + # cost_usd is only emitted by the amplifier-agent stack. Track + # whether we ever saw it so a $0 from a stack that does not record + # cost is not reported as a real, free run. + if "cost_usd" in event_usage: + usage.saw_cost = True + usage.cost_usd += _to_float(event_usage.get("cost_usd")) + + return { + **usage.as_dict(), + "duplicate_responses": duplicate_responses, + "unidentified_responses": unidentified_responses, + } + + +def _find(output_dir: Path | str, filename: str) -> list[str]: + """Return every `filename` under a run's output dir, sorted for determinism. + + The extractor pulls session logs under `output_dir/sessions/`, preserving + each session's own layout, so the whole tree is globbed: narrowing the glob + to pick a "primary" file would be fragile, because the layout differs per + agent and would silently yield zero on any change. + + For `events.jsonl` this deliberately returns EVERY file, including the + several a single session produces when it composes more than one logging + hook; the duplicates are handled where they can be handled correctly, by + `parse_events` de-duplicating on response identity. For `opencode.db` it + matches the main db only, not the `-wal`/`-shm` sidecars -- sqlite3 reads a + co-located WAL automatically and the extractor preserves the layout, so the + sidecars land next to the db. + """ + root = Path(output_dir).expanduser().resolve() + return sorted(str(p) for p in root.rglob(filename)) + + +def find_events_files(output_dir: Path | str) -> list[str]: + """Return all extracted `events.jsonl` paths under a run's output dir.""" + return _find(output_dir, "events.jsonl") + + +def find_opencode_db_files(output_dir: Path | str) -> list[str]: + """Return all extracted `opencode.db` paths under a run's output dir.""" + return _find(output_dir, "opencode.db") + + +def normalize_metrics(events_paths: list[str], *, source: str | None = None) -> dict[str, Any]: + """Produce the normalized metrics.json record from extracted events. + + Applies the `not_available` discipline: + - If no events file was readable, every token/cost/response/agent-wallclock + field is `"not_available"` (the source is genuinely absent). + - `cost_usd` is `"not_available"` when no event carried a `cost_usd` field + (e.g. the Python amplifier stack), never a fabricated 0. + - `agent_wallclock_s` is `"not_available"` when no event timestamp was + found; otherwise it is the earliest-to-latest event span, a floor on true + agent time. + + Args: + events_paths: Paths to extracted events.jsonl files. + source: Optional identifier for the agent/stack (e.g. the agent id). + + Returns: + A JSON-safe dict with every METRIC_FIELDS key present, plus `notes`, + `source`, and `events_files` (the files this record was computed from). + """ + parsed = parse_events(events_paths) + files_read = parsed["files_read"] + notes: list[str] = [] + + if files_read == 0: + notes.append( + "No events.jsonl files were readable under the extraction output dir; " + "all token/cost/response/agent-wallclock fields are not_available." + ) + else: + notes.append( + f"Parsed {parsed['llm_responses']} llm:response event(s) across " + f"{files_read} events.jsonl file(s). Token keys read with dual-shape " + f"fallback (input_tokens/input, etc.)." + ) + # State the de-duplication explicitly. Without this the corrected figure + # is indistinguishable from a run that simply made fewer calls. + dupes = parsed["duplicate_responses"] + if dupes: + notes.append( + f"Dropped {dupes} duplicate llm:response event(s): this session composes more " + f"than one logging hook, so each call was written to disk more than once. " + f"Counted once each by response identity." + ) + unknown = parsed["unidentified_responses"] + if unknown: + notes.append( + f"{unknown} llm:response event(s) carried no response id and no timestamp, so " + f"they could not be de-duplicated and are counted as-is; if this session " + f"composes multiple logging hooks these may be overcounted." + ) + if parsed["cost_from_events"]: + notes.append("cost_usd summed from per-response cost_usd fields.") + else: + notes.append("cost_usd is not_available: no event carried a cost_usd field.") + if parsed["had_timestamps"]: + notes.append( + "agent_wallclock_s is the earliest-to-latest event timestamp span " + "(a floor on true agent time; agent_run_s is the measured duration)." + ) + else: + notes.append("agent_wallclock_s is not_available: no timestamps found.") + + return _finalize(parsed, source=source, source_files=events_paths, notes=notes) + + +def normalize_opencode_metrics( + db_paths: list[str], + *, + source: str | None = None, + workspace_dir: str, +) -> dict[str, Any]: + """Produce the normalized metrics.json record from a vanilla opencode db. + + Same `not_available` discipline and output schema as `normalize_metrics`, + but the token/cost source is the opencode SQLite `session` table rather than + Amplifier events.jsonl. `workspace_dir` is required for the reason given in + `parse_opencode_db`: a wrong value silently matches no session. + """ + parsed = parse_opencode_db(db_paths, workspace_dir) + files_read = parsed["files_read"] + extra_notes: list[str] = list(parsed["extra_notes"]) + notes: list[str] = [] + + if files_read == 0: + # Only claim "nothing was readable" when nothing explains it better. A + # source that WAS read but contributed no rows has already said so, and + # the two statements together read as a contradiction. + if not extra_notes: + notes.append( + "No usable opencode.db was readable under the extraction output dir; " + "all token/cost/response/agent-wallclock fields are not_available." + ) + else: + notes.append( + f"Parsed {parsed['llm_responses']} assistant message(s) across " + f"{files_read} opencode.db file(s). ONLY sessions with directory == " + f"{workspace_dir} were counted; every other session in the database was " + f"excluded, with no fallback." + ) + if parsed["cost_from_events"]: + notes.append("cost_usd summed from the opencode session table `cost` column.") + else: + notes.append("cost_usd is not_available: the opencode session carried no cost.") + if parsed["had_timestamps"]: + notes.append( + "agent_wallclock_s is the task session's time_created-to-time_updated span " + "(opencode epoch-ms timestamps; a floor on true agent time)." + ) + else: + notes.append("agent_wallclock_s is not_available: no timestamps found.") + + # Parser-level warnings must reach metrics.json either way -- e.g. a db that + # held sessions but none for this workspace, which is the difference between + # "no data" and "wrong query". + notes.extend(extra_notes) + return _finalize(parsed, source=source, source_files=db_paths, notes=notes) + + +def _finalize( + parsed: dict[str, Any], + *, + source: str | None, + source_files: list[str], + notes: list[str], +) -> dict[str, Any]: + """Turn a parser's `parsed` dict into the normalized metrics.json record. + + Shared by both the events.jsonl and opencode.db branches: they produce the + same `parsed` shape, so the `not_available` discipline and the rounding live + here once. Branch-specific prose is composed by the caller and passed in. + """ + if parsed["files_read"] == 0: + # No session sources were pulled: everything derived from them is absent. + record: dict[str, Any] = {name: NOT_AVAILABLE for name in METRIC_FIELDS} + else: + record = { + "cost_usd": parsed["cost_usd"] if parsed["cost_from_events"] else NOT_AVAILABLE, + "input_tokens": parsed["input_tokens"], + "output_tokens": parsed["output_tokens"], + "cache_read": parsed["cache_read_tokens"], + "cache_write": parsed["cache_write_tokens"], + "total_tokens": parsed["total_tokens"], + "llm_responses": parsed["llm_responses"], + "agent_wallclock_s": ( + parsed["agent_wallclock_s"] if parsed["had_timestamps"] else NOT_AVAILABLE + ), + } + + # Whole-trial elapsed is not measured here; the adapter records the agent + # command duration instead. Never fabricate a number for it. + record["total_wallclock_s"] = NOT_AVAILABLE + + # Round the span/cost figures for readability when they are numbers. + if isinstance(record["agent_wallclock_s"], (int, float)): + record["agent_wallclock_s"] = round(float(record["agent_wallclock_s"]), 3) + if isinstance(record["cost_usd"], (int, float)): + record["cost_usd"] = round(float(record["cost_usd"]), 6) + + # A source whose input figure was smaller than its own cache_read violates + # the convention `parse_events` normalizes against, so the fresh-input + # figure for those records is a clamped 0 rather than the truth. Say so. + if parsed.get("negative_fresh_input"): + notes = [ + *notes, + ( + f"{parsed['negative_fresh_input']} record(s) reported input_tokens " + "below their own cache_read, which contradicts the " + "fresh-plus-cache_read convention this parser normalizes; fresh " + "input was clamped to 0 for those, so input_tokens is a FLOOR and " + "total_tokens may undercount." + ), + ] + + record["source"] = source + record["events_files"] = list(source_files) + record["notes"] = " ".join([*notes, "total_wallclock_s is not measured by this harness."]) + return record diff --git a/.amplifier/evaluation/jobbench/src/jobbench/orphans.py b/.amplifier/evaluation/jobbench/src/jobbench/orphans.py new file mode 100644 index 00000000..6e20d38d --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/orphans.py @@ -0,0 +1,89 @@ +"""Reap leaked `jb-`-prefixed DTU instances around a matrix run. + +Every trial's DTU is uniquely named and destroyed in its own `finally` (see +`jobbench.trial._dtu_name` / `run_trial`), so under normal operation nothing +here ever has work to do. The gap this covers is a hard kill of the harness +process itself (Ctrl-C, OOM, node reboot) mid-sweep: that skips every +in-flight trial's `finally` and leaves its container running. At 260 trials +that gap is not hypothetical. + +This is a coarse safety net around the whole matrix, not a substitute for +per-trial cleanup. It is deliberately only ever invoked from `run.py` BEFORE +the matrix starts and AFTER it finishes -- never while trials are in flight -- +so it cannot destroy a container one of THIS PROCESS's own trials still owns. +That is the entire safety property, and it holds only within one process. + +UNSAFE ACROSS PROCESSES: the sweep destroys every `jb-`-prefixed instance the +CLI reports, and a name cannot distinguish "leaked from a dead run" from +"owned by a live peer run" -- telling those apart is the whole point of the +pre-run sweep, so no naming scheme fixes this. Two harness processes running +against the same host WILL destroy each other's live containers. Run only one +harness process per host, or pass `--no-orphan-sweep` to the concurrent ones +(which then leak their own containers on a hard kill; reap them by hand). +""" + +from __future__ import annotations + +import asyncio +import json +import logging + +from jobbench.dtu import CLI, DTU + +logger = logging.getLogger(__name__) + +JB_PREFIX = "jb-" + + +async def list_instances() -> list[dict]: + """Raw `amplifier-digital-twin list` output (a JSON array of instance + dicts, each carrying at least `id`). Best-effort: a sweep that can't + enumerate instances must not abort the run, only skip reaping this time. + """ + proc = await asyncio.create_subprocess_exec( + CLI, + "list", + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + ) + stdout_b, stderr_b = await proc.communicate() + if proc.returncode != 0: + logger.warning( + "orphan sweep: `%s list` failed: %s", + CLI, + stderr_b.decode("utf-8", errors="replace").strip(), + ) + return [] + try: + payload = json.loads(stdout_b.decode("utf-8", errors="replace")) + except json.JSONDecodeError: + logger.warning("orphan sweep: `%s list` did not return JSON", CLI) + return [] + return payload if isinstance(payload, list) else [] + + +async def sweep_orphans() -> list[str]: + """Destroy every `jb-`-prefixed DTU instance currently reported by the + CLI, and return the ids destroyed. + + Safe with respect to THIS process's trials only: it is called when this + harness has none in flight (see module docstring), so every `jb-` instance + it sees is a leak as far as this process can tell. It has no way to tell a + leak from a container a PEER harness process on the same host is actively + using, and destroys both alike -- so running two harness processes against + one host concurrently is unsafe. `run --no-orphan-sweep` opts out. + """ + instances = await list_instances() + destroyed: list[str] = [] + for inst in instances: + inst_id = inst.get("id") + if not inst_id or not isinstance(inst_id, str) or not inst_id.startswith(JB_PREFIX): + continue + logger.info("orphan sweep: reaping %s", inst_id) + # profile_path is unused by destroy(); DTU is just an id handle here. + await DTU(id=inst_id, profile_path="").destroy() + destroyed.append(inst_id) + return destroyed + + +__all__ = ["JB_PREFIX", "list_instances", "sweep_orphans"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/prompt.py b/.amplifier/evaluation/jobbench/src/jobbench/prompt.py new file mode 100644 index 00000000..379701d4 --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/prompt.py @@ -0,0 +1,76 @@ +# Portions of this file are derived from github.com/Job-Bench/job-bench-eval, +# licensed under the Apache License, Version 2.0. +# The `_TEMPLATE` string below is reproduced verbatim from upstream's +# eval/run_benchmark_codex_cli.sh (the `prompt_msg` assignment), where it is +# byte-identical across all three reference runners. Modification by Microsoft +# Corporation: the three interpolated paths (workspace, task_folder, +# output_dir) point at the container-side layout in WORKSPACE/TASK_FOLDER/ +# OUTPUT_DIR below rather than upstream's host-side /tmp scratch directory. +# The template wording itself is unchanged, and must stay that way -- see the +# module docstring. +# See ../../THIRD-PARTY-NOTICES.md for the full license text. +"""The prompt handed to the agent under test. + +This wrapper is reproduced verbatim from the JobBench reference runners, where +it is byte-identical across all three (Claude Code, Codex CLI, OpenCode) and +varies only in the three interpolated paths. It is the benchmark's actual task +contract: it tells the agent where its inputs are, where deliverables must be +written, and that it may search the web but must not roam the filesystem. + +Do not edit the wording. Scores produced under a different prompt are not +comparable to any other run of this benchmark, published or local. The only +thing that legitimately varies is the three paths, which differ here because +trials run inside a container rather than in a /tmp scratch directory on the +host. + +Upstream: eval/run_benchmark_codex_cli.sh (and its two siblings), the +`prompt_msg` assignment. +""" + +from __future__ import annotations + +# Container-side layout. The agent may see everything under WORKSPACE and +# nothing else: rubrics, task cards, and any search-discoverable reference +# material stay on the host. +WORKSPACE = "/workspace" +TASK_FOLDER = f"{WORKSPACE}/task_folder" +OUTPUT_DIR = f"{WORKSPACE}/output" +PROMPT_PATH = f"{WORKSPACE}/prompt.txt" + +_TEMPLATE = """=== TASK FOLDER === +{task_folder} + +=== INSTRUCTIONS === +1. Read the TASK_INSTRUCTIONS.txt file in the task folder above +2. Based on the Reference Files section in TASK_INSTRUCTIONS.txt, read the corresponding files from the same task folder using appropriate tools. +3. Complete the task as specified in TASK_INSTRUCTIONS.txt +4. Only save the final deliverables to the output directory specified below. Do not save any intermediate or temporary files. + +=== OUTPUT DIRECTORY === +{output_dir} + +IMPORTANT: +- All reference files are in the task folder: {task_folder} +- Only save the final deliverables to the output directory {output_dir}. Do not save any intermediate or temporary files. +- You MUST only access files within {workspace} or search online for new reference files if you find needed. Do NOT access any files or directories in this system outside of this path. +- If you encounter ambiguous or conflicting information, analyze the conflict, explain your reasoning, and justify the approach you choose. +- If a file cannot be read directly (e.g., .xlsx, .docx, .db, .pptx), use appropriate tools, MCP servers, or code to extract and process its contents.""" + + +def render( + *, + workspace: str = WORKSPACE, + task_folder: str = TASK_FOLDER, + output_dir: str = OUTPUT_DIR, +) -> str: + """The exact bytes sent to the agent. + + Note what is absent: the task's own TASK_INSTRUCTIONS.txt is NOT inlined. + Upstream points the agent at the file and expects it to read it, so the + agent's ability to navigate its own workspace is part of what is measured. + """ + return _TEMPLATE.format( + workspace=workspace, + task_folder=task_folder, + output_dir=output_dir, + ) diff --git a/.amplifier/evaluation/jobbench/src/jobbench/scheduler.py b/.amplifier/evaluation/jobbench/src/jobbench/scheduler.py new file mode 100644 index 00000000..91d339ab --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/scheduler.py @@ -0,0 +1,232 @@ +"""Bounded-concurrency execution of a JobBench (agent, task) matrix. + +Ported from amplifier-bundle-evaluation's harness/scheduler.py: an +`asyncio.Semaphore` caps concurrency, one `asyncio.Task` per trial, and +results come back in input order via `asyncio.gather`. The one property that +matters most at 260-trial scale is failure isolation -- a trial raising must +never kill the batch or hold its semaphore slot forever, so every trial is +wrapped in a `try/except` that turns any escape into a recorded failure +instead. `trial.run_trial` already catches everything itself and always +writes trial.json; this is defense in depth for the (grading, skip-check, +scheduler-glue) code that runs alongside it. + +No state-machine resume lives here on purpose: recovery is `--run-id` plus +`--skip-existing` re-selecting a subset, not automatic. See run.py. +""" + +from __future__ import annotations + +import asyncio +import json +import logging +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from jobbench import grading +from jobbench import trial as trial_mod +from jobbench.matrix import Pair + +logger = logging.getLogger(__name__) + +OnLine = Callable[[str], None] +AgentKwargsFor = Callable[[str], dict[str, Any]] + + +@dataclass +class PairOutcome: + """What run.py needs to print and tally for one matrix cell. + + This is run bookkeeping (what already lives in trial.json), not the + cross-task aggregator -- no scoring roll-up beyond the one trial's own + total_score/max_score. + """ + + pair: Pair + trial_dir: Path + skipped: bool + status: str # trial status ("completed"/"timeout"/"crashed"/"no_deliverables"), or "skipped" + agent_run_s: float | None + cost_usd: float | str + total_score: float | None + max_score: float | None + has_warnings: bool + error: str | None + graded_ok: bool + + +def _read_trial_json(trial_dir: Path) -> dict[str, Any] | None: + path = trial_dir / "trial.json" + if not path.is_file(): + return None + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + return None + + +def _is_completed(trial_dir: Path) -> bool: + """True when trial_dir already holds a trial.json with status=completed. + + Anything else (missing, unparseable, or a non-completed status such as + `crashed`/`timeout`/`no_deliverables`) is NOT completed, so --skip-existing + re-runs it -- only a genuinely finished trial is worth skipping. + """ + data = _read_trial_json(trial_dir) + return data is not None and data.get("status") == "completed" + + +def _skip_outcome(pair: Pair, trial_dir: Path) -> PairOutcome: + data = _read_trial_json(trial_dir) or {} + return PairOutcome( + pair=pair, + trial_dir=trial_dir, + skipped=True, + status="skipped", + agent_run_s=data.get("agent_run_s"), + cost_usd=data.get("cost_usd", "not_available"), + total_score=data.get("total_score"), + max_score=data.get("max_score"), + has_warnings=bool(data.get("warnings")), + error=None, + graded_ok=True, + ) + + +async def run_matrix( + pairs: list[Pair], + run_root: Path, + *, + model: str, + timeout_s: float, + max_parallel: int, + agent_kwargs_for: AgentKwargsFor, + skip_existing: bool, + grade: bool, + judge_model: str, + judge_api_base: str | None, + judge_api_key: str | None, + judge_max_workers: int, + judge_timeout_per_rubric: int, + on_line: OnLine | None = None, +) -> list[PairOutcome]: + """Run every pair in `pairs`, capped at `max_parallel` concurrent trials. + + Each pair writes its own complete artifact set to + `run_root///` exactly as a single-pair `run` always has; + this function only adds bounded concurrency, skip-existing, and + per-pair-prefixed progress on top of that. Returns outcomes in the same + order as `pairs`. + """ + if max_parallel < 1: + raise ValueError("max_parallel must be >= 1") + + def emit(line: str) -> None: + if on_line is not None: + on_line(line) + else: + print(line, flush=True) + + sem = asyncio.Semaphore(max_parallel) + + async def _one(pair: Pair) -> PairOutcome: + trial_dir = run_root / pair.agent / pair.task.id + + # Skip-existing is checked BEFORE acquiring the semaphore: it's a + # cheap local read and shouldn't hold a concurrency slot waiting on + # trials that are actually going to run. + if skip_existing: + existing = _read_trial_json(trial_dir) + if existing is not None and existing.get("status") == "completed": + emit(f"[{pair.label}] skip-existing: trial.json already status=completed") + return _skip_outcome(pair, trial_dir) + if existing is not None: + emit( + f"[{pair.label}] skip-existing: existing trial.json has " + f"status={existing.get('status')!r} (not completed) -- re-running" + ) + + async with sem: + + def stage(msg: str) -> None: + emit(f"[{pair.label}] {msg}") + + try: + stage("launching trial") + result = await trial_mod.run_trial( + pair.agent, + pair.task, + trial_dir, + model=model, + timeout_s=timeout_s, + on_stage=stage, + agent_kwargs=agent_kwargs_for(pair.agent) or None, + ) + except Exception as exc: + # run_trial is supposed to catch everything itself and always + # write trial.json; this is defensive in case something in + # the scheduler-facing call itself (not run_trial's own body) + # escapes -- the batch continues either way. + logger.exception("trial %s raised outside run_trial's own guard", pair.label) + stage(f"UNHANDLED ERROR: {type(exc).__name__}: {exc}") + return PairOutcome( + pair=pair, + trial_dir=trial_dir, + skipped=False, + status="crashed", + agent_run_s=None, + cost_usd="not_available", + total_score=None, + max_score=None, + has_warnings=False, + error=f"unhandled in scheduler: {type(exc).__name__}: {exc}", + graded_ok=False, + ) + + graded_ok = True + total_score: float | None = None + max_score: float | None = None + if grade: + stage(f"grading with {judge_model}") + try: + graded_ok = grading.grade_and_record( + trial_dir, + pair.task, + agent=pair.agent, + judge_model=judge_model, + api_base=judge_api_base, + api_key=judge_api_key, + max_workers=judge_max_workers, + timeout_per_rubric=judge_timeout_per_rubric, + on_stage=stage, + ) + except Exception as exc: + logger.exception("grading raised for %s", pair.label) + stage(f"grading UNHANDLED ERROR: {type(exc).__name__}: {exc}") + graded_ok = False + if graded_ok: + trial_data = _read_trial_json(trial_dir) or {} + total_score = trial_data.get("total_score") + max_score = trial_data.get("max_score") + + stage(f"done status={result.status}") + return PairOutcome( + pair=pair, + trial_dir=trial_dir, + skipped=False, + status=result.status, + agent_run_s=result.agent_run_s, + cost_usd=result.cost_usd, + total_score=total_score, + max_score=max_score, + has_warnings=bool(result.warnings), + error=result.error, + graded_ok=graded_ok, + ) + + tasks = [asyncio.create_task(_one(p), name=f"pair:{p.label}") for p in pairs] + return list(await asyncio.gather(*tasks)) + + +__all__ = ["PairOutcome", "run_matrix"] diff --git a/.amplifier/evaluation/jobbench/src/jobbench/trial.py b/.amplifier/evaluation/jobbench/src/jobbench/trial.py new file mode 100644 index 00000000..61e691bf --- /dev/null +++ b/.amplifier/evaluation/jobbench/src/jobbench/trial.py @@ -0,0 +1,463 @@ +"""Run one (agent, task) trial: launch a DTU, drive the agent, pull deliverables. + +Prove one agent can complete one real task end to end, deliverables land on +disk, telemetry is captured for cost/token accounting, and `trial.json` +honestly distinguishes a crash, a timeout, and a legitimate zero-deliverable +run from each other and from success. + +A trial always writes `trial.json` and always destroys its DTU, even when a +stage fails partway through -- a failed trial is data, not a dead end. +""" + +from __future__ import annotations + +import json +import logging +import re +import shutil +import time +import uuid +from collections.abc import Callable +from dataclasses import asdict, dataclass +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +from jobbench import agents, images, prompt +from jobbench.dataset import Task +from jobbench.dtu import DTU, DTUError +from jobbench.metrics import ( + NOT_AVAILABLE, + find_events_files, + find_opencode_db_files, + normalize_metrics, + normalize_opencode_metrics, +) + +logger = logging.getLogger(__name__) + +TEMPLATE_PATH = Path(__file__).resolve().parents[2] / "profiles" / "task.template.yaml" +IMAGE_PLACEHOLDER = "__AGENT_IMAGE__" + +DEFAULT_TIMEOUT_S = 3600.0 + +# Slack added on top of the agent's own wall-clock budget for the DTU exec +# round trip itself (CLI startup, JSON envelope write). Keeps a legitimate +# near-the-limit run from being misclassified as a DTU-layer timeout. +_EXEC_SLACK_S = 60.0 + +# Signatures left in a captured agent.log when amplifier-module-provider-anthropic +# injects a synthetic error message because a tool_call had no paired +# tool_result in conversation history (see +# amplifier_module_provider_anthropic/__init__.py:2014). The provider only +# logs a logger.warning when this happens -- exit code, deliverables, and +# score all look completely normal -- so without this scan the run silently +# measures an agent re-deciding it never read its own instructions instead of +# the agent's real capability. Fixing the provider/opencode is out of scope +# for this harness; this list only makes the condition visible. Kept as a +# module-level constant, not inline in the scan function, so a new signature +# discovered later is a one-line addition. +TOOL_RESULT_LOSS_SIGNATURES: tuple[str, ...] = ( + "[SYSTEM ERROR: Tool result missing from conversation history]", + "Tool execution was interrupted and no result was captured", +) + +# The signatures above are the direct, high-confidence evidence, but they are +# frequently NOT observable: the provider injects that text into the message +# history it sends to the model, and only logs a logger.warning locally. For a +# CLI whose stdout carries just the assistant's prose (opencode), the literal +# string never appears in agent.log at all -- measured: 0 hits on a run that +# looped 27 times. What IS observable is the model narrating the injected error +# in its own words, re-deciding it never read a file it already read. +# +# So this second pass is an INDIRECT heuristic on that narration. It is kept +# separate from the literal signatures above, reported with a lower confidence, +# and gated behind a threshold, because a single "let me read the file" line is +# ordinary agent behavior. Repetition is the tell. +# +# Measured discriminating power on one identical task: +# amplifier-agent 0, amplifier-foundation 0, opencode-vanilla 0, +# opencode-amplifier 21 and 27. +TOOL_RESULT_LOSS_NARRATION = re.compile( + r"(?:" + r"(?:still |actually |now )?(?:need to|have to|must) actually (?:read|open)" + r"|haven[''`]?t (?:actually )?(?:read|opened)" + r"|(?:realize|notice) (?:I|that I) (?:have not|haven[''`]?t|never)" + r"|operating (?:on|from) infer(?:red|ence)" + r"|without (?:actually )?(?:opening|reading)" + r")", + re.IGNORECASE, +) + +# Below this many narration hits the signal is indistinguishable from an agent +# legitimately deciding to re-read something, so it is not worth flagging. +TOOL_RESULT_LOSS_NARRATION_THRESHOLD = 3 + +# agent.log is stdout, then this marker, then stderr. The narration heuristic +# scans only the stdout half; see `_detect_tool_result_loss`. +STDERR_SEPARATOR = "--- stderr ---" + + +class TrialError(RuntimeError): + """Trial setup failed before there was a DTU to attribute it to.""" + + +@dataclass +class TrialResult: + """Everything `run.py run` needs to report, and everything a later + grading phase needs to know before it touches the deliverables.""" + + agent: str + task_id: str + task_selector: str + split: str + model: str + dtu_id: str | None + image_alias: str + status: str # "completed" | "timeout" | "crashed" | "no_deliverables" + exit_code: int | None + agent_run_s: float | None + started_at: str + finished_at: str + deliverable_count: int + deliverable_bytes: int + error: str | None + # Telemetry summary, duplicated from metrics.json so a single file + # answers "did it run, and what did it cost" without a second read. + # `not_available` (never a fabricated 0) when no session telemetry + # could be collected -- see jobbench.metrics's not_available discipline. + cost_usd: float | str + total_tokens: int | str + llm_responses: int | str + # Quality signals distinct from `status`: a run can complete (exit 0, + # deliverables produced, a real score) while still being a degenerate + # measurement of something other than the agent's real capability. + # Never folded into `status` -- see `_detect_tool_result_loss`. Empty + # list, never omitted, so a clean run and an ungraded field are never + # ambiguous in trial.json. + warnings: list[dict[str, Any]] + + def write(self, path: Path) -> None: + path.write_text(json.dumps(asdict(self), indent=2) + "\n", encoding="utf-8") + + +def _now() -> str: + return datetime.now(UTC).isoformat() + + +def _detect_tool_result_loss(log_path: Path) -> dict[str, Any] | None: + """Scan a captured agent.log for provider tool-result-loss signatures. + + Returns a `trial.json` warnings-list entry when any signature appears, + else None. This is a quality signal, not a run-status signal: a trial + that hits this can still exit 0 and produce deliverables, because the + model narrates the synthetic error back to itself and keeps going -- + the exit code, deliverable count, and score all look normal even though + the agent burned wall-clock time fighting its own context rather than + doing the task. Deliberately does not touch `status`; see the docstring + on `TrialResult.warnings`. + + Missing or unreadable logs are not an error here -- some agents/trial + outcomes (e.g. a crash before the log was created) never produce one. + """ + if not log_path.is_file(): + return None + try: + text = log_path.read_text(encoding="utf-8", errors="replace") + except OSError: + return None + + direct = sum(text.count(sig) for sig in TOOL_RESULT_LOSS_SIGNATURES) + if direct: + matched = [sig for sig in TOOL_RESULT_LOSS_SIGNATURES if sig in text] + return { + "kind": "tool_result_loss", + "confidence": "direct", + "count": direct, + "detail": ( + f"{direct} occurrence(s) of the provider-injected tool-result-loss " + f"signature in agent.log ({matched}). The agent is narrating a " + "synthetic [SYSTEM ERROR] back to itself and re-deciding it never " + "read its own instructions, so this run measures the agent fighting " + "its own context rather than its task capability. Root cause is " + "outside this harness: " + "amplifier_module_provider_anthropic/__init__.py:2014." + ), + } + + # Fall back to the narration heuristic. Only the assistant's prose is worth + # scanning: the stderr half of the log is the CLI's own TUI rendering, whose + # re-rendered checklist lines ("Read TASK_INSTRUCTIONS.txt directly") would + # otherwise inflate the count without evidence of an actual re-decision. + prose = text.split(STDERR_SEPARATOR, 1)[0] + narration = len(TOOL_RESULT_LOSS_NARRATION.findall(prose)) + if narration < TOOL_RESULT_LOSS_NARRATION_THRESHOLD: + return None + + return { + "kind": "tool_result_loss", + "confidence": "heuristic", + "count": narration, + "detail": ( + f"{narration} occurrence(s) of the agent re-deciding it had not read a " + "file it already read (threshold " + f"{TOOL_RESULT_LOSS_NARRATION_THRESHOLD}). The literal provider " + "signature was absent, which is expected for CLIs whose stdout carries " + "only assistant prose: the injected [SYSTEM ERROR] reaches the model " + "but is never printed. Treat this run's wall-clock and token figures as " + "contaminated. Root cause is outside this harness: " + "amplifier_module_provider_anthropic/__init__.py:2014." + ), + } + + +def _render_launch_profile(image_alias: str, dest: Path) -> Path: + """Substitute the image-alias placeholder and write the rendered profile. + + Plain string substitution, not DTU's own `${VAR}` launch-variable + mechanism -- that substitution is documented to reach + `provision.setup_cmds`, not `base.image`, so we don't lean on it for the + one field that picks the whole environment. + """ + text = TEMPLATE_PATH.read_text(encoding="utf-8") + if IMAGE_PLACEHOLDER not in text: + raise TrialError(f"{TEMPLATE_PATH} is missing the {IMAGE_PLACEHOLDER!r} placeholder") + dest.write_text(text.replace(IMAGE_PLACEHOLDER, image_alias), encoding="utf-8") + return dest + + +def _flatten_pulled_deliverables(deliverables_dir: Path) -> None: + """Undo `dtu.file_pull`'s `cp -r` basename convention for the output dir. + + Pulling `/workspace/output/` to `deliverables/` lands files at + `deliverables/output/`, because the CLI preserves the source + directory's basename under the destination the same way `file_push` + does (see dtu.py's `_pushed_dir_root`). Every downstream consumer + (grading, deliverable counting) expects `deliverables/` directly, + so move the nested `output/` contents up one level. + """ + nested = deliverables_dir / "output" + if not nested.is_dir(): + return + for item in nested.iterdir(): + target = deliverables_dir / item.name + if target.exists(): + if target.is_dir(): + shutil.rmtree(target) + else: + target.unlink() + item.rename(target) + nested.rmdir() + + +def _dtu_name(agent_name: str, task: Task) -> str: + """`jb---t-`, kept to Incus's + naming budget and unique per (agent, task) so concurrent trials never + collide on the container name. + """ + uuid6 = uuid.uuid4().hex[:6] + occupation = task.occupation[:12] + tail = f"-t{task.task_num}-{uuid6}" + fixed = len("jb-") + len("-") + len(occupation) + len(tail) + agent_short = agent_name[: max(60 - fixed, 1)] + return f"jb-{agent_short}-{occupation}{tail}"[:60] + + +async def run_trial( + agent_name: str, + task: Task, + trial_dir: Path, + *, + model: str, + timeout_s: float = DEFAULT_TIMEOUT_S, + on_stage: Callable[[str], None] | None = None, + agent_kwargs: dict[str, Any] | None = None, +) -> TrialResult: + """Run one trial end to end. + + Sequence: render the launch profile, launch a DTU, let the adapter write + its per-trial config, seed the task folder and prompt, run the agent + under a wall-clock timeout, pull whatever landed in /workspace/output, + destroy the DTU, write trial.json. + + `on_stage`, if given, is called with a short human-readable message at + each major step -- the mechanism for a caller's progress reporting. + Must not raise; this function does not guard against it. + + `agent_kwargs`, if given, is forwarded to the adapter's constructor (e.g. + `{"bundle": "..."}` for amplifier-foundation). Most agents take none. + """ + + def stage(msg: str) -> None: + if on_stage is not None: + on_stage(msg) + + trial_dir.mkdir(parents=True, exist_ok=True) + adapter = agents.get(agent_name, **(agent_kwargs or {})) + image_alias = images.agent_alias(agent_name) + started_at = _now() + + dtu: DTU | None = None + status = "crashed" + exit_code: int | None = None + agent_run_s: float | None = None + error: str | None = None + deliverable_count = 0 + deliverable_bytes = 0 + cost_usd: float | str = NOT_AVAILABLE + total_tokens: int | str = NOT_AVAILABLE + llm_responses: int | str = NOT_AVAILABLE + warnings: list[dict[str, Any]] = [] + + try: + stage(f"rendering launch profile ({image_alias})") + profile_path = _render_launch_profile(image_alias, trial_dir / "launch_profile.yaml") + dtu_name = _dtu_name(agent_name, task) + stage(f"launching DTU {dtu_name}") + dtu = await DTU.launch(profile_path, name=dtu_name) + + stage("configuring agent") + await adapter.configure(dtu, model=model) + + stage("seeding task folder and prompt") + prompt_text = prompt.render() + (trial_dir / "prompt.txt").write_text(prompt_text, encoding="utf-8") + await dtu.file_push(task.task_folder, f"{prompt.WORKSPACE}/") + await dtu.file_push(trial_dir / "prompt.txt", prompt.PROMPT_PATH) + + log_path = trial_dir / "agent.log" + stage(f"running agent (timeout={timeout_s:.0f}s)") + start = time.monotonic() + try: + result = await dtu.exec_cmd( + adapter.command(), + timeout_s=timeout_s + _EXEC_SLACK_S, + stream_to_logfile=log_path, + ) + except DTUError as exc: + agent_run_s = time.monotonic() - start + status = "timeout" + error = str(exc) + with log_path.open("a", encoding="utf-8") as f: + f.write(f"\n--- trial harness: {exc} ---\n") + else: + agent_run_s = time.monotonic() - start + exit_code = result.returncode + status = "completed" if exit_code == 0 else "crashed" + + # Pull whatever landed even after a timeout -- partial output is + # still signal, and a pull of an empty/missing dir is harmless. + stage("pulling deliverables") + deliverables_dir = trial_dir / "deliverables" + try: + await dtu.file_pull(f"{prompt.OUTPUT_DIR}/", deliverables_dir) + except DTUError as exc: + if error is None: + error = str(exc) + + if deliverables_dir.is_dir(): + _flatten_pulled_deliverables(deliverables_dir) + files = [p for p in deliverables_dir.rglob("*") if p.is_file()] + deliverable_count = len(files) + deliverable_bytes = sum(p.stat().st_size for p in files) + + # Upstream also force-fails an exit-0 run that produced nothing -- + # a deliverable-free "success" is not success. + if status == "completed" and deliverable_count == 0: + status = "no_deliverables" + + # Quality signal, deliberately kept separate from `status` above: a + # trial can complete cleanly while the agent spent real wall-clock + # time fighting a provider-injected synthetic error instead of doing + # the task. See `_detect_tool_result_loss`. + tool_result_loss = _detect_tool_result_loss(log_path) + if tool_result_loss is not None: + warnings.append(tool_result_loss) + + # Session/trajectory state, for later cost and token accounting. + # Best effort -- a pull failure here must never take down an + # otherwise good trial, but is logged loudly so a gap in telemetry + # is visible rather than silently absent. + stage("collecting session telemetry") + sessions_dir = trial_dir / "sessions" + for session_path in adapter.session_dirs: + basename = Path(session_path).name + try: + await dtu.file_pull(session_path, sessions_dir / basename) + except Exception as exc: # noqa: BLE001 - telemetry is best effort, must not fail the trial + logger.warning( + "session pull failed for %s (%s): %s -- metrics will be incomplete", + session_path, + agent_name, + exc, + ) + + # metrics.json: the normalized cost/token record for this trial. + # Routed on the adapter's own declared metrics_source, never on agent + # name -- opencode has no events.jsonl at all, so hardcoding the + # events.jsonl path here would search for a file that agent never + # writes and silently report all-not_available, indistinguishable + # from a real collection failure. + # agent_wallclock_s (event-timestamp span) is replaced with our own + # agent_run_s (measured around the agent command) -- the harness's + # own clock is ground truth for how long the trial actually ran. + try: + if adapter.metrics_source == "opencode_db": + metrics_record = normalize_opencode_metrics( + find_opencode_db_files(sessions_dir), + source=adapter.name, + workspace_dir=prompt.WORKSPACE, + ) + else: + metrics_record = normalize_metrics( + find_events_files(sessions_dir), source=adapter.name + ) + metrics_record.pop("agent_wallclock_s", None) + metrics_record["agent_run_s"] = agent_run_s + (trial_dir / "metrics.json").write_text( + json.dumps(metrics_record, indent=2) + "\n", encoding="utf-8" + ) + cost_usd = metrics_record["cost_usd"] + total_tokens = metrics_record["total_tokens"] + llm_responses = metrics_record["llm_responses"] + except Exception as exc: # noqa: BLE001 - telemetry is best effort, must not fail the trial + logger.warning( + "metrics computation failed: %s -- trial.json cost fields stay not_available", exc + ) + + except Exception as exc: # noqa: BLE001 - trial.json must record ANY failure honestly + if status != "timeout": + status = "crashed" + error = error or str(exc) + finally: + finished_at = _now() + if dtu is not None: + stage(f"destroying DTU {dtu.id}") + await dtu.destroy() + + trial_result = TrialResult( + agent=agent_name, + task_id=task.id, + task_selector=task.selector, + split=task.split, + model=model, + dtu_id=dtu.id if dtu is not None else None, + image_alias=image_alias, + status=status, + exit_code=exit_code, + agent_run_s=agent_run_s, + started_at=started_at, + finished_at=finished_at, + deliverable_count=deliverable_count, + deliverable_bytes=deliverable_bytes, + error=error, + cost_usd=cost_usd, + total_tokens=total_tokens, + llm_responses=llm_responses, + warnings=warnings, + ) + trial_result.write(trial_dir / "trial.json") + return trial_result + + +__all__ = ["DEFAULT_TIMEOUT_S", "TrialError", "TrialResult", "run_trial"] diff --git a/.amplifier/evaluation/jobbench/tests/conftest.py b/.amplifier/evaluation/jobbench/tests/conftest.py new file mode 100644 index 00000000..e8dfbad4 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/conftest.py @@ -0,0 +1,19 @@ +"""Make `jobbench` importable without installing the package. + +run.py does the same sys.path insert at its own entry point; tests need it +too since pytest doesn't go through run.py. + +The harness root goes on the path as well so `import run` works: run.py is +the CLI shell, not a package module, but it owns real logic worth pinning +(see test_judge_attribution.py). +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +_ROOT = Path(__file__).parent.parent + +sys.path.insert(0, str(_ROOT / "src")) +sys.path.insert(0, str(_ROOT)) diff --git a/.amplifier/evaluation/jobbench/tests/test_dtu_name.py b/.amplifier/evaluation/jobbench/tests/test_dtu_name.py new file mode 100644 index 00000000..eb4ffe2a --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_dtu_name.py @@ -0,0 +1,68 @@ +"""Unit tests for jobbench.trial._dtu_name -- DTU naming under a concurrent matrix. + +The property that matters for a parallel sweep: launching several agents +against the SAME task must never produce colliding container names, since +Incus refuses to reuse a name that's still in use. +""" + +from __future__ import annotations + +from pathlib import Path + +from jobbench.dataset import Task +from jobbench.trial import _dtu_name + + +def _task(occupation: str = "biostatisticians", num: int = 1) -> Task: + return Task(split="easy", occupation=occupation, task_num=num, root=Path("/nonexistent")) + + +def test_dtu_name_carries_the_agent_name(): + task = _task() + name = _dtu_name("amplifier-agent", task) + assert name.startswith("jb-") + assert "amplifier-agent" in name + + +def test_dtu_name_stays_within_incus_budget(): + task = _task() + for agent in ( + "amplifier-agent", + "amplifier-foundation", + "opencode-amplifier", + "opencode-vanilla", + ): + assert len(_dtu_name(agent, task)) <= 60 + + +def test_dtu_names_differ_across_agents_on_the_same_task(): + """4 agents launched concurrently against one task must not collide.""" + task = _task() + names = { + _dtu_name(agent, task) + for agent in ( + "amplifier-agent", + "amplifier-foundation", + "opencode-amplifier", + "opencode-vanilla", + ) + } + assert len(names) == 4 + # Each name should still be identifiable as belonging to its own agent, + # not just distinct by the random uuid suffix. + for agent in ( + "amplifier-agent", + "amplifier-foundation", + "opencode-amplifier", + "opencode-vanilla", + ): + matching = [n for n in names if agent in n] + assert matching, f"no dtu name carries agent {agent!r}: {names}" + + +def test_dtu_names_are_unique_across_repeated_calls(): + """Repeated launches of the same (agent, task) pair (e.g. --skip-existing + re-runs) must not collide either -- uniqueness comes from the uuid tail.""" + task = _task() + names = {_dtu_name("amplifier-agent", task) for _ in range(20)} + assert len(names) == 20 diff --git a/.amplifier/evaluation/jobbench/tests/test_exec_envelope.py b/.amplifier/evaluation/jobbench/tests/test_exec_envelope.py new file mode 100644 index 00000000..467e5c07 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_exec_envelope.py @@ -0,0 +1,182 @@ +"""Unit tests for the DTU CLI exec JSON-envelope unwrap (fail-loud setup_cmds). + +Current DTU CLI versions report the INNER command's result as a JSON envelope +on stdout ({id, command, exit_code, stdout, stderr}) and exit 0 themselves. +Any caller that checks the outer `CommandResult.returncode` without unwrapping +tests the CLI process, not the command -- a failed in-DTU gate is recorded but +never enforced. Observed live in an earlier evaluation run: all six trials +recorded a FAILED code-identity gate (envelope "exit_code": 1) and ran to +full metered completion. + +Adapted from amplifier-bundle-evaluation's harness test suite. The +install_agent/grader-path composed-state tests were dropped -- those modules +aren't vendored here -- but every assertion on `_unwrap_exec_envelope` itself +is kept intact; that's the function this harness actually depends on. +""" + +from __future__ import annotations + +import json + +from jobbench.dtu import _unwrap_exec_envelope + +# A realistically shaped failing envelope: the outer CLI exits 0 while the +# inner command exited 1, with multi-line stdout and stderr both present. +FAILED_ENVELOPE = json.dumps( + { + "id": "dtu-1a2b3c4d", + "command": "bash -lc 'set -e\\n...setup verification...'", + "exit_code": 1, + "stdout": ( + "--- setup verification ---\n" + "checking installed package version\n" + "found version 0.1.0, expected 0.2.0\n" + ), + "stderr": ( + "ERROR: setup verification failed -- installed version does not " + "match the requested version\n" + "hint: rerun the install step before continuing\n" + ), + } +) + + +# --------------------------------------------------------------------------- +# _unwrap_exec_envelope unit contract +# --------------------------------------------------------------------------- + + +def test_unwrap_failed_envelope(): + rc, stdout, stderr = _unwrap_exec_envelope(0, FAILED_ENVELOPE, "") + assert rc == 1 + assert "found version 0.1.0, expected 0.2.0" in stdout + assert "setup verification failed" in stderr + + +def test_unwrap_success_envelope(): + env = json.dumps( + { + "id": "dtu-x", + "command": "bash -lc 'true'", + "exit_code": 0, + "stdout": "ok\n", + "stderr": "", + } + ) + rc, stdout, stderr = _unwrap_exec_envelope(0, env, "") + assert (rc, stdout, stderr) == (0, "ok\n", "") + + +def test_plain_stdout_passthrough(): + """Plain (non-JSON) stdout -- e.g. `--stream` mode output or a mock + backend that doesn't wrap -- passes through untouched.""" + rc, stdout, stderr = _unwrap_exec_envelope(1, "boom\n", "err\n") + assert (rc, stdout, stderr) == (1, "boom\n", "err\n") + rc, stdout, stderr = _unwrap_exec_envelope(0, "hello world\n", "") + assert (rc, stdout, stderr) == (0, "hello world\n", "") + + +def test_json_lookalike_stdout_passthrough(): + """A command whose real output is JSON without the envelope keys passes through.""" + payload = json.dumps({"result": "ok", "count": 3}) + rc, stdout, stderr = _unwrap_exec_envelope(0, payload, "") + assert (rc, stdout, stderr) == (0, payload, "") + + +def test_non_int_exit_code_passthrough(): + env = json.dumps({"command": "x", "exit_code": "1", "stdout": "", "stderr": ""}) + assert _unwrap_exec_envelope(0, env, "")[0] == 0 + + +def test_warning_on_unrecognizable_stdout(caplog): + """Outer success + unrecognizable stdout passes through, but LOUDLY: + the real CLI always envelopes, so plain output at rc 0 means the + envelope shape drifted -- that must show up in logs, not vanish.""" + import logging + + with caplog.at_level(logging.WARNING, logger="jobbench.dtu"): + rc, stdout, stderr = _unwrap_exec_envelope(0, "hello world\n", "") + assert (rc, stdout, stderr) == (0, "hello world\n", "") + assert any("not a recognizable JSON envelope" in r.message for r in caplog.records) + + # A proper envelope unwraps silently -- no drift warning. + caplog.clear() + with caplog.at_level(logging.WARNING, logger="jobbench.dtu"): + _unwrap_exec_envelope(0, FAILED_ENVELOPE, "") + assert not caplog.records + + # Outer CLI failure passes through silently too (raw contract applies). + caplog.clear() + with caplog.at_level(logging.WARNING, logger="jobbench.dtu"): + _unwrap_exec_envelope(7, "boom\n", "cli blew up") + assert not caplog.records + + +def test_last_line_envelope_scan(): + """An envelope preceded by other output on the same stream (e.g. a + wrapper banner) is still found via the last-line fallback.""" + stdout = "some banner line\nanother line\n" + FAILED_ENVELOPE + "\n" + rc, inner_stdout, stderr = _unwrap_exec_envelope(0, stdout, "") + assert rc == 1 + assert "found version 0.1.0, expected 0.2.0" in inner_stdout + assert "setup verification failed" in stderr + + +def test_nested_output_envelope_unwrapped(): + """Some CLI versions nest the envelope fields under an "output" key -- + unwrapped only after the flat 4-key gate fails.""" + env = json.dumps( + { + "id": "dtu-x", + "output": { + "command": "bash -lc 'exit 3'", + "exit_code": 3, + "stdout": "partial\n", + "stderr": "gate failed\n", + }, + } + ) + rc, stdout, stderr = _unwrap_exec_envelope(0, env, "") + assert (rc, stdout, stderr) == (3, "partial\n", "gate failed\n") + + +def test_flat_envelope_wins_over_nested_output(): + """Flat-gate-first ordering: a flat envelope that also carries an + "output" sub-object is never shadowed by it.""" + env = json.dumps( + { + "command": "c", + "exit_code": 5, + "stdout": "flat\n", + "stderr": "", + "output": { + "command": "x", + "exit_code": 9, + "stdout": "nested\n", + "stderr": "", + }, + } + ) + rc, stdout, _ = _unwrap_exec_envelope(0, env, "") + assert (rc, stdout) == (5, "flat\n") + + +def test_lookalike_with_output_subobject_passthrough(): + """JSON command output whose "output" value is not an envelope still + passes through -- the nested tolerance doesn't widen the lookalike net.""" + payload = json.dumps({"output": {"result": "ok"}, "count": 3}) + rc, stdout, stderr = _unwrap_exec_envelope(0, payload, "") + assert (rc, stdout, stderr) == (0, payload, "") + + +def test_outer_failure_never_unwrapped(): + """Outer CLI failure (timeout, container gone) is reported as-is.""" + rc, _, _ = _unwrap_exec_envelope(7, FAILED_ENVELOPE, "cli blew up") + assert rc == 7 + + +def test_outer_stderr_preserved_alongside_inner(): + rc, _, stderr = _unwrap_exec_envelope(0, FAILED_ENVELOPE, "outer warning\n") + assert rc == 1 + assert "setup verification failed" in stderr + assert "outer warning" in stderr diff --git a/.amplifier/evaluation/jobbench/tests/test_file_push.py b/.amplifier/evaluation/jobbench/tests/test_file_push.py new file mode 100644 index 00000000..041071f5 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_file_push.py @@ -0,0 +1,141 @@ +"""Unit tests for `file-push` command construction and fail-loud guards. + +NOTE: current DTU CLI versions auto-detect directory sources and push them +recursively regardless of `--recursive` (DTU PR #18). We still pass the +flag for directory sources as compatibility with older CLI versions that +predate auto-detect, and OMIT it for plain files (with `--recursive` the +CLI treats the destination as a parent directory instead of an exact file +path). These tests pin that argv contract for `harness.dtu.DTU.file_push`. + +Directory pushes also carry a fail-loud guard: if the CLI reports success +but the directory did not land inside the DTU, the push raises instead of +proceeding silently (silently-empty mounts corrupt grading). + +Adapted from amplifier-bundle-evaluation's harness test suite. The +grader._push_mounts tests were dropped -- that module isn't vendored here. +""" + +from __future__ import annotations + +import asyncio +from pathlib import Path + +import pytest + +from jobbench import dtu as dtu_module +from jobbench.dtu import CLI, DTU, CommandResult, DTUError + + +class FakeRun: + """Stand-in for `dtu._run` that records argv and returns a fixed result.""" + + def __init__(self, returncode: int = 0, stderr: str = ""): + self.calls: list[list[str]] = [] + self.returncode = returncode + self.stderr = stderr + + async def __call__(self, args, *, timeout=None, env=None): + self.calls.append(list(args)) + return (self.returncode, "", self.stderr) + + +class FakeExec: + """Stand-in for `DTU.exec_cmd` (bound as an instance attribute).""" + + def __init__(self, returncode: int = 0): + self.calls: list[list[str]] = [] + self.returncode = returncode + + async def __call__(self, command, *, timeout_s=None, stream_to_logfile=None): + self.calls.append(list(command)) + return CommandResult(returncode=self.returncode, stdout="", stderr="", elapsed_s=0.0) + + +def _dtu() -> DTU: + return DTU(id="dtu-test", profile_path="profile.yaml") + + +# --------------------------------------------------------------------------- +# DTU.file_push +# --------------------------------------------------------------------------- + + +def test_file_push_file_omits_recursive(tmp_path: Path, monkeypatch): + src = tmp_path / "a.txt" + src.write_text("hello", encoding="utf-8") + run = FakeRun() + monkeypatch.setattr(dtu_module, "_run", run) + + asyncio.run(_dtu().file_push(src, "/workspace/a.txt")) + + assert run.calls == [[CLI, "file-push", "dtu-test", str(src), "/workspace/a.txt"]] + + +def test_file_push_dir_adds_recursive_and_verifies(tmp_path: Path, monkeypatch): + src = tmp_path / "data" + src.mkdir() + (src / "x.txt").write_text("x", encoding="utf-8") + run = FakeRun() + monkeypatch.setattr(dtu_module, "_run", run) + dtu = _dtu() + fake_exec = FakeExec(returncode=0) + dtu.exec_cmd = fake_exec # type: ignore[method-assign] + + asyncio.run(dtu.file_push(src, "/workspace/")) + + assert run.calls == [[CLI, "file-push", "--recursive", "dtu-test", str(src), "/workspace/"]] + # Post-push verification checks the landed directory (name preserved). + assert len(fake_exec.calls) == 1 + shell, flag, script = fake_exec.calls[0] + assert (shell, flag) == ("sh", "-c") + assert "test -d /workspace/data" in script + assert "ls -A" in script # non-empty source requires non-empty destination + + +def test_file_push_empty_dir_skips_content_check(tmp_path: Path, monkeypatch): + src = tmp_path / "empty" + src.mkdir() + monkeypatch.setattr(dtu_module, "_run", FakeRun()) + dtu = _dtu() + fake_exec = FakeExec(returncode=0) + dtu.exec_cmd = fake_exec # type: ignore[method-assign] + + asyncio.run(dtu.file_push(src, "/workspace/")) + + script = fake_exec.calls[0][2] + assert "test -d /workspace/empty" in script + assert "ls -A" not in script + + +def test_file_push_dir_undelivered_raises(tmp_path: Path, monkeypatch): + src = tmp_path / "data" + src.mkdir() + (src / "x.txt").write_text("x", encoding="utf-8") + monkeypatch.setattr(dtu_module, "_run", FakeRun()) + dtu = _dtu() + dtu.exec_cmd = FakeExec(returncode=1) # type: ignore[method-assign] + + with pytest.raises(DTUError, match="missing or empty"): + asyncio.run(dtu.file_push(src, "/workspace/")) + + +def test_file_push_cli_error_raises(tmp_path: Path, monkeypatch): + src = tmp_path / "data" + src.mkdir() + monkeypatch.setattr(dtu_module, "_run", FakeRun(returncode=2, stderr="boom")) + dtu = _dtu() + fake_exec = FakeExec() + dtu.exec_cmd = fake_exec # type: ignore[method-assign] + + with pytest.raises(DTUError, match="file-push failed"): + asyncio.run(dtu.file_push(src, "/workspace/")) + assert fake_exec.calls == [] # no verification after a failed push + + +def test_file_push_missing_source_raises(tmp_path: Path, monkeypatch): + run = FakeRun() + monkeypatch.setattr(dtu_module, "_run", run) + + with pytest.raises(DTUError, match="source missing"): + asyncio.run(_dtu().file_push(tmp_path / "nope", "/workspace/")) + assert run.calls == [] diff --git a/.amplifier/evaluation/jobbench/tests/test_grading.py b/.amplifier/evaluation/jobbench/tests/test_grading.py new file mode 100644 index 00000000..af810d88 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_grading.py @@ -0,0 +1,367 @@ +"""Unit tests for jobbench.grading: score/provenance coupling in trial.json. + +grading.py is the module that WRITES the scores, so the invariant under test +is not "does the judge judge well" but "can a reader ever pair a score with a +judge that did not produce it". The answer must be no, in both directions: + + - success -> score fields AND `judge_model` written together + - failure -> score fields AND `judge_model` ALL cleared, `grade_error` set + +The failure direction is the one that bites. A re-grade of an existing run +starts from a trial.json that may already hold judge A's score; if judge B +then fails and only `grade_error` is set, the file still shows judge A's +number while the run manifest claims judge B graded the run. That is +fabricated provenance, and it is worse than a missing score. + +`grading.grade` (the judge subprocess) is monkeypatched in every test here -- +nothing invokes a real judge, spends an API call, or reads a real rubric. All +task/report content is synthetic; no real JobBench task text, rubric text, or +judge output appears in this file. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from jobbench import grading +from jobbench.dataset import Task +from jobbench.grading import GradingError, _safe_name, grade_and_record + + +def _task() -> Task: + """Filesystem-free task handle. Only `.selector` is read on these paths + (for the progress line), never the task folder or rubric file. + """ + return Task(split="easy", occupation="biostatisticians", task_num=1, root=Path("/nonexistent")) + + +def _report(total: float = 7.0, maximum: float = 10.0, passed: int = 2, count: int = 3) -> dict: + """A synthetic judge details report, shaped like judge.py's + `build_details_report` output but carrying no rubric text. + """ + return { + "total_score": total, + "max_score": maximum, + "passed_count": passed, + "total_count": count, + "rubrics": [], + } + + +def _write_trial(trial_dir: Path, data: dict) -> Path: + trial_dir.mkdir(parents=True, exist_ok=True) + path = trial_dir / "trial.json" + path.write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8") + return path + + +def _read_trial(trial_dir: Path) -> dict: + return json.loads((trial_dir / "trial.json").read_text(encoding="utf-8")) + + +def _record(trial_dir: Path, *, judge_model: str, stages: list[str] | None = None) -> bool: + """Call grade_and_record with the arguments run.py passes it.""" + return grade_and_record( + trial_dir, + _task(), + agent="synthetic-agent", + judge_model=judge_model, + api_base=None, + api_key=None, + max_workers=1, + timeout_per_rubric=1, + on_stage=(stages.append if stages is not None else None), + ) + + +# --------------------------------------------------------------------------- +# _safe_name: judge model ids become a path component +# --------------------------------------------------------------------------- + + +def test_safe_name_passes_through_ordinary_model_ids(): + assert _safe_name("gpt-5.6-terra") == "gpt-5.6-terra" + + +def test_safe_name_neutralizes_path_separators(): + """A vendor-prefixed id (`openai/gpt-x`) must not become a subdirectory.""" + result = _safe_name("openai/gpt-x") + assert "/" not in result + assert result == "openai-gpt-x" + + +def test_safe_name_neutralizes_traversal(): + """`../..` must not be able to walk out of the grade directory. + + Dots are deliberately KEPT (real model ids carry them, e.g. `gpt-5.6`), + so the traversal defense is the separator collapse, not dot removal: + `../../etc/passwd` becomes one inert filename fragment, not three + directory levels. + """ + result = _safe_name("../../etc/passwd") + assert "/" not in result + assert result == "..-..-etc-passwd" + assert not result.startswith("-") + + +def test_safe_name_neutralizes_windows_separators_and_colons(): + result = _safe_name("azure:models\\gpt-x") + assert ":" not in result + assert "\\" not in result + + +def test_safe_name_never_returns_empty(): + """An id made entirely of stripped characters must still yield a usable + filename fragment rather than collapsing to '' (which would make the + details file a bare '_judge.json' or, worse, a directory write). + """ + assert _safe_name("///") == "judge" + assert _safe_name("") == "judge" + + +def test_safe_name_result_is_a_single_path_component(tmp_path: Path): + """The property that actually matters: joining it stays inside the dir.""" + joined = tmp_path / f"{_safe_name('../../../evil')}_judge.json" + assert joined.parent == tmp_path + + +# --------------------------------------------------------------------------- +# grade_and_record: success path +# --------------------------------------------------------------------------- + + +def test_success_writes_score_and_stamps_the_judge(tmp_path: Path, monkeypatch): + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "completed", "deliverable_count": 3}) + monkeypatch.setattr(grading, "grade", lambda *a, **k: _report()) + + assert _record(trial_dir, judge_model="judge-b") is True + + data = _read_trial(trial_dir) + assert data["total_score"] == 7.0 + assert data["max_score"] == 10.0 + assert data["passed_count"] == 2 + assert data["total_count"] == 3 + assert data["grade_error"] is None + # Provenance travels with the score, so a reader never has to consult the + # run manifest to learn which judge produced this number. + assert data["judge_model"] == "judge-b" + + +def test_success_preserves_unrelated_trial_fields(tmp_path: Path, monkeypatch): + """Grading merges into trial.json; it must not rewrite the run record.""" + trial_dir = tmp_path / "trial" + _write_trial( + trial_dir, + {"status": "completed", "agent": "synthetic-agent", "exit_code": 0, "warnings": []}, + ) + monkeypatch.setattr(grading, "grade", lambda *a, **k: _report()) + + _record(trial_dir, judge_model="judge-b") + + data = _read_trial(trial_dir) + assert data["status"] == "completed" + assert data["agent"] == "synthetic-agent" + assert data["exit_code"] == 0 + assert data["warnings"] == [] + + +def test_legitimate_zero_score_is_a_success_not_a_failure(tmp_path: Path, monkeypatch): + """An all-fail report is a real grade. It must return True and record 0, + NOT be conflated with the judge having failed to run. + """ + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "no_deliverables"}) + monkeypatch.setattr( + grading, "grade", lambda *a, **k: _report(total=0.0, maximum=10.0, passed=0, count=3) + ) + + assert _record(trial_dir, judge_model="judge-b") is True + + data = _read_trial(trial_dir) + assert data["total_score"] == 0.0 + assert data["grade_error"] is None + assert data["judge_model"] == "judge-b" + assert data["status"] == "no_deliverables" + + +def test_missing_trial_json_is_created_on_success(tmp_path: Path, monkeypatch): + """Grading a trial dir whose trial.json never got written still records a + score rather than raising -- the score is the point. + """ + trial_dir = tmp_path / "trial" + trial_dir.mkdir() + monkeypatch.setattr(grading, "grade", lambda *a, **k: _report()) + + assert _record(trial_dir, judge_model="judge-b") is True + assert _read_trial(trial_dir)["judge_model"] == "judge-b" + + +# --------------------------------------------------------------------------- +# grade_and_record: failure path -- no stale score may survive +# --------------------------------------------------------------------------- + + +def test_failed_regrade_does_not_leave_a_prior_judges_score_behind( + tmp_path: Path, monkeypatch +) -> None: + """The bug this pins: judge A scored the trial, judge B is asked to + re-grade, judge B fails. If only `grade_error` were set, trial.json would + still show 9.0/10 while the run now claims judge B graded it -- a number + attributed to a judge that never produced it. + """ + trial_dir = tmp_path / "trial" + _write_trial( + trial_dir, + { + "status": "completed", + "total_score": 9.0, + "max_score": 10.0, + "passed_count": 3, + "total_count": 3, + "judge_model": "judge-a", + "grade_error": None, + }, + ) + + def _boom(*args, **kwargs): + raise GradingError("judge exited 1") + + monkeypatch.setattr(grading, "grade", _boom) + + assert _record(trial_dir, judge_model="judge-b") is False + + data = _read_trial(trial_dir) + assert data["total_score"] is None, "judge A's score survived a failed judge-B re-grade" + assert data["max_score"] is None + assert data["passed_count"] is None + assert data["total_count"] is None + assert data["judge_model"] is None, "judge A's attribution survived a failed judge-B re-grade" + assert "judge exited 1" in data["grade_error"] + + +def test_failure_does_not_clobber_status(tmp_path: Path, monkeypatch): + """`status` (did it run) and the grade fields (what did it score) are + separate keys. A grading failure must not overwrite a crash into + something else, or a crashed trial and a scored-zero trial stop being + distinguishable. + """ + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "crashed", "exit_code": 1, "error": "synthetic failure"}) + + def _boom(*args, **kwargs): + raise GradingError("judge wrote no details file") + + monkeypatch.setattr(grading, "grade", _boom) + + assert _record(trial_dir, judge_model="judge-b") is False + + data = _read_trial(trial_dir) + assert data["status"] == "crashed" + assert data["exit_code"] == 1 + assert data["error"] == "synthetic failure" + assert data["grade_error"] is not None + + +def test_failure_after_a_clean_first_grade_still_clears(tmp_path: Path, monkeypatch): + """Two passes in sequence through the real code path: pass 1 succeeds with + judge A, pass 2 fails with judge B. Nothing from pass 1 may remain. + """ + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "completed"}) + + monkeypatch.setattr(grading, "grade", lambda *a, **k: _report()) + assert _record(trial_dir, judge_model="judge-a") is True + assert _read_trial(trial_dir)["judge_model"] == "judge-a" + + def _boom(*args, **kwargs): + raise GradingError("judge exited 137") + + monkeypatch.setattr(grading, "grade", _boom) + assert _record(trial_dir, judge_model="judge-b") is False + + data = _read_trial(trial_dir) + assert data["total_score"] is None + assert data["judge_model"] is None + assert "137" in data["grade_error"] + + +def test_only_gradingerror_is_caught(tmp_path: Path, monkeypatch): + """An unexpected exception is a harness bug, not a grading outcome. It + must propagate rather than be silently recorded as `grade_error` -- a + swallowed bug would look identical to a judge that merely failed. + """ + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "completed"}) + + def _bug(*args, **kwargs): + raise ValueError("harness bug, not a grading failure") + + monkeypatch.setattr(grading, "grade", _bug) + + with pytest.raises(ValueError, match="harness bug"): + _record(trial_dir, judge_model="judge-b") + + +# --------------------------------------------------------------------------- +# on_stage hook +# --------------------------------------------------------------------------- + + +def test_on_stage_receives_the_score_line_instead_of_stdout(tmp_path: Path, monkeypatch, capsys): + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "completed"}) + monkeypatch.setattr(grading, "grade", lambda *a, **k: _report()) + stages: list[str] = [] + + _record(trial_dir, judge_model="judge-b", stages=stages) + + assert any("score" in line for line in stages) + captured = capsys.readouterr() + assert captured.out == "" + + +def test_on_stage_receives_the_error_line_instead_of_stderr(tmp_path: Path, monkeypatch, capsys): + trial_dir = tmp_path / "trial" + _write_trial(trial_dir, {"status": "completed"}) + + def _boom(*args, **kwargs): + raise GradingError("judge exited 1") + + monkeypatch.setattr(grading, "grade", _boom) + stages: list[str] = [] + + _record(trial_dir, judge_model="judge-b", stages=stages) + + assert any("grading failed" in line for line in stages) + captured = capsys.readouterr() + assert captured.err == "" + + +# --------------------------------------------------------------------------- +# resolve_credentials +# --------------------------------------------------------------------------- + + +def test_resolve_credentials_prefers_cli_args_over_env(monkeypatch): + monkeypatch.setenv("OPENAI_BASE_URL", "https://env.invalid") + monkeypatch.setenv("OPENAI_API_KEY", "env-key") + assert grading.resolve_credentials("https://arg.invalid", "arg-key") == ( + "https://arg.invalid", + "arg-key", + ) + + +def test_resolve_credentials_falls_back_to_env(monkeypatch): + monkeypatch.setenv("OPENAI_BASE_URL", "https://env.invalid") + monkeypatch.setenv("OPENAI_API_KEY", "env-key") + assert grading.resolve_credentials(None, None) == ("https://env.invalid", "env-key") + + +def test_resolve_credentials_returns_none_when_unset(monkeypatch): + monkeypatch.delenv("OPENAI_BASE_URL", raising=False) + monkeypatch.delenv("OPENAI_API_KEY", raising=False) + assert grading.resolve_credentials(None, None) == (None, None) diff --git a/.amplifier/evaluation/jobbench/tests/test_judge_attribution.py b/.amplifier/evaluation/jobbench/tests/test_judge_attribution.py new file mode 100644 index 00000000..a4729e31 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_judge_attribution.py @@ -0,0 +1,118 @@ +"""Unit tests for run.py's `_record_judge_attribution` -- run-level provenance. + +A top-level `judge_model` in the run manifest is a claim that this judge +produced EVERY score in the run. The failure mode being pinned is the same +one grading.py guards at the per-trial level, one layer up: a run graded by +judge A, then partially re-graded by judge B, must not end up with judge A's +run-level claim still standing over a mix of scores neither judge fully +produced. + +The helper is pure dict surgery -- no filesystem, no judge, no API calls, no +benchmark content. The manifests here are synthetic. +""" + +from __future__ import annotations + +import run +from run import _JUDGE_ATTRIBUTION_KEYS, _record_judge_attribution + + +def test_complete_pass_claims_the_judge(): + manifest: dict = {"run_id": "synthetic"} + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=0) + + assert manifest["grading_complete"] is True + assert manifest["judge_model"] == "judge-b" + assert manifest["judge_reasoning_effort"] == run.JUDGE_REASONING_EFFORT + # A complete pass makes no partial-pass claims. + assert "attempted_judge_model" not in manifest + assert "ungraded_trials" not in manifest + + +def test_partial_pass_records_attempt_not_a_claim(): + manifest: dict = {"run_id": "synthetic"} + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=2) + + assert manifest["grading_complete"] is False + assert manifest["attempted_judge_model"] == "judge-b" + assert manifest["attempted_judge_reasoning_effort"] == run.JUDGE_REASONING_EFFORT + assert manifest["ungraded_trials"] == 2 + # The bare key is the whole-run claim; a partial pass has not earned it. + assert "judge_model" not in manifest + + +def test_failed_regrade_removes_the_previous_judges_claim(): + """The important one: judge A graded the run completely, judge B then + re-grades and misses 3 trials. Judge A's top-level `judge_model` must be + REMOVED, not left standing -- otherwise the manifest asserts judge A + graded every score in a run that judge B has since partially rewritten. + """ + manifest: dict = {"run_id": "synthetic"} + _record_judge_attribution(manifest, judge_model="judge-a", ungraded=0) + assert manifest["judge_model"] == "judge-a" + + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=3) + + assert "judge_model" not in manifest, "judge A's whole-run claim survived a partial re-grade" + assert "judge_reasoning_effort" not in manifest + assert manifest["grading_complete"] is False + assert manifest["attempted_judge_model"] == "judge-b" + assert manifest["ungraded_trials"] == 3 + + +def test_recovered_regrade_clears_the_partial_markers(): + """The reverse direction: a partial pass followed by a complete one must + not leave `ungraded_trials`/`attempted_judge_model` behind contradicting + the new complete claim. + """ + manifest: dict = {"run_id": "synthetic"} + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=3) + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=0) + + assert manifest["judge_model"] == "judge-b" + assert manifest["grading_complete"] is True + assert "attempted_judge_model" not in manifest + assert "attempted_judge_reasoning_effort" not in manifest + assert "ungraded_trials" not in manifest + + +def test_every_owned_key_is_rewritten_wholesale(): + """Whatever a previous pass wrote, none of the owned keys may survive + untouched into the next pass. Pinned against the key list itself so a + newly-added attribution key cannot be forgotten in the reset loop. + """ + manifest: dict = {key: "stale-value-from-a-previous-pass" for key in _JUDGE_ATTRIBUTION_KEYS} + manifest["run_id"] = "synthetic" + + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=1) + + for key, value in manifest.items(): + if key == "run_id": + continue + assert value != "stale-value-from-a-previous-pass", f"{key} was not rewritten" + + +def test_unrelated_manifest_fields_are_untouched(): + manifest: dict = { + "run_id": "synthetic", + "agents": ["synthetic-agent"], + "split": "easy", + "trials": 4, + } + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=0) + + assert manifest["run_id"] == "synthetic" + assert manifest["agents"] == ["synthetic-agent"] + assert manifest["split"] == "easy" + assert manifest["trials"] == 4 + + +def test_grading_complete_is_always_written(): + """Present on both branches, so 'was this run fully graded' is never + answered by the absence of a key. + """ + for ungraded in (0, 1, 99): + manifest: dict = {} + _record_judge_attribution(manifest, judge_model="judge-b", ungraded=ungraded) + assert "grading_complete" in manifest + assert manifest["grading_complete"] is (ungraded == 0) diff --git a/.amplifier/evaluation/jobbench/tests/test_matrix.py b/.amplifier/evaluation/jobbench/tests/test_matrix.py new file mode 100644 index 00000000..bfb38c5d --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_matrix.py @@ -0,0 +1,136 @@ +"""Unit tests for jobbench.matrix: --agent/--task parsing, validation, matrix build. + +Fixtures build a synthetic split under a temp JOBBENCH_CACHE_DIR rather than +touching the real (gitignored) dataset-cache -- these tests never fetch or +read real JobBench task/rubric content. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from jobbench import matrix +from jobbench.agents import names as agent_names +from jobbench.dataset import DatasetError + + +def _write_task(root: Path, occupation: str, task_num: int, *, weight: int = 5) -> None: + task_dir = root / occupation / f"task{task_num}" + folder = task_dir / "task_folder" + folder.mkdir(parents=True) + (folder / "TASK_INSTRUCTIONS.txt").write_text("do the synthetic thing\n", encoding="utf-8") + (task_dir / "RUBRICS.json").write_text( + json.dumps({"rubrics": [{"weight": weight, "criterion": "did the synthetic thing"}]}), + encoding="utf-8", + ) + + +@pytest.fixture +def fake_split(tmp_path, monkeypatch): + """A synthetic 'easy' split: 2 occupations, 3 tasks total. Content is + entirely fabricated -- no real JobBench task/rubric text. + """ + cache_dir = tmp_path / "cache" + monkeypatch.setenv("JOBBENCH_CACHE_DIR", str(cache_dir)) + split_dir = cache_dir / "easy" + _write_task(split_dir, "biostatisticians", 1) + _write_task(split_dir, "biostatisticians", 2) + _write_task(split_dir, "lawyer", 1) + return "easy" + + +# --------------------------------------------------------------------------- +# --agent parsing / validation +# --------------------------------------------------------------------------- + + +def test_resolve_agent_names_repeated_flags(): + names = agent_names() + a, b = names[0], names[1] + assert matrix.resolve_agent_names([a, b]) == [a, b] + + +def test_resolve_agent_names_comma_separated(): + names = agent_names() + a, b = names[0], names[1] + assert matrix.resolve_agent_names([f"{a},{b}"]) == [a, b] + + +def test_resolve_agent_names_mixed_repeat_and_comma_dedupes(): + names = agent_names() + a, b = names[0], names[1] + # repeated flag AND comma-separated, with a duplicate thrown in -- order + # of first appearance wins, dupes collapse. + result = matrix.resolve_agent_names([f"{a},{b}", a]) + assert result == [a, b] + + +def test_resolve_agent_names_all_expands_from_registry_not_hardcoded(): + assert matrix.resolve_agent_names(["all"]) == agent_names() + + +def test_resolve_agent_names_unknown_raises_before_anything_else(): + with pytest.raises(matrix.MatrixError, match="unknown agent"): + matrix.resolve_agent_names(["amplifier-agent", "not-a-real-agent"]) + + +def test_resolve_agent_names_empty_raises(): + with pytest.raises(matrix.MatrixError): + matrix.resolve_agent_names(None) + with pytest.raises(matrix.MatrixError): + matrix.resolve_agent_names([]) + + +# --------------------------------------------------------------------------- +# --task / --all-tasks +# --------------------------------------------------------------------------- + + +def test_resolve_tasks_repeated_and_comma_selectors(fake_split): + tasks = matrix.resolve_tasks( + split=fake_split, + raw=["biostatisticians/task1,biostatisticians/task2"], + all_tasks=False, + ) + assert [t.selector for t in tasks] == ["biostatisticians/task1", "biostatisticians/task2"] + + +def test_resolve_tasks_all_tasks_wins_over_raw(fake_split): + tasks = matrix.resolve_tasks(split=fake_split, raw=["ignored/selector"], all_tasks=True) + assert len(tasks) == 3 + + +def test_resolve_tasks_requires_task_or_all_tasks(fake_split): + with pytest.raises(matrix.MatrixError): + matrix.resolve_tasks(split=fake_split, raw=None, all_tasks=False) + + +def test_resolve_tasks_unknown_selector_raises(fake_split): + with pytest.raises(DatasetError): + matrix.resolve_tasks(split=fake_split, raw=["nope/task99"], all_tasks=False) + + +# --------------------------------------------------------------------------- +# matrix construction +# --------------------------------------------------------------------------- + + +def test_build_matrix_is_agent_major_cross_product(fake_split): + tasks = matrix.resolve_tasks(split=fake_split, raw=None, all_tasks=True) + pairs = matrix.build_matrix(["agent-a", "agent-b"], tasks) + assert len(pairs) == 2 * len(tasks) + assert [p.agent for p in pairs[: len(tasks)]] == ["agent-a"] * len(tasks) + assert [p.agent for p in pairs[len(tasks) :]] == ["agent-b"] * len(tasks) + assert pairs[0].label == f"agent-a {tasks[0].selector}" + + +def test_estimate_cost_scales_linearly_with_trial_count(): + low1, mean1, high1 = matrix.estimate_cost(1) + low10, mean10, high10 = matrix.estimate_cost(10) + assert low10 == pytest.approx(low1 * 10) + assert mean10 == pytest.approx(mean1 * 10) + assert high10 == pytest.approx(high1 * 10) + assert low1 < mean1 < high1 diff --git a/.amplifier/evaluation/jobbench/tests/test_metrics.py b/.amplifier/evaluation/jobbench/tests/test_metrics.py new file mode 100644 index 00000000..47e31d78 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_metrics.py @@ -0,0 +1,219 @@ +"""Pin the cross-source token-accounting convention. + +The two token sources this module normalizes disagree on what "input" means at +the source, so `input_tokens` is normalized to FRESH-ONLY in both branches and +the four token fields are disjoint: + + input_tokens fresh input only, never previously cached + cache_read_tokens input served from cache + cache_write_tokens input written into cache + output_tokens generated output + total_tokens the sum of all four + +- opencode's `session.tokens_input` column is already fresh-only, so + `parse_opencode_db` accumulates it as-is. +- The amplifier stacks fold cache_read INTO their reported `input_tokens` (but + not cache_write), so `parse_events` subtracts it back out. + +These tests pin both sides with synthetic data, so a regression in either +parser is caught here rather than as a silent cross-run number mismatch. + +All values below are synthetic and invented for this test; none reflect any +real benchmark task or rubric content. +""" + +from __future__ import annotations + +import json +import sqlite3 +from pathlib import Path + +from jobbench.metrics import normalize_metrics, normalize_opencode_metrics + + +def _build_opencode_db( + path: Path, + *, + directory: str, + tokens_input: int, + tokens_output: int, + tokens_cache_read: int, + tokens_cache_write: int, + cost: float = 0.01, + model: str = "claude-sonnet-5", +) -> None: + """Write a minimal, synthetic single-session opencode.db to `path`. + + Only the columns `_opencode_sessions`/`parse_opencode_db` actually read + are populated; this is not a full schema mirror of real opencode.db. + """ + con = sqlite3.connect(path) + try: + con.execute( + """ + CREATE TABLE session ( + id TEXT, + directory TEXT, + tokens_input INTEGER, + tokens_output INTEGER, + tokens_cache_read INTEGER, + tokens_cache_write INTEGER, + cost REAL, + model TEXT, + time_created INTEGER, + time_updated INTEGER + ) + """ + ) + con.execute("CREATE TABLE message (session_id TEXT, data TEXT)") + con.execute( + """ + INSERT INTO session VALUES + ('sess-1', ?, ?, ?, ?, ?, ?, ?, 1000, 2000) + """, + ( + directory, + tokens_input, + tokens_output, + tokens_cache_read, + tokens_cache_write, + cost, + json.dumps({"id": model, "providerID": "anthropic", "variant": "default"}), + ), + ) + con.execute( + "INSERT INTO message VALUES ('sess-1', ?)", + (json.dumps({"role": "assistant"}),), + ) + con.commit() + finally: + con.close() + + +def test_opencode_input_is_fresh_only_and_total_sums_all_four(tmp_path: Path) -> None: + """opencode's column is already fresh-only, so it is accumulated as-is. + + Shape mirrors the real discrepancy: a tiny tokens_input alongside a huge + cache_read. The cache reads must land in total_tokens, which the old + `input + output` formula dropped entirely. + """ + db_path = tmp_path / "opencode.db" + _build_opencode_db( + db_path, + directory="/workspace", + tokens_input=134, + tokens_output=46015, + tokens_cache_read=4_077_645, + tokens_cache_write=104_929, + ) + + record = normalize_opencode_metrics( + [str(db_path)], source="opencode-vanilla-synthetic", workspace_dir="/workspace" + ) + + assert record["input_tokens"] == 134 + assert record["cache_read"] == 4_077_645 + assert record["cache_write"] == 104_929 + assert record["total_tokens"] == 134 + 4_077_645 + 104_929 + 46015 + + +def test_opencode_cost_is_unaffected_by_the_token_normalization(tmp_path: Path) -> None: + """Cost bills each token type at its own rate and must not track total_tokens.""" + db_path = tmp_path / "opencode.db" + _build_opencode_db( + db_path, + directory="/workspace", + tokens_input=100, + tokens_output=10, + tokens_cache_read=1000, + tokens_cache_write=0, + model="claude-sonnet-5", + ) + + record = normalize_opencode_metrics( + [str(db_path)], source="opencode-vanilla-synthetic", workspace_dir="/workspace" + ) + + # claude-sonnet-5 rates: input $3.00/M, output $15.00/M, cache_read $0.30/M. + expected_cost = (100 * 3.00 + 10 * 15.00 + 1000 * 0.30) / 1_000_000.0 + assert record["cost_usd"] == round(expected_cost, 6) + # Pricing 1100 tokens at the plain input rate would be ~10x this. Guard it. + assert record["cost_usd"] < 0.01 + + +def _write_event(path: Path, usage: dict) -> str: + event = { + "event": "llm:response", + "ts": "2026-01-01T00:00:00.000000+00:00", + "data": {"session_id": "synthetic-session", "usage": usage}, + } + path.write_text(json.dumps(event) + "\n", encoding="utf-8") + return str(path) + + +def test_events_branch_strips_cache_read_out_of_input(tmp_path: Path) -> None: + """The amplifier stacks fold cache_read into input; it must come back out. + + Synthetic analogue of the real event that established the convention: + input=12724 with cache_read=11850 and cache_write=504, i.e. 874 fresh. + Without the subtraction those 11,850 cached tokens count twice in total. + """ + path = _write_event( + tmp_path / "events.jsonl", + { + "input_tokens": 12724, + "output_tokens": 72, + "cache_read_tokens": 11850, + "cache_write_tokens": 504, + "cost_usd": "0.009147", + }, + ) + + record = normalize_metrics([path], source="amplifier-agent-synthetic") + + assert record["input_tokens"] == 12724 - 11850 + assert record["cache_read"] == 11850 + assert record["cache_write"] == 504 + assert record["total_tokens"] == 874 + 11850 + 504 + 72 + + +def test_events_branch_handles_cache_write_larger_than_input(tmp_path: Path) -> None: + """cache_write is NOT part of input, so a huge write must not go negative. + + This is the shape that disproved the inclusive-of-everything reading: + input=872 alongside cache_write=12354 on a real first-turn event. + """ + path = _write_event( + tmp_path / "events.jsonl", + { + "input_tokens": 872, + "output_tokens": 40, + "cache_read_tokens": 0, + "cache_write_tokens": 12354, + "cost_usd": "0.01", + }, + ) + + record = normalize_metrics([path], source="amplifier-agent-synthetic") + + assert record["input_tokens"] == 872 + assert record["total_tokens"] == 872 + 0 + 12354 + 40 + + +def test_events_branch_clamps_and_flags_a_broken_convention(tmp_path: Path) -> None: + """If input < cache_read the assumption broke: clamp, never emit a negative.""" + path = _write_event( + tmp_path / "events.jsonl", + { + "input_tokens": 10, + "output_tokens": 5, + "cache_read_tokens": 500, + "cache_write_tokens": 0, + "cost_usd": "0.001", + }, + ) + + record = normalize_metrics([path], source="amplifier-agent-synthetic") + + assert record["input_tokens"] == 0, "must clamp, not go negative" + assert "clamped to 0" in record["notes"], "a wrong figure must announce itself" diff --git a/.amplifier/evaluation/jobbench/tests/test_orphans.py b/.amplifier/evaluation/jobbench/tests/test_orphans.py new file mode 100644 index 00000000..b97c338e --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_orphans.py @@ -0,0 +1,229 @@ +"""Unit tests for jobbench.orphans: a DESTRUCTIVE prefix match, fully mocked. + +`sweep_orphans()` destroys every DTU instance whose id starts with `jb-`. The +blast radius is real containers, so the properties worth pinning are the ones +that bound it: + + - the prefix is the ONLY selector -- a non-`jb-` instance is never touched + - malformed CLI output (missing id, non-string id, non-dict entry) is + skipped rather than raising mid-sweep, which would leave the rest of the + leaked containers alive + - `list_instances()` degrades to [] on a failing or non-JSON CLI, because a + sweep that cannot enumerate must skip reaping, not abort the run + +Both the `amplifier-digital-twin` subprocess and `DTU.destroy` are mocked in +every test here -- nothing in this file invokes the real CLI or touches a real +container. No benchmark content appears in this file. +""" + +from __future__ import annotations + +import asyncio + +import pytest + +from jobbench import orphans +from jobbench.orphans import JB_PREFIX, list_instances, sweep_orphans + + +class FakeProc: + """Stand-in for the process returned by asyncio.create_subprocess_exec.""" + + def __init__(self, returncode: int = 0, stdout: bytes = b"[]", stderr: bytes = b""): + self.returncode = returncode + self._stdout = stdout + self._stderr = stderr + + async def communicate(self) -> tuple[bytes, bytes]: + return self._stdout, self._stderr + + +def _fake_cli( + monkeypatch, + *, + returncode: int = 0, + stdout: bytes = b"[]", + stderr: bytes = b"", +) -> list[tuple]: + """Replace the CLI subprocess; returns the list argv calls are recorded to.""" + calls: list[tuple] = [] + + async def _exec(*args, **kwargs): + calls.append(args) + return FakeProc(returncode=returncode, stdout=stdout, stderr=stderr) + + monkeypatch.setattr(asyncio, "create_subprocess_exec", _exec) + return calls + + +def _stub_instances(monkeypatch, instances: list[dict]) -> None: + """Bypass the CLI entirely and hand `sweep_orphans` a fixed instance list.""" + + async def _list() -> list[dict]: + return instances + + monkeypatch.setattr(orphans, "list_instances", _list) + + +def _capture_destroys(monkeypatch) -> list[str]: + """Replace DTU.destroy; returns the list destroyed ids are recorded to.""" + destroyed: list[str] = [] + + async def _destroy(self, *, timeout_s: float = 120.0) -> None: + destroyed.append(self.id) + + monkeypatch.setattr(orphans.DTU, "destroy", _destroy) + return destroyed + + +# --------------------------------------------------------------------------- +# list_instances: best-effort enumeration +# --------------------------------------------------------------------------- + + +def test_list_instances_parses_a_json_array(monkeypatch): + calls = _fake_cli(monkeypatch, stdout=b'[{"id": "jb-one"}, {"id": "other"}]') + + result = asyncio.run(list_instances()) + + assert result == [{"id": "jb-one"}, {"id": "other"}] + assert calls[0] == (orphans.CLI, "list") + + +def test_list_instances_returns_empty_when_cli_exits_nonzero(monkeypatch): + """A failing CLI must not abort the run -- only skip reaping this time.""" + _fake_cli(monkeypatch, returncode=1, stdout=b"", stderr=b"daemon unreachable") + + assert asyncio.run(list_instances()) == [] + + +def test_list_instances_returns_empty_on_non_json(monkeypatch): + _fake_cli(monkeypatch, stdout=b"Error: something went wrong\n") + + assert asyncio.run(list_instances()) == [] + + +def test_list_instances_returns_empty_on_empty_stdout(monkeypatch): + _fake_cli(monkeypatch, stdout=b"") + + assert asyncio.run(list_instances()) == [] + + +def test_list_instances_returns_empty_for_non_array_json(monkeypatch): + """A JSON object (e.g. an error payload) is valid JSON but not a list.""" + _fake_cli(monkeypatch, stdout=b'{"error": "not a list"}') + + assert asyncio.run(list_instances()) == [] + + +def test_list_instances_tolerates_undecodable_bytes(monkeypatch): + """Decoding uses errors='replace', so a mangled byte stream degrades to [] + via the JSON check rather than raising UnicodeDecodeError. + """ + _fake_cli(monkeypatch, stdout=b"\xff\xfe not json") + + assert asyncio.run(list_instances()) == [] + + +# --------------------------------------------------------------------------- +# sweep_orphans: the prefix is the only selector +# --------------------------------------------------------------------------- + + +def test_only_jb_prefixed_instances_are_destroyed(monkeypatch): + """The load-bearing safety property. Everything not named `jb-...` on this + host -- another team's container, a hand-launched debug DTU -- belongs to + somebody else, and the sweep must leave it running. + """ + _stub_instances( + monkeypatch, + [ + {"id": "jb-leaked-one"}, + {"id": "dtu-someone-elses"}, + {"id": "jb-leaked-two"}, + {"id": "prod-database"}, + {"id": "not-jb-prefixed"}, + ], + ) + destroyed = _capture_destroys(monkeypatch) + + returned = asyncio.run(sweep_orphans()) + + assert returned == ["jb-leaked-one", "jb-leaked-two"] + assert destroyed == ["jb-leaked-one", "jb-leaked-two"] + + +def test_prefix_match_is_anchored_at_the_start(monkeypatch): + """`jb-` appearing anywhere else in the id is not a match.""" + _stub_instances( + monkeypatch, + [ + {"id": "my-jb-container"}, + {"id": "xjb-thing"}, + {"id": "JB-UPPERCASE"}, + {"id": "jb-real"}, + ], + ) + destroyed = _capture_destroys(monkeypatch) + + assert asyncio.run(sweep_orphans()) == ["jb-real"] + assert destroyed == ["jb-real"] + + +def test_empty_instance_list_destroys_nothing(monkeypatch): + """The normal case: per-trial cleanup worked and there is nothing to reap.""" + _stub_instances(monkeypatch, []) + destroyed = _capture_destroys(monkeypatch) + + assert asyncio.run(sweep_orphans()) == [] + assert destroyed == [] + + +# --------------------------------------------------------------------------- +# sweep_orphans: malformed entries are skipped, never fatal +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "malformed", + [ + {}, # no id at all + {"id": None}, + {"id": ""}, + {"id": 12345}, # non-string + {"id": ["jb-list-not-a-string"]}, + {"name": "jb-wrong-key"}, + ], + ids=["no-id", "null-id", "empty-id", "int-id", "list-id", "wrong-key"], +) +def test_malformed_entries_are_skipped_without_raising(monkeypatch, malformed): + """A malformed entry must not abort the sweep partway through -- the + remaining leaked containers still need reaping. + """ + _stub_instances(monkeypatch, [malformed, {"id": "jb-good"}]) + destroyed = _capture_destroys(monkeypatch) + + assert asyncio.run(sweep_orphans()) == ["jb-good"] + assert destroyed == ["jb-good"] + + +def test_sweep_degrades_to_no_op_when_listing_fails(monkeypatch): + """End to end through the real `list_instances`: a failing CLI yields an + empty sweep, not an exception that would take down the run around it. + """ + _fake_cli(monkeypatch, returncode=1, stderr=b"daemon unreachable") + destroyed = _capture_destroys(monkeypatch) + + assert asyncio.run(sweep_orphans()) == [] + assert destroyed == [] + + +def test_sweep_uses_the_documented_prefix_constant(monkeypatch): + """Pinned against JB_PREFIX itself so the destructive selector and the + trial naming scheme (jobbench.trial._dtu_name) cannot drift apart silently. + """ + _stub_instances(monkeypatch, [{"id": f"{JB_PREFIX}synthetic-trial"}]) + destroyed = _capture_destroys(monkeypatch) + + assert asyncio.run(sweep_orphans()) == [f"{JB_PREFIX}synthetic-trial"] + assert destroyed == [f"{JB_PREFIX}synthetic-trial"] diff --git a/.amplifier/evaluation/jobbench/tests/test_prompt.py b/.amplifier/evaluation/jobbench/tests/test_prompt.py new file mode 100644 index 00000000..c9a2c0b6 --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_prompt.py @@ -0,0 +1,128 @@ +"""Golden-hash lock on jobbench.prompt.render() -- the agent-facing wording. + +prompt.py is reproduced verbatim from the JobBench reference runners, and its +own docstring says why that matters: scores produced under a different prompt +are not comparable to any other run of this benchmark, published or local. +A reworded prompt does not fail -- it produces plausible numbers that silently +mean something else. Nothing else in the harness would notice. + +So the wording is pinned by hash. The only thing that may legitimately vary +is the three interpolated paths, which differ from upstream because trials run +inside a container rather than a host /tmp directory. + +The prompt is the harness's own wrapper text, not JobBench task or rubric +content: it points the agent AT its task folder and deliberately does not +inline TASK_INSTRUCTIONS.txt. No benchmark content appears in this file. +""" + +from __future__ import annotations + +import hashlib + +from jobbench import prompt + +# sha256 of `prompt.render()` with its default (container) paths. +# +# If this test fails you have changed the prompt handed to every agent under +# test. That is not a formatting detail. Every score this harness has ever +# published was produced under the OLD wording, so the new wording silently +# invalidates cross-run comparability: old and new numbers can still be +# averaged, charted, and compared, and they will be meaningless together. +# Upstream's published JobBench numbers become incomparable too. +# +# Update this constant ONLY when you intend that, and say so in the commit +# message along with which runs are no longer comparable. If you are here +# because of a typo fix or a lint autofix, revert the source change instead. +GOLDEN_SHA256 = "cc5ff13cf7a08ed34ac47dc37b0ff4173490ca83393c85de2c4900df57c0190b" + +_WHY = ( + "The agent-facing prompt wording changed. Scores produced under a " + "different prompt are NOT comparable to any other run of this benchmark, " + "published or local -- and nothing else in the harness will notice, " + "because a reworded prompt still produces plausible-looking numbers. " + "If the change was intentional, update GOLDEN_SHA256 and record which " + "prior runs stop being comparable. Otherwise revert src/jobbench/prompt.py." +) + + +def _sha(text: str) -> str: + return hashlib.sha256(text.encode("utf-8")).hexdigest() + + +def test_rendered_prompt_matches_the_golden_hash(): + assert _sha(prompt.render()) == GOLDEN_SHA256, _WHY + + +def test_template_is_stable_across_calls(): + """render() is pure -- no timestamp, uuid, or env-dependent content leaks + into the bytes sent to the agent, or two trials of the same task would + not be running the same benchmark. + """ + assert prompt.render() == prompt.render() + + +# --------------------------------------------------------------------------- +# The three interpolated paths -- the only part that may legitimately vary +# --------------------------------------------------------------------------- + + +def test_default_paths_are_the_container_layout(): + assert prompt.WORKSPACE == "/workspace" + assert prompt.TASK_FOLDER == "/workspace/task_folder" + assert prompt.OUTPUT_DIR == "/workspace/output" + assert prompt.PROMPT_PATH == "/workspace/prompt.txt" + + +def test_task_folder_lands_in_both_places_it_is_named(): + """Once under the TASK FOLDER header, once in the IMPORTANT reminder.""" + text = prompt.render(task_folder="/synthetic/tasks") + assert text.count("/synthetic/tasks") == 2 + assert text.startswith("=== TASK FOLDER ===\n/synthetic/tasks\n") + assert "All reference files are in the task folder: /synthetic/tasks" in text + + +def test_output_dir_lands_in_both_places_it_is_named(): + """Once under the OUTPUT DIRECTORY header, once in the IMPORTANT reminder. + + Both must agree: an agent told two different output paths writes its + deliverables where the harness will not pull them, and the trial grades + as no_deliverables through no fault of the agent. + """ + text = prompt.render(output_dir="/synthetic/out") + assert text.count("/synthetic/out") == 2 + assert "=== OUTPUT DIRECTORY ===\n/synthetic/out\n" in text + assert "Only save the final deliverables to the output directory /synthetic/out" in text + + +def test_workspace_scopes_the_filesystem_restriction(): + """The workspace path appears exactly once, in the access restriction -- + the sentence that keeps the agent out of the rest of the container. + """ + text = prompt.render(workspace="/synthetic/ws") + assert text.count("/synthetic/ws") == 1 + assert "You MUST only access files within /synthetic/ws" in text + + +def test_all_three_paths_are_independently_substituted(): + text = prompt.render( + workspace="/ws-only", + task_folder="/task-only", + output_dir="/out-only", + ) + assert text.count("/ws-only") == 1 + assert text.count("/task-only") == 2 + assert text.count("/out-only") == 2 + # No placeholder survives unsubstituted. + assert "{" not in text + assert "}" not in text + + +def test_task_instructions_are_not_inlined(): + """Upstream points the agent AT the instructions file and expects it to + read it, so navigating its own workspace is part of what is measured. + Inlining the text would change what the benchmark tests -- and would put + real task content into the harness's own prompt. + """ + text = prompt.render() + assert "TASK_INSTRUCTIONS.txt" in text + assert "Read the TASK_INSTRUCTIONS.txt file in the task folder above" in text diff --git a/.amplifier/evaluation/jobbench/tests/test_scheduler.py b/.amplifier/evaluation/jobbench/tests/test_scheduler.py new file mode 100644 index 00000000..0366bbaf --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_scheduler.py @@ -0,0 +1,181 @@ +"""Unit tests for jobbench.scheduler: skip-existing, failure isolation, and +the max-parallel concurrency bound. + +`trial.run_trial` is monkeypatched to a fake in every test here -- these +never launch a real DTU. Task fixtures point at a nonexistent root; nothing +in these tests touches Task.instructions()/rubrics(), only the filesystem-free +selector/id properties. +""" + +from __future__ import annotations + +import asyncio +import json +from pathlib import Path + +from jobbench import scheduler +from jobbench.dataset import Task +from jobbench.matrix import Pair +from jobbench.trial import TrialResult + + +def _task(occupation: str, num: int) -> Task: + return Task(split="easy", occupation=occupation, task_num=num, root=Path("/nonexistent")) + + +def _fake_result(agent: str, task: Task, *, status: str = "completed") -> TrialResult: + return TrialResult( + agent=agent, + task_id=task.id, + task_selector=task.selector, + split=task.split, + model="test-model", + dtu_id="jb-fake-000000", + image_alias=f"jobbench-{agent}", + status=status, + exit_code=0, + agent_run_s=1.5, + started_at="2026-01-01T00:00:00+00:00", + finished_at="2026-01-01T00:00:01+00:00", + deliverable_count=1, + deliverable_bytes=10, + error=None, + cost_usd=3.14, + total_tokens=100, + llm_responses=1, + warnings=[], + ) + + +_COMMON_KWARGS = { + "model": "m", + "timeout_s": 10.0, + "agent_kwargs_for": lambda _agent: {}, + "grade": False, + "judge_model": "j", + "judge_api_base": None, + "judge_api_key": None, + "judge_max_workers": 1, + "judge_timeout_per_rubric": 10, +} + + +async def test_skip_existing_skips_completed_and_reruns_non_completed(tmp_path, monkeypatch): + run_root = tmp_path / "run" + pair_done = Pair(agent="agent-a", task=_task("occ", 1)) + pair_stale = Pair(agent="agent-a", task=_task("occ", 2)) + + done_dir = run_root / pair_done.agent / pair_done.task.id + done_dir.mkdir(parents=True) + (done_dir / "trial.json").write_text( + json.dumps({"status": "completed", "cost_usd": 1.23, "agent_run_s": 9.0}), + encoding="utf-8", + ) + + # Not completed (e.g. a previous crash) -- must be re-run, not skipped. + stale_dir = run_root / pair_stale.agent / pair_stale.task.id + stale_dir.mkdir(parents=True) + (stale_dir / "trial.json").write_text(json.dumps({"status": "crashed"}), encoding="utf-8") + + calls: list[str] = [] + + async def fake_run_trial(agent, task, trial_dir, **kwargs): + calls.append(task.id) + return _fake_result(agent, task) + + monkeypatch.setattr(scheduler.trial_mod, "run_trial", fake_run_trial) + + outcomes = await scheduler.run_matrix( + [pair_done, pair_stale], + run_root, + max_parallel=2, + skip_existing=True, + **_COMMON_KWARGS, + ) + + assert calls == [pair_stale.task.id] + + by_id = {o.pair.task.id: o for o in outcomes} + assert by_id[pair_done.task.id].skipped is True + assert by_id[pair_done.task.id].cost_usd == 1.23 + assert by_id[pair_stale.task.id].skipped is False + assert by_id[pair_stale.task.id].status == "completed" + + +async def test_skip_existing_off_reruns_everything(tmp_path, monkeypatch): + run_root = tmp_path / "run" + pair_done = Pair(agent="agent-a", task=_task("occ", 1)) + done_dir = run_root / pair_done.agent / pair_done.task.id + done_dir.mkdir(parents=True) + (done_dir / "trial.json").write_text(json.dumps({"status": "completed"}), encoding="utf-8") + + calls: list[str] = [] + + async def fake_run_trial(agent, task, trial_dir, **kwargs): + calls.append(task.id) + return _fake_result(agent, task) + + monkeypatch.setattr(scheduler.trial_mod, "run_trial", fake_run_trial) + + outcomes = await scheduler.run_matrix( + [pair_done], run_root, max_parallel=1, skip_existing=False, **_COMMON_KWARGS + ) + + assert calls == [pair_done.task.id] + assert outcomes[0].skipped is False + + +async def test_one_trial_raising_does_not_kill_the_batch(tmp_path, monkeypatch): + run_root = tmp_path / "run" + pair_bad = Pair(agent="agent-a", task=_task("occ", 1)) + pair_good = Pair(agent="agent-a", task=_task("occ", 2)) + + async def fake_run_trial(agent, task, trial_dir, **kwargs): + if task.id == pair_bad.task.id: + raise RuntimeError("boom") + return _fake_result(agent, task) + + monkeypatch.setattr(scheduler.trial_mod, "run_trial", fake_run_trial) + + outcomes = await scheduler.run_matrix( + [pair_bad, pair_good], + run_root, + max_parallel=2, + skip_existing=False, + **_COMMON_KWARGS, + ) + + by_id = {o.pair.task.id: o for o in outcomes} + assert by_id[pair_bad.task.id].status == "crashed" + assert "boom" in (by_id[pair_bad.task.id].error or "") + assert by_id[pair_good.task.id].status == "completed" + + +async def test_max_parallel_bounds_concurrency(tmp_path, monkeypatch): + run_root = tmp_path / "run" + pairs = [Pair(agent="agent-a", task=_task("occ", i)) for i in range(1, 5)] + + active = 0 + peak = 0 + lock = asyncio.Lock() + + async def fake_run_trial(agent, task, trial_dir, **kwargs): + nonlocal active, peak + async with lock: + active += 1 + peak = max(peak, active) + await asyncio.sleep(0.05) + async with lock: + active -= 1 + return _fake_result(agent, task) + + monkeypatch.setattr(scheduler.trial_mod, "run_trial", fake_run_trial) + + await scheduler.run_matrix( + pairs, run_root, max_parallel=2, skip_existing=False, **_COMMON_KWARGS + ) + + assert peak <= 2 + # 4 trials sleeping simultaneously under a cap of 2 should actually bind + # the cap, not just happen to stay under it by luck of scheduling order. + assert peak == 2 diff --git a/.amplifier/evaluation/jobbench/tests/test_tool_result_loss.py b/.amplifier/evaluation/jobbench/tests/test_tool_result_loss.py new file mode 100644 index 00000000..fbdcf37d --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_tool_result_loss.py @@ -0,0 +1,173 @@ +"""Pin harness-side detection of provider tool-result loss. + +Root cause lives outside this harness: when a tool_call has no paired +tool_result, amplifier-module-provider-anthropic injects a synthetic +"[SYSTEM ERROR: Tool result missing from conversation history]" message and +only logs a `logger.warning` (see +amplifier_module_provider_anthropic/__init__.py:2014). The model then +narrates that error back in its own words, burning wall-clock time on a +degenerate loop while exit code, deliverables, and score all stay normal. +That combination is what makes the condition dangerous: it publishes a +plausible number rather than failing. + +Fixing the provider is out of scope here; these tests pin only the harness's +detection of it, using synthetic agent.log fixtures. No real benchmark +content appears in this file. +""" + +from __future__ import annotations + +from pathlib import Path + +from jobbench.trial import TOOL_RESULT_LOSS_SIGNATURES, _detect_tool_result_loss + + +def test_clean_log_produces_no_warning(tmp_path: Path) -> None: + log_path = tmp_path / "agent.log" + log_path.write_text( + "$ some-agent-cli run\n" + "doing the task normally, no issues here\n" + "\n--- stderr ---\n" + "\n--- exit 0 (12.3s) ---\n\n", + encoding="utf-8", + ) + assert _detect_tool_result_loss(log_path) is None + + +def test_missing_log_produces_no_warning(tmp_path: Path) -> None: + """A log that was never created (e.g. crash before exec) is not an error.""" + assert _detect_tool_result_loss(tmp_path / "does-not-exist.log") is None + + +def test_single_signature_detected_with_count(tmp_path: Path) -> None: + log_path = tmp_path / "agent.log" + log_path.write_text( + "$ some-agent-cli run\n" + "I should check what I've already done here.\n" + "[SYSTEM ERROR: Tool result missing from conversation history]\n" + "Let me re-read the task instructions to be sure...\n" + "\n--- stderr ---\n" + "\n--- exit 0 (240.1s) ---\n\n", + encoding="utf-8", + ) + warning = _detect_tool_result_loss(log_path) + assert warning is not None + assert warning["kind"] == "tool_result_loss" + assert warning["count"] == 1 + assert "SYSTEM ERROR" in warning["detail"] + + +def test_repeated_occurrences_counted(tmp_path: Path) -> None: + """Multiple occurrences across a long-running loop are all counted, not + just detected as a boolean.""" + log_path = tmp_path / "agent.log" + body = "[SYSTEM ERROR: Tool result missing from conversation history]\n" * 3 + log_path.write_text( + f"$ agent run\n{body}\n--- stderr ---\n\n--- exit 0 (300s) ---\n\n", encoding="utf-8" + ) + warning = _detect_tool_result_loss(log_path) + assert warning is not None + assert warning["count"] == 3 + + +def test_second_signature_also_detected(tmp_path: Path) -> None: + """The interrupted-tool-execution variant is recognized independently + of the SYSTEM ERROR wording.""" + log_path = tmp_path / "agent.log" + log_path.write_text( + "$ agent run\n" + "Tool execution was interrupted and no result was captured\n" + "\n--- stderr ---\n\n--- exit 0 (5s) ---\n\n", + encoding="utf-8", + ) + warning = _detect_tool_result_loss(log_path) + assert warning is not None + assert warning["count"] == 1 + + +def test_both_signatures_sum_into_one_count(tmp_path: Path) -> None: + log_path = tmp_path / "agent.log" + log_path.write_text( + "$ agent run\n" + f"{TOOL_RESULT_LOSS_SIGNATURES[0]}\n" + f"{TOOL_RESULT_LOSS_SIGNATURES[1]}\n" + "\n--- stderr ---\n\n--- exit 0 (10s) ---\n\n", + encoding="utf-8", + ) + warning = _detect_tool_result_loss(log_path) + assert warning is not None + assert warning["count"] == 2 + + +# --------------------------------------------------------------------------- +# Narration heuristic +# +# The literal signatures above are frequently NOT observable: the provider +# injects that text into the message history sent to the model and only logs a +# local warning. For a CLI whose stdout carries just assistant prose, the +# string never reaches agent.log -- measured 0 hits on a real run that looped +# 27 times. These tests pin the indirect fallback that catches that case. +# --------------------------------------------------------------------------- + + +def _log(prose: str, stderr: str = "") -> str: + return f"$ agent run\n{prose}\n\n--- stderr ---\n{stderr}\n--- exit 0 (1.0s) ---\n" + + +def test_narration_below_threshold_is_not_flagged(tmp_path: Path) -> None: + """One re-read is ordinary agent behavior, not evidence of a fault.""" + log_path = tmp_path / "agent.log" + log_path.write_text(_log("I need to actually read the config file first."), encoding="utf-8") + + assert _detect_tool_result_loss(log_path) is None + + +def test_repeated_narration_is_flagged_as_heuristic(tmp_path: Path) -> None: + """Repetition is the tell: the model keeps re-deciding it never read a file.""" + log_path = tmp_path / "agent.log" + prose = ( + "I still need to actually read the instructions file.\n" + "Good, that confirms my understanding.\n" + "I realize I haven't actually read the instructions yet.\n" + "I notice I have been operating on inferred context.\n" + "Let me correct that without actually opening it earlier." + ) + log_path.write_text(_log(prose), encoding="utf-8") + + result = _detect_tool_result_loss(log_path) + assert result is not None + assert result["kind"] == "tool_result_loss" + assert result["confidence"] == "heuristic" + assert result["count"] >= 3 + + +def test_direct_signature_outranks_narration(tmp_path: Path) -> None: + """A literal provider signature is proof, so it must not be downgraded.""" + log_path = tmp_path / "agent.log" + prose = "\n".join( + [ + TOOL_RESULT_LOSS_SIGNATURES[0], + "I need to actually read the file.", + "I haven't actually read it yet.", + "I realize I have not opened it.", + ] + ) + log_path.write_text(_log(prose), encoding="utf-8") + + result = _detect_tool_result_loss(log_path) + assert result is not None + assert result["confidence"] == "direct" + + +def test_stderr_half_is_not_scanned_for_narration(tmp_path: Path) -> None: + """The CLI's own TUI echo would otherwise inflate the count without evidence. + + A terminal UI re-renders its todo list on every update, so a checked item + like "Read the instructions directly" can appear many times without the + model having re-decided anything. + """ + log_path = tmp_path / "agent.log" + echoed = "\n".join(["[x] Need to actually read the instructions"] * 10) + log_path.write_text(_log("did the task cleanly", stderr=echoed), encoding="utf-8") + + assert _detect_tool_result_loss(log_path) is None diff --git a/.amplifier/evaluation/jobbench/tests/test_trial_status.py b/.amplifier/evaluation/jobbench/tests/test_trial_status.py new file mode 100644 index 00000000..7d97197c --- /dev/null +++ b/.amplifier/evaluation/jobbench/tests/test_trial_status.py @@ -0,0 +1,504 @@ +"""Unit tests for jobbench.trial: the status decision tree and deliverable flattening. + +trial.py's docstring promises trial.json "honestly distinguishes a crash, a +timeout, and a legitimate zero-deliverable run from each other and from +success". That promise is the whole basis for reading a results table: a +crashed trial and a trial that legitimately produced nothing score the same +zero, and only `status` tells them apart. + +Every DTU interaction is faked here -- no container is launched, no CLI is +invoked, no agent runs. Deliverable files are synthetic scratch bytes; no real +JobBench task text, rubric text, or agent output appears in this file. + +See `test_status_cli_enforced_overrun_is_classified_crashed` for a discrepancy +between this behaviour and the documented intent; it is pinned as-is, not +fixed here. +""" + +from __future__ import annotations + +import asyncio +import json +from pathlib import Path + +import pytest + +from jobbench import trial as trial_mod +from jobbench.dataset import Task +from jobbench.dtu import CommandResult, DTUError +from jobbench.metrics import NOT_AVAILABLE +from jobbench.trial import _flatten_pulled_deliverables, run_trial + + +def _task(tmp_path: Path) -> Task: + """A synthetic task whose task_folder exists so the seeding step has + something to push. Contents are scratch bytes, not benchmark text. + """ + root = tmp_path / "task" + folder = root / "task_folder" + folder.mkdir(parents=True) + (folder / "TASK_INSTRUCTIONS.txt").write_text("synthetic instructions\n", encoding="utf-8") + return Task(split="easy", occupation="synthetic", task_num=1, root=root) + + +class FakeAdapter: + """Minimal stand-in for an agents.Adapter.""" + + name = "synthetic-agent" + image_alias = "jobbench-synthetic-agent" + session_dirs: tuple[str, ...] = () + metrics_source = "events" + + async def configure(self, dtu, *, model: str) -> None: + return None + + def command(self) -> list[str]: + return ["synthetic-agent", "run"] + + +class FakeDTU: + """Stand-in for a launched DTU. + + `exec_result` is either a CommandResult to return or an exception to + raise; `pull_writes` is a callable given the local destination for the + deliverables pull, so a test can decide what "landed". + """ + + def __init__(self, *, exec_result, pull_writes=None, pull_raises=None): + self.id = "jb-synthetic-0001" + self.destroyed = False + self._exec_result = exec_result + self._pull_writes = pull_writes + self._pull_raises = pull_raises + self.pushes: list[tuple[str, str]] = [] + + async def file_push(self, src, destination) -> None: + self.pushes.append((str(src), destination)) + + async def file_pull(self, remote, local) -> None: + if self._pull_raises is not None: + raise self._pull_raises + if self._pull_writes is not None: + self._pull_writes(Path(local)) + + async def exec_cmd(self, command, *, timeout_s=None, stream_to_logfile=None): + if stream_to_logfile is not None: + stream_to_logfile.parent.mkdir(parents=True, exist_ok=True) + stream_to_logfile.write_text( + f"$ {' '.join(command)}\nsynthetic agent output\n" + "\n--- stderr ---\n\n--- exit 0 (1.0s) ---\n\n", + encoding="utf-8", + ) + if isinstance(self._exec_result, BaseException): + raise self._exec_result + return self._exec_result + + async def destroy(self) -> None: + self.destroyed = True + + +def _install(monkeypatch, dtu: FakeDTU) -> None: + """Wire trial.py's collaborators to fakes: adapter registry, image alias, + launch-profile rendering, and DTU.launch. + """ + monkeypatch.setattr(trial_mod.agents, "get", lambda name, **kw: FakeAdapter()) + monkeypatch.setattr(trial_mod.images, "agent_alias", lambda name: "jobbench-synthetic-agent") + monkeypatch.setattr( + trial_mod, "_render_launch_profile", lambda alias, dest: (dest.touch(), dest)[1] + ) + + async def _launch(profile_path, *, name=None, **kwargs): + dtu.id = name or dtu.id + return dtu + + monkeypatch.setattr(trial_mod.DTU, "launch", _launch) + + +def _run(tmp_path: Path, dtu: FakeDTU, **kwargs): + trial_dir = tmp_path / "trial" + return asyncio.run( + run_trial( + "synthetic-agent", + _task(tmp_path), + trial_dir, + model="synthetic-model", + **kwargs, + ) + ) + + +def _writes_one_file(local: Path) -> None: + """Simulate `file_pull` landing /workspace/output/ under deliverables/, + which the CLI nests one level deep (see _flatten_pulled_deliverables). + """ + nested = local / "output" + nested.mkdir(parents=True, exist_ok=True) + (nested / "result.txt").write_text("synthetic deliverable\n", encoding="utf-8") + + +def _writes_nothing(local: Path) -> None: + (local / "output").mkdir(parents=True, exist_ok=True) + + +def _ok(returncode: int = 0) -> CommandResult: + return CommandResult(returncode=returncode, stdout="", stderr="", elapsed_s=1.0) + + +# --------------------------------------------------------------------------- +# status decision tree +# --------------------------------------------------------------------------- + + +def test_status_completed_on_exit_zero_with_deliverables(tmp_path: Path, monkeypatch): + dtu = FakeDTU(exec_result=_ok(0), pull_writes=_writes_one_file) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.status == "completed" + assert result.exit_code == 0 + assert result.deliverable_count == 1 + assert result.error is None + + +def test_status_crashed_on_nonzero_exit(tmp_path: Path, monkeypatch): + """A non-zero agent exit is a crash even if it left files behind -- + deliverables do not launder a failed run into a success. + """ + dtu = FakeDTU(exec_result=_ok(1), pull_writes=_writes_one_file) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.status == "crashed" + assert result.exit_code == 1 + assert result.deliverable_count == 1 + + +def test_status_no_deliverables_on_exit_zero_with_empty_output(tmp_path: Path, monkeypatch): + """A deliverable-free "success" is not success. It must be distinguishable + from `crashed` (the agent ran fine) and from `completed` (it produced + nothing to grade). + """ + dtu = FakeDTU(exec_result=_ok(0), pull_writes=_writes_nothing) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.status == "no_deliverables" + assert result.exit_code == 0 + assert result.deliverable_count == 0 + # The agent itself succeeded; that must not be rewritten as a crash. + assert result.error is None + + +def test_status_timeout_when_the_exec_layer_raises_dtuerror(tmp_path: Path, monkeypatch): + """The only path that produces `timeout`: `dtu.exec_cmd` raising DTUError, + which happens when the harness's own asyncio wait expires. + """ + dtu = FakeDTU( + exec_result=DTUError("DTU command timed out after 3720.0s: amplifier-digital-twin exec"), + pull_writes=_writes_nothing, + ) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu, timeout_s=10.0) + + assert result.status == "timeout" + assert result.exit_code is None + assert "timed out" in (result.error or "") + + +def test_timeout_survives_a_later_no_deliverables_check(tmp_path: Path, monkeypatch): + """A timed-out trial produced nothing, but must stay `timeout` -- the + `no_deliverables` downgrade applies only to a run that exited 0. + """ + dtu = FakeDTU(exec_result=DTUError("timed out"), pull_writes=_writes_nothing) + _install(monkeypatch, dtu) + + assert _run(tmp_path, dtu, timeout_s=10.0).status == "timeout" + + +def test_timeout_still_pulls_partial_deliverables(tmp_path: Path, monkeypatch): + """Partial output after a timeout is still signal and must be captured.""" + dtu = FakeDTU(exec_result=DTUError("timed out"), pull_writes=_writes_one_file) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu, timeout_s=10.0) + + assert result.status == "timeout" + assert result.deliverable_count == 1 + + +def test_status_crashed_when_setup_raises_before_exec(tmp_path: Path, monkeypatch): + """A failure in seeding (or any earlier stage) is a crash, recorded with + the reason rather than swallowed. + """ + dtu = FakeDTU(exec_result=_ok(0)) + _install(monkeypatch, dtu) + + async def _boom(src, destination): + raise DTUError("file-push failed: synthetic") + + dtu.file_push = _boom # type: ignore[method-assign] + + result = _run(tmp_path, dtu) + + assert result.status == "crashed" + assert result.exit_code is None + assert "file-push failed" in (result.error or "") + + +def test_timeout_is_not_overwritten_by_a_later_crash(tmp_path: Path, monkeypatch): + """The outer handler explicitly preserves `timeout`. If a post-timeout + stage then blows up, the trial must still read as a timeout -- that is + what actually happened to the agent. + """ + dtu = FakeDTU(exec_result=DTUError("timed out")) + _install(monkeypatch, dtu) + + async def _boom(remote, local): + raise RuntimeError("synthetic post-timeout failure") + + dtu.file_pull = _boom # type: ignore[method-assign] + + result = _run(tmp_path, dtu, timeout_s=10.0) + + assert result.status == "timeout" + + +def test_deliverable_pull_failure_does_not_mask_the_agent_result(tmp_path: Path, monkeypatch): + """A DTUError from the deliverables pull is recorded in `error` but must + not rewrite a successful agent run into a crash -- it is a harness-side + retrieval problem, and the count already says nothing landed. + """ + dtu = FakeDTU(exec_result=_ok(0), pull_raises=DTUError("file-pull failed: synthetic")) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.status == "no_deliverables" + assert "file-pull failed" in (result.error or "") + + +def test_status_cli_enforced_overrun_is_classified_crashed(tmp_path: Path, monkeypatch): + """DISCREPANCY, pinned as observed rather than as intended. + + trial.py asks for `exec_cmd(timeout_s=timeout_s + 60)`; dtu.exec_cmd + passes that same value to the CLI as `--timeout` and sets its own asyncio + wait to `timeout_s + 60 + 60`. The CLI therefore enforces its limit 60s + BEFORE the asyncio wait can fire, and it reports that enforcement as a + non-zero exit (JSON mode catches subprocess.TimeoutExpired and exits 1), + not as a DTUError. Only the DTUError path yields `status = "timeout"`. + + Net effect: an agent that merely overruns its wall-clock budget is + recorded as `crashed`, and `timeout` is reachable only when the CLI + process itself hangs past its own limit. The reason is not recorded in + `error` either -- that field is only written on the DTUError path -- so + the sole surviving evidence is the CLI's stderr in agent.log. + + This test pins the current behaviour so the discrepancy is visible; it is + not an endorsement of it. + """ + timed_out = CommandResult( + returncode=1, + stdout="", + stderr="Error: Command '['incus', 'exec', ...]' timed out after 70 seconds", + elapsed_s=70.0, + ) + dtu = FakeDTU(exec_result=timed_out, pull_writes=_writes_nothing) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu, timeout_s=10.0) + + assert result.status == "crashed" + assert result.status != "timeout" + # The overrun is not recorded as the reason anywhere in trial.json. + assert result.error is None + + +# --------------------------------------------------------------------------- +# trial.json / bookkeeping around the status +# --------------------------------------------------------------------------- + + +def test_trial_json_is_written_for_every_status(tmp_path: Path, monkeypatch): + dtu = FakeDTU(exec_result=DTUError("timed out"), pull_writes=_writes_nothing) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu, timeout_s=10.0) + + data = json.loads((tmp_path / "trial" / "trial.json").read_text(encoding="utf-8")) + assert data["status"] == result.status == "timeout" + assert data["dtu_id"] == dtu.id + assert data["warnings"] == [] + + +def test_dtu_is_destroyed_even_when_the_trial_crashes(tmp_path: Path, monkeypatch): + dtu = FakeDTU(exec_result=_ok(0)) + _install(monkeypatch, dtu) + + async def _boom(src, destination): + raise DTUError("synthetic") + + dtu.file_push = _boom # type: ignore[method-assign] + + _run(tmp_path, dtu) + + assert dtu.destroyed is True + + +def test_metrics_absent_is_not_available_never_zero(tmp_path: Path, monkeypatch): + """No session telemetry must read as not_available, not a fabricated 0 -- + a zero cost and an uncollected cost are different facts. + """ + dtu = FakeDTU(exec_result=_ok(0), pull_writes=_writes_one_file) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.cost_usd == NOT_AVAILABLE + assert result.total_tokens == NOT_AVAILABLE + assert result.llm_responses == NOT_AVAILABLE + + +def test_status_and_warnings_stay_separate(tmp_path: Path, monkeypatch): + """A quality warning is not a status. A run that trips the tool-result-loss + detector still completed; the warning rides alongside. + """ + dtu = FakeDTU(exec_result=_ok(0), pull_writes=_writes_one_file) + _install(monkeypatch, dtu) + + async def _exec(command, *, timeout_s=None, stream_to_logfile=None): + stream_to_logfile.parent.mkdir(parents=True, exist_ok=True) + stream_to_logfile.write_text( + f"$ {' '.join(command)}\n" + f"{trial_mod.TOOL_RESULT_LOSS_SIGNATURES[0]}\n" + "\n--- stderr ---\n\n--- exit 0 (1.0s) ---\n\n", + encoding="utf-8", + ) + return _ok(0) + + dtu.exec_cmd = _exec # type: ignore[method-assign] + + result = _run(tmp_path, dtu) + + assert result.status == "completed" + assert [w["kind"] for w in result.warnings] == ["tool_result_loss"] + + +# --------------------------------------------------------------------------- +# _flatten_pulled_deliverables -- decides deliverable_count, hence the status +# --------------------------------------------------------------------------- + + +def test_flatten_moves_nested_output_contents_up(tmp_path: Path): + """`file_pull` of /workspace/output/ lands files at deliverables/output/ + (cp -r basename convention). Downstream expects deliverables/. + """ + deliverables = tmp_path / "deliverables" + nested = deliverables / "output" + nested.mkdir(parents=True) + (nested / "a.txt").write_text("a", encoding="utf-8") + (nested / "b.txt").write_text("b", encoding="utf-8") + + _flatten_pulled_deliverables(deliverables) + + assert not nested.exists() + assert sorted(p.name for p in deliverables.iterdir()) == ["a.txt", "b.txt"] + + +def test_flatten_preserves_subdirectories(tmp_path: Path): + """A deliverable that is itself a directory moves up intact, so its files + still count toward deliverable_count. + """ + deliverables = tmp_path / "deliverables" + sub = deliverables / "output" / "report" + sub.mkdir(parents=True) + (sub / "page1.txt").write_text("p1", encoding="utf-8") + + _flatten_pulled_deliverables(deliverables) + + assert (deliverables / "report" / "page1.txt").read_text(encoding="utf-8") == "p1" + assert not (deliverables / "output").exists() + + +def test_flatten_is_a_noop_without_a_nested_output_dir(tmp_path: Path): + """Some pulls land flat already. Flattening must not disturb them.""" + deliverables = tmp_path / "deliverables" + deliverables.mkdir() + (deliverables / "a.txt").write_text("a", encoding="utf-8") + + _flatten_pulled_deliverables(deliverables) + + assert [p.name for p in deliverables.iterdir()] == ["a.txt"] + + +def test_flatten_handles_an_empty_nested_output_dir(tmp_path: Path): + """The no_deliverables case: the directory exists but is empty. It must be + removed cleanly so deliverable_count is 0 rather than 0-plus-a-stray-dir. + """ + deliverables = tmp_path / "deliverables" + (deliverables / "output").mkdir(parents=True) + + _flatten_pulled_deliverables(deliverables) + + assert not (deliverables / "output").exists() + assert list(deliverables.iterdir()) == [] + + +def test_flatten_missing_deliverables_dir_is_a_noop(tmp_path: Path): + """A trial that never pulled anything must not raise here.""" + _flatten_pulled_deliverables(tmp_path / "nonexistent") + + +def test_flatten_overwrites_a_colliding_file(tmp_path: Path): + """The nested copy is the freshly-pulled one and wins.""" + deliverables = tmp_path / "deliverables" + nested = deliverables / "output" + nested.mkdir(parents=True) + (deliverables / "a.txt").write_text("stale", encoding="utf-8") + (nested / "a.txt").write_text("fresh", encoding="utf-8") + + _flatten_pulled_deliverables(deliverables) + + assert (deliverables / "a.txt").read_text(encoding="utf-8") == "fresh" + + +def test_flatten_overwrites_a_colliding_directory(tmp_path: Path): + deliverables = tmp_path / "deliverables" + stale = deliverables / "report" + stale.mkdir(parents=True) + (stale / "old.txt").write_text("stale", encoding="utf-8") + fresh = deliverables / "output" / "report" + fresh.mkdir(parents=True) + (fresh / "new.txt").write_text("fresh", encoding="utf-8") + + _flatten_pulled_deliverables(deliverables) + + assert (deliverables / "report" / "new.txt").exists() + assert not (deliverables / "report" / "old.txt").exists() + + +@pytest.mark.parametrize( + ("writer", "expected_status", "expected_count"), + [ + (_writes_one_file, "completed", 1), + (_writes_nothing, "no_deliverables", 0), + ], + ids=["one-file", "empty"], +) +def test_flattening_drives_the_no_deliverables_classification( + tmp_path: Path, monkeypatch, writer, expected_status, expected_count +): + """End to end: whether flattening finds files is exactly what decides + `completed` vs `no_deliverables` for an exit-0 run. + """ + dtu = FakeDTU(exec_result=_ok(0), pull_writes=writer) + _install(monkeypatch, dtu) + + result = _run(tmp_path, dtu) + + assert result.status == expected_status + assert result.deliverable_count == expected_count + assert not (tmp_path / "trial" / "deliverables" / "output").exists() diff --git a/.amplifier/evaluation/jobbench/uv.lock b/.amplifier/evaluation/jobbench/uv.lock new file mode 100644 index 00000000..cc642e20 --- /dev/null +++ b/.amplifier/evaluation/jobbench/uv.lock @@ -0,0 +1,1534 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" +resolution-markers = [ + "python_full_version >= '3.14' and sys_platform == 'win32'", + "python_full_version >= '3.14' and sys_platform == 'emscripten'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'", + "python_full_version < '3.12' and sys_platform == 'win32'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'", + "python_full_version < '3.12' and sys_platform == 'emscripten'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version < '3.12' and sys_platform != 'emscripten' and sys_platform != 'win32'", +] + +[[package]] +name = "annotated-types" +version = "0.8.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5f/56/a8120250d128bed162cd73c76d45f6ef9991f3e068f62a8ee060afa3104a/annotated_types-0.8.0.tar.gz", hash = "sha256:13b2beaad985e05e2d6407ee4c4f35590b11f8d693a258a561055cac8f64cab7", size = 15893, upload-time = "2026-07-23T20:16:13.995Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/99/91/8acff4f5e50511b911bbccb72b8628a49c68ce14148cd9f6431094859a90/annotated_types-0.8.0-py3-none-any.whl", hash = "sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0", size = 13427, upload-time = "2026-07-23T20:16:12.938Z" }, +] + +[[package]] +name = "anyio" +version = "4.14.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "idna" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/61/cc/a381afa6efea9f496eff839d4a6a1aed3bfafc7b3ab4b0d1b243a12573dd/anyio-4.14.2.tar.gz", hash = "sha256:cfa139f3ed1a23ee8f88a145ddb5ac7605b8bbfd8592baacd7ce3d8bb4313c7f", size = 260176, upload-time = "2026-07-12T20:29:07.082Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/da/35/f2287558c17e29fafc8ef3daf819bb9834061cfa43bff8014f7df7f63bdc/anyio-4.14.2-py3-none-any.whl", hash = "sha256:9f505dda5ac9f0c8309b5e8bd445a8c2bf7246f3ce950121e45ea15bc41d1494", size = 125813, upload-time = "2026-07-12T20:29:05.763Z" }, +] + +[[package]] +name = "certifi" +version = "2026.7.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz", hash = "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55", size = 138112, upload-time = "2026-07-22T03:35:12.644Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" }, +] + +[[package]] +name = "cffi" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pycparser", marker = "implementation_name != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/70/d2/16d99a0c4948febc0ebd133a13b2f688ff7f8cb04da971e1128872ce0c03/cffi-2.1.1-cp311-cp311-macosx_10_15_x86_64.whl", hash = "sha256:c8d2c9fd1f2d16f780d15127abb050d13d1a76c03a4bd87d7e4980e45e511e12", size = 183838, upload-time = "2026-08-03T21:19:29.637Z" }, + { url = "https://files.pythonhosted.org/packages/cd/95/31b535a9f0220ae9f357de4a08d57ce89cb417653c2fd9f075f50822a388/cffi-2.1.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:398aff33cee2767e3e781d2554c54bd0dff386bb437581e0d8011fde1a942ec1", size = 184168, upload-time = "2026-08-03T21:19:30.764Z" }, + { url = "https://files.pythonhosted.org/packages/ad/5a/4707a0dc1f203f5dde5a907b0d4e3c25d71120241048bd5bc6f1bb9d4e71/cffi-2.1.1-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:154852545011f779917b11c78db2358d095da62a9a172b78ad0a583ee5adc0d0", size = 211805, upload-time = "2026-08-03T21:19:31.867Z" }, + { url = "https://files.pythonhosted.org/packages/ad/66/c19feabb28485b6e0bbaaafa90837a1ef5d302e90f2178bd33f17a49879b/cffi-2.1.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:3311ed60d36f83378794e1009ac6258bafbf81f7888b4caa7b35a521e3f95813", size = 218716, upload-time = "2026-08-03T21:19:32.896Z" }, + { url = "https://files.pythonhosted.org/packages/a7/92/500760486c8baab49a7a8a58ba7fc3355ec3974b454b8a09e528efde9e1d/cffi-2.1.1-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6e192623c49c94421616a5778fba35cf0d5a8d000650c1967ef4448ee5cdd990", size = 205569, upload-time = "2026-08-03T21:19:34.142Z" }, + { url = "https://files.pythonhosted.org/packages/a5/a7/a67c733254d6e7373f7822f8082d8d6beade791e0cf12a7611f376fa61c7/cffi-2.1.1-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a6e721d4b0e45d5b65e87534470e67b18dcd092c83f68fba09f152b9cbc061af", size = 204907, upload-time = "2026-08-03T21:19:35.174Z" }, + { url = "https://files.pythonhosted.org/packages/f7/a4/4399daaf8f7dfee9d7c3327fdb0426ee041cc63edc358b93911ceb2bfc7a/cffi-2.1.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:34e261f78cb6ceaaa36f42f2613f4380d94d9c759a9c73c769ee6e0247364632", size = 217807, upload-time = "2026-08-03T21:19:36.286Z" }, + { url = "https://files.pythonhosted.org/packages/28/f7/dabe6da2466ecbd82dc62e7342dc6b1065dad990c06f00f0ede9ebf2a0ed/cffi-2.1.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:7225e4514edb64eb6740324353e0da0711954fd8d7da4576755b1c6e09b697cd", size = 221252, upload-time = "2026-08-03T21:19:37.416Z" }, + { url = "https://files.pythonhosted.org/packages/ce/87/616202d8e51342c07d2534c510111c4cc37201775ce8f60802c9335d1edd/cffi-2.1.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:df913725b79db7bcf03448f36b7bf8815363417d5b58deecf9305e3e30f0f21a", size = 214214, upload-time = "2026-08-03T21:19:38.507Z" }, + { url = "https://files.pythonhosted.org/packages/b4/c6/ab025d75d2c26c19b087c0124e75ee31cb65032f4fe345d356d8c507ab97/cffi-2.1.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f5cfbc5fe74540d335175b656c725d74d90e3730c626d92575eea35029d9afaa", size = 219408, upload-time = "2026-08-03T21:19:39.809Z" }, + { url = "https://files.pythonhosted.org/packages/db/e2/7e8109f65445bdc673a7b54f02c677de462db75674220fd1335efc8eb598/cffi-2.1.1-cp311-cp311-win32.whl", hash = "sha256:f8ec5e643a9a937f64e1999eb9f75d072263751912dc5cd06d3c85f8f44be7c3", size = 174470, upload-time = "2026-08-03T21:19:41.246Z" }, + { url = "https://files.pythonhosted.org/packages/73/c0/77ba02423c2f7d7091143c45cd49e0e6575c4c1967394bb542bd923a9b74/cffi-2.1.1-cp311-cp311-win_amd64.whl", hash = "sha256:42f6930c31dc7f50732c9ae793c2786c7b6b044195967bbdde40bb9be81c4cc0", size = 185096, upload-time = "2026-08-03T21:19:42.615Z" }, + { url = "https://files.pythonhosted.org/packages/7c/47/9f1f85f9672ceda4984dc6c4f8824e8558992a2972c3d3c81fb8eb28d4ba/cffi-2.1.1-cp311-cp311-win_arm64.whl", hash = "sha256:c7659f22557c5a0bc4855cd635f55edec690cc008a40768527762cb9fb263455", size = 179941, upload-time = "2026-08-03T21:19:43.747Z" }, + { url = "https://files.pythonhosted.org/packages/10/69/43965eccfdead3b9220015fd1320e117be8c6ed01a62ffab76eeb752f5d5/cffi-2.1.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0", size = 184821, upload-time = "2026-08-03T21:19:44.887Z" }, + { url = "https://files.pythonhosted.org/packages/54/7d/16e5a096677b5e313ca80cd5e5170efa3ea44624a82bb111925522da64b1/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf", size = 184719, upload-time = "2026-08-03T21:19:46.129Z" }, + { url = "https://files.pythonhosted.org/packages/56/e6/8941622732edec876dd17d0453dce07317ae96db34f2ec1436c9d3785986/cffi-2.1.1-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a", size = 214799, upload-time = "2026-08-03T21:19:47.218Z" }, + { url = "https://files.pythonhosted.org/packages/44/de/f98430906df1545ffde0d543dd124a7a439bc2cd32b36b9c53f805df7333/cffi-2.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890", size = 222389, upload-time = "2026-08-03T21:19:48.331Z" }, + { url = "https://files.pythonhosted.org/packages/6a/5b/717f1526b9957b34456313c31645c5b82b8fb5c3fe9e4752999be7128bfc/cffi-2.1.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50", size = 210249, upload-time = "2026-08-03T21:19:49.543Z" }, + { url = "https://files.pythonhosted.org/packages/64/b3/f8aa4f3e34986c7e4ec45072d1b1b9dd295b6b18007b45518d79726dd725/cffi-2.1.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e", size = 208775, upload-time = "2026-08-03T21:19:50.918Z" }, + { url = "https://files.pythonhosted.org/packages/b1/db/dceb9dd5b231e1da801793f8acc9f3c52a7e1afe40bb1aae37e02b0faad5/cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf", size = 221822, upload-time = "2026-08-03T21:19:52.054Z" }, + { url = "https://files.pythonhosted.org/packages/a0/d2/6cd24ae3be000a634109c247d1475d62e5616d0dc78c82770942ec384248/cffi-2.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517", size = 225232, upload-time = "2026-08-03T21:19:53.109Z" }, + { url = "https://files.pythonhosted.org/packages/cb/52/3fa190537004dd7f0ab860a6dc7c0175b8667f68d1e618a46f5498d30250/cffi-2.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735", size = 223597, upload-time = "2026-08-03T21:19:54.515Z" }, + { url = "https://files.pythonhosted.org/packages/80/fb/0bb75b7039588c074b37ae99f40d9bfddf990ecb2fbc346ebccd2e56b9be/cffi-2.1.1-cp312-cp312-win32.whl", hash = "sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e", size = 175292, upload-time = "2026-08-03T21:19:55.566Z" }, + { url = "https://files.pythonhosted.org/packages/d9/79/615cc094e2fb508cade7de88d3b4f6c4ec2bab695c97bce9153dc65aadf5/cffi-2.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a", size = 185919, upload-time = "2026-08-03T21:19:56.89Z" }, + { url = "https://files.pythonhosted.org/packages/70/c6/d0ea84713fe46b243a436a18fcd47d639732747e21635c8a27191b06dc30/cffi-2.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80", size = 180093, upload-time = "2026-08-03T21:19:58.155Z" }, + { url = "https://files.pythonhosted.org/packages/9d/f4/035513d4117049066b4779dc3b7c0c0fdad175fa13731c9f4003f1cd1478/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e", size = 194248, upload-time = "2026-08-03T21:19:59.399Z" }, + { url = "https://files.pythonhosted.org/packages/76/af/2aeb4dbb5fc41a04161ae9ff1518de7cec08e164f44a8ce6a4cf7fd2cd1d/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c", size = 196908, upload-time = "2026-08-03T21:20:00.746Z" }, + { url = "https://files.pythonhosted.org/packages/a7/46/2e5fdde8555706dd98139a910ca11be02809f3f605ce956f655d0214e100/cffi-2.1.1-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6", size = 184805, upload-time = "2026-08-03T21:20:02.02Z" }, + { url = "https://files.pythonhosted.org/packages/55/41/4c7042f317b9217502988f0873af87e16ad606dc20f84e546e3e6ce9764c/cffi-2.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971", size = 184764, upload-time = "2026-08-03T21:20:03.141Z" }, + { url = "https://files.pythonhosted.org/packages/43/1f/1c3d90d91811c8f86ced9ed637956c54bfe5b79ca98fe976d7f8c8979f6b/cffi-2.1.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c", size = 214722, upload-time = "2026-08-03T21:20:04.377Z" }, + { url = "https://files.pythonhosted.org/packages/37/6f/3b5ce4c3b2192d250f04908f2bfd91ef34552ec8f7716a5d4abdb8d67bb2/cffi-2.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125", size = 222369, upload-time = "2026-08-03T21:20:05.544Z" }, + { url = "https://files.pythonhosted.org/packages/02/10/4b3c75dde3d9663c9e02ba05c2668b954f671d4bbe346413ca8c696b295a/cffi-2.1.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264", size = 210175, upload-time = "2026-08-03T21:20:06.75Z" }, + { url = "https://files.pythonhosted.org/packages/df/62/14f74b9543e605d17701dc797b815958b8bb70b7624ce1b832ddad48ed6c/cffi-2.1.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3", size = 208670, upload-time = "2026-08-03T21:20:08.04Z" }, + { url = "https://files.pythonhosted.org/packages/95/95/86342356ff5953b3fb06f7ef7c5bee212d45e770abc7218d451b9148313c/cffi-2.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2", size = 221824, upload-time = "2026-08-03T21:20:09.274Z" }, + { url = "https://files.pythonhosted.org/packages/eb/ff/7b3429ff53aafe931ed8a5fc69f481bbef7ba6de87ddcbb63d08f483f613/cffi-2.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b", size = 225148, upload-time = "2026-08-03T21:20:10.7Z" }, + { url = "https://files.pythonhosted.org/packages/34/34/a95870b9221e09cf4f2ce3178b1a210abdfe63a1bd357da940418d7b8d15/cffi-2.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7", size = 223564, upload-time = "2026-08-03T21:20:12.165Z" }, + { url = "https://files.pythonhosted.org/packages/70/ea/839b50531021a647fb5e929f72cf97bc1ff702b5472166164b5b6e76b851/cffi-2.1.1-cp313-cp313-win32.whl", hash = "sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac", size = 175263, upload-time = "2026-08-03T21:20:13.559Z" }, + { url = "https://files.pythonhosted.org/packages/60/a6/8b149b2c3f2e11aaa1618ef64500b45f50f22c57a977a4dff1aff1f91042/cffi-2.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d", size = 185688, upload-time = "2026-08-03T21:20:14.69Z" }, + { url = "https://files.pythonhosted.org/packages/01/9a/11f687cb39d6a3504060d5242f04f48c735afb4d3d533958a20594890cb2/cffi-2.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973", size = 180078, upload-time = "2026-08-03T21:20:15.917Z" }, + { url = "https://files.pythonhosted.org/packages/d3/7b/d6bbf82b8b96e7391438898c42f5bd96dd02030fd5b64937d248220003e2/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c", size = 194064, upload-time = "2026-08-03T21:20:17.148Z" }, + { url = "https://files.pythonhosted.org/packages/94/e6/bcc91b283be94735e268487a054004f0aa19947b6348fa367db53230abc8/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb", size = 196720, upload-time = "2026-08-03T21:20:18.268Z" }, + { url = "https://files.pythonhosted.org/packages/d9/99/c4b0c17cacdc9c3b8f280026286a9826d6a208c0f047591a3c3ce99b91fd/cffi-2.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54", size = 184964, upload-time = "2026-08-03T21:20:19.708Z" }, + { url = "https://files.pythonhosted.org/packages/b3/a9/9db617d05d7367c1ad0ab00b3aa6e6f9281edd689b4ee9ea0e5a84e89c97/cffi-2.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72", size = 184962, upload-time = "2026-08-03T21:20:20.833Z" }, + { url = "https://files.pythonhosted.org/packages/67/b8/b42132ca113dc567d37684437b46ca1dafc885902b02a110a02d5b511857/cffi-2.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1", size = 222328, upload-time = "2026-08-03T21:20:22.118Z" }, + { url = "https://files.pythonhosted.org/packages/80/10/c5c0cbf0a657aecf59ef511409734230bf556f05a0d6c9eed7aa5c0a0166/cffi-2.1.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062", size = 209985, upload-time = "2026-08-03T21:20:23.401Z" }, + { url = "https://files.pythonhosted.org/packages/d5/6c/bfa0b87b03b9238148beca990292843c9396ba069b54496596594173de7b/cffi-2.1.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03", size = 208530, upload-time = "2026-08-03T21:20:24.628Z" }, + { url = "https://files.pythonhosted.org/packages/e9/02/4e7d553a7ac4b4238b38b3c1b80d486e9d4436f8d2acbf87a0997fe3f402/cffi-2.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96", size = 221525, upload-time = "2026-08-03T21:20:25.758Z" }, + { url = "https://files.pythonhosted.org/packages/82/1d/a4aaf9babd75acb4d5f223bff71533bee748dd770a382619a798960ee9ba/cffi-2.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527", size = 225053, upload-time = "2026-08-03T21:20:26.985Z" }, + { url = "https://files.pythonhosted.org/packages/81/10/5dc0e7bdd18e22107054288283380fc97a06ae3f1656a106908d666a3c88/cffi-2.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13", size = 223213, upload-time = "2026-08-03T21:20:28.277Z" }, + { url = "https://files.pythonhosted.org/packages/0b/e9/d0061c364cde06ee43168a0d076ac1da512cbc380d44767b844ba34fe2b6/cffi-2.1.1-cp314-cp314-win32.whl", hash = "sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c", size = 177682, upload-time = "2026-08-03T21:20:44.288Z" }, + { url = "https://files.pythonhosted.org/packages/a7/06/1c3e01e3ba14c39f6d10bfbac52753b7e22259e38088e5cfe1d704918690/cffi-2.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48", size = 187949, upload-time = "2026-08-03T21:20:45.623Z" }, + { url = "https://files.pythonhosted.org/packages/87/5b/da4e39efe18eeb89cf580ea9cfc66b6a7c3eadb808fc0cc1d3a295cb5a5d/cffi-2.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836", size = 182947, upload-time = "2026-08-03T21:20:46.955Z" }, + { url = "https://files.pythonhosted.org/packages/23/59/40338bf421c5accea1d45158170c87006ef1cd371b05c077e76476949728/cffi-2.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3", size = 188504, upload-time = "2026-08-03T21:20:29.495Z" }, + { url = "https://files.pythonhosted.org/packages/7d/47/5ecf1023850036e674c77ec4de86182d309ae344e39e7cba984b7df5d647/cffi-2.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2", size = 188259, upload-time = "2026-08-03T21:20:31.291Z" }, + { url = "https://files.pythonhosted.org/packages/2a/9c/92934c3bea9f785b23eba304538c0b4d37a2a96d2431eb3a1bc87a11aa19/cffi-2.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94", size = 223864, upload-time = "2026-08-03T21:20:32.571Z" }, + { url = "https://files.pythonhosted.org/packages/4d/45/ba4c93527bc38616a8bd36488acb69a2212d60486794f0c1f318949bbb76/cffi-2.1.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc", size = 211538, upload-time = "2026-08-03T21:20:33.808Z" }, + { url = "https://files.pythonhosted.org/packages/80/e9/b6ef565e452acb932fb0cb5443f44a78efbd1233e566f02b5a83855e9115/cffi-2.1.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29", size = 210688, upload-time = "2026-08-03T21:20:34.974Z" }, + { url = "https://files.pythonhosted.org/packages/9a/95/eff5f0cee78d2eabc7eebffec40d3fc1876b5f3c95582e018bb4b99601f2/cffi-2.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676", size = 223803, upload-time = "2026-08-03T21:20:36.564Z" }, + { url = "https://files.pythonhosted.org/packages/fa/01/579d39fb8bef00a335a23d83757b44feb24cd6345a2c451b64cb67b9c362/cffi-2.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e", size = 226763, upload-time = "2026-08-03T21:20:37.816Z" }, + { url = "https://files.pythonhosted.org/packages/8d/b0/0b44f47c60b01b57b6e2bbd92343f13a85a1d93bc46ccf6e47e244acd99c/cffi-2.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f", size = 225688, upload-time = "2026-08-03T21:20:38.959Z" }, + { url = "https://files.pythonhosted.org/packages/eb/d2/3b7176cb570a1d3e27faf67b72f591af508036e0d8b2be2ef9af9e8c84bb/cffi-2.1.1-cp314-cp314t-win32.whl", hash = "sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4", size = 182868, upload-time = "2026-08-03T21:20:40.388Z" }, + { url = "https://files.pythonhosted.org/packages/56/78/31f00c1bcd97c9bbf55f1bfdf5bc809a5de8887473e90bb9960dca825e80/cffi-2.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e", size = 194104, upload-time = "2026-08-03T21:20:41.725Z" }, + { url = "https://files.pythonhosted.org/packages/7b/1b/58496f2ed0a35de575250c02a43ab3cc2c04d494a88fed31c1cabc0fd176/cffi-2.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5", size = 186402, upload-time = "2026-08-03T21:20:43.042Z" }, + { url = "https://files.pythonhosted.org/packages/c1/8f/9ebe220eab48a093d1a5a5e339ab0dc7316eef3bb04d63c42f0251b61f50/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d", size = 194043, upload-time = "2026-08-03T21:20:48.179Z" }, + { url = "https://files.pythonhosted.org/packages/ff/69/844bad3ece306c4782c2ecb93597035b6690d48704b803914c199da1e8b3/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b", size = 196737, upload-time = "2026-08-03T21:20:49.457Z" }, + { url = "https://files.pythonhosted.org/packages/1b/8a/af668013284634733f02d683458a0728739c7d6ddb5e14cb0c20832266fe/cffi-2.1.1-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4", size = 184933, upload-time = "2026-08-03T21:20:50.639Z" }, + { url = "https://files.pythonhosted.org/packages/0c/75/2f5207ff6d1a613133b23a5203cc0c2a628313b5eb3974d7956ae3c57950/cffi-2.1.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8", size = 185002, upload-time = "2026-08-03T21:20:52.173Z" }, + { url = "https://files.pythonhosted.org/packages/e2/31/9e1313b0a6e30e91b3b3d3fff51ae99c857c07738e3afcce1f7334e1b7ab/cffi-2.1.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6", size = 222271, upload-time = "2026-08-03T21:20:53.462Z" }, + { url = "https://files.pythonhosted.org/packages/50/e3/f6234a833e6e08c7007003074723c406559eecf9b48dfc97471e5a8eb7a0/cffi-2.1.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80", size = 209919, upload-time = "2026-08-03T21:20:54.783Z" }, + { url = "https://files.pythonhosted.org/packages/0d/fc/5f74e293fced6edb51af3a46c4ccf6c23c9943774ecb375ddbd522c76add/cffi-2.1.1-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779", size = 208529, upload-time = "2026-08-03T21:20:56.066Z" }, + { url = "https://files.pythonhosted.org/packages/44/16/29e6d01b388bef055ecd6ca8244b3f4d336bd09e92d5d892187b9601084e/cffi-2.1.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399", size = 221630, upload-time = "2026-08-03T21:20:57.336Z" }, + { url = "https://files.pythonhosted.org/packages/a4/18/fa7f1f6857d5eb88a4ca99ffcbfb7c387a287ccc154c64a73e86314745d7/cffi-2.1.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688", size = 225134, upload-time = "2026-08-03T21:20:58.675Z" }, + { url = "https://files.pythonhosted.org/packages/e0/9f/e8e3dfa04a1b4c241f8c91faacad872b4d4efd051d49764ad4e2fd4b9fea/cffi-2.1.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7", size = 223197, upload-time = "2026-08-03T21:20:59.968Z" }, + { url = "https://files.pythonhosted.org/packages/f8/7e/8debeb04f1ab9fe2a6963964cd6f1aaf7192627b83926586a6a4e089c9fa/cffi-2.1.1-cp315-cp315-win32.whl", hash = "sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac", size = 177683, upload-time = "2026-08-03T21:21:14.901Z" }, + { url = "https://files.pythonhosted.org/packages/e0/31/5158704cc474ab65c1647932e88be78dc0873f47130e253be38bcaf13d01/cffi-2.1.1-cp315-cp315-win_amd64.whl", hash = "sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960", size = 187897, upload-time = "2026-08-03T21:21:16.108Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4b/b3a2da8570c704ffc0f9762cdc3ec0f02c8573798e0b5cf7f11c82bbb70f/cffi-2.1.1-cp315-cp315-win_arm64.whl", hash = "sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1", size = 182935, upload-time = "2026-08-03T21:21:17.271Z" }, + { url = "https://files.pythonhosted.org/packages/d0/ef/5443574510a1207e6f6bc38ba6e1f1de36cb48fef07b2728bb896a21f430/cffi-2.1.1-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc", size = 188464, upload-time = "2026-08-03T21:21:01.163Z" }, + { url = "https://files.pythonhosted.org/packages/7e/ae/a56fa8c4686ad50e148fcbc8d3ae0d03915ff5c30d795058988c24118cef/cffi-2.1.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab", size = 188262, upload-time = "2026-08-03T21:21:02.382Z" }, + { url = "https://files.pythonhosted.org/packages/53/b2/6187f46f2912276a3ae284076109cc5c8680482f11f766ccf26db4a86427/cffi-2.1.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e", size = 223779, upload-time = "2026-08-03T21:21:03.553Z" }, + { url = "https://files.pythonhosted.org/packages/8a/f6/c3ad28bd19f77047a03084424fbd4cbe997303267c14423737324be0385d/cffi-2.1.1-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358", size = 211520, upload-time = "2026-08-03T21:21:04.863Z" }, + { url = "https://files.pythonhosted.org/packages/a0/cd/ccac9013a5bd9fd764de118674ab9c805b5ca10c19270d90ee273f8b2240/cffi-2.1.1-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231", size = 210673, upload-time = "2026-08-03T21:21:06.223Z" }, + { url = "https://files.pythonhosted.org/packages/52/86/2976131c639aead931c5bee5aba67e4b09fbeb8018b6f282f70803f923a7/cffi-2.1.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6", size = 223835, upload-time = "2026-08-03T21:21:07.539Z" }, + { url = "https://files.pythonhosted.org/packages/ac/0c/33a7aeab2f9c76918c52e084beb39c570db3588133412929e8ec06fab90b/cffi-2.1.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94", size = 226705, upload-time = "2026-08-03T21:21:08.774Z" }, + { url = "https://files.pythonhosted.org/packages/e3/26/2cde30fdde421130bfc18f70395731a6e6b2053c6a1978a5258ff04e72fa/cffi-2.1.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5", size = 225539, upload-time = "2026-08-03T21:21:09.911Z" }, + { url = "https://files.pythonhosted.org/packages/6d/cd/a361394c94b2129d604bb846f624a8e88255a3ee33129c434a00d715e64f/cffi-2.1.1-cp315-cp315t-win32.whl", hash = "sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66", size = 182707, upload-time = "2026-08-03T21:21:11.226Z" }, + { url = "https://files.pythonhosted.org/packages/9b/b5/ba2b299993c26577d529b6ae29841f9e15b9fcf004d65f423f4fcf94ade9/cffi-2.1.1-cp315-cp315t-win_amd64.whl", hash = "sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3", size = 193772, upload-time = "2026-08-03T21:21:12.39Z" }, + { url = "https://files.pythonhosted.org/packages/aa/29/35e016098c814cd93de9cd320c66b5bfba14dc6ecedd3cb518fa7c408c69/cffi-2.1.1-cp315-cp315t-win_arm64.whl", hash = "sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692", size = 186360, upload-time = "2026-08-03T21:21:13.636Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.5.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cb/31/4971872b3ed8715346231fb6eb4da8fcba65a4143c189db151ee28a2812b/charset_normalizer-3.5.0.tar.gz", hash = "sha256:49bd5feb59b0bf3cbf6ebcf4352e371c95b9da9bacd4449f8b64d0ad2c10a26e", size = 169295, upload-time = "2026-08-12T14:35:31.624Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/50/42/71e4e3bfe59202feef062c68487f54c6adf501cfbe087ecd93e3cd597fea/charset_normalizer-3.5.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:e46a37ea7fcf9ae01d71b2e5ece19f1565987f3e308394b829197cbefc061f92", size = 349110, upload-time = "2026-08-12T14:32:07.832Z" }, + { url = "https://files.pythonhosted.org/packages/0f/dd/fd3386d0fbd358d3b5c7a2fa5bf312afe6159b04fafeb67d39fa971d7448/charset_normalizer-3.5.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1cdfed4d7a59333c8220c67dd3be4e7a6c887b67453a64394022dcc919570add", size = 250773, upload-time = "2026-08-12T14:32:09.113Z" }, + { url = "https://files.pythonhosted.org/packages/17/ad/4901a66d6d3b17f1096725d7e50266132c16555aa6a70047fe1cf262b4b2/charset_normalizer-3.5.0-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9491f594859b68052edebd69e05fb045055a713b57a67974e6c1553b4e503c39", size = 240229, upload-time = "2026-08-12T14:32:10.43Z" }, + { url = "https://files.pythonhosted.org/packages/43/b7/1d790e0425e0f4c99e9b89a3956a94a6d2d0c6f01b2a5eca93d8f082d5ac/charset_normalizer-3.5.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:420b19411959eec115063229536788e6b32d0a7fa907d6b940317919120d702d", size = 280757, upload-time = "2026-08-12T14:32:11.628Z" }, + { url = "https://files.pythonhosted.org/packages/44/bb/4b8c8086c67636e52d6354ad17697f54a00b40041ca53dc765737e21709b/charset_normalizer-3.5.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a565303d118ea3b94a4b6c076bf568069726be414e43b06d58f7070b076ce11d", size = 276174, upload-time = "2026-08-12T14:32:12.808Z" }, + { url = "https://files.pythonhosted.org/packages/25/fa/690a11924c40c766d258d2b74d817cc5efe7bbcfceeedc5c0f35256d7524/charset_normalizer-3.5.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:815f143a91983ba3041bba066e492ae3c42de523fb1c699685a1abf3313b7d1b", size = 261808, upload-time = "2026-08-12T14:32:14.138Z" }, + { url = "https://files.pythonhosted.org/packages/53/2d/4a6945eff0c8f684e3f5b7b978644ab18ba9198da060dbaa1d9206bc6cc9/charset_normalizer-3.5.0-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c5e981a5ac8641381efe6f0029467500661616a530d27bc6eedfe45f840599f8", size = 259922, upload-time = "2026-08-12T14:32:15.485Z" }, + { url = "https://files.pythonhosted.org/packages/60/4d/10ac7e07bbf7ea569effeb9524e32f345f7e643800b20d538fb4706eab4e/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1a573e1e428f93908e79e04b349717f400e720f2f82285f0aaaf3ee0ff7f4c79", size = 252315, upload-time = "2026-08-12T14:32:16.764Z" }, + { url = "https://files.pythonhosted.org/packages/13/bf/1ddacf7aa7da12097f229a6a4e71a70200937a621b3617f6dc819fb99a66/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:32e6d56dd825205f81e5c45bcebb4df6a11fb2bbf4969a01ef156d6ced90c224", size = 240600, upload-time = "2026-08-12T14:32:17.956Z" }, + { url = "https://files.pythonhosted.org/packages/c2/88/293444d48c8d86fe51552262117134a9fc666d68cbbbc9b8c1b2b35a29be/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:30ae26a1adcd943690dcbbc47f28be762bae9e08ad7442b78c86b1c0dd5a626c", size = 280788, upload-time = "2026-08-12T14:32:19.1Z" }, + { url = "https://files.pythonhosted.org/packages/12/ed/0e34e40584f51eda38d4a5daf25fd8586366347efd4b2470dbf64710e778/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:5f51a19dc52197a20218b05ec5336d0c6b3b09935f838724722032c8d45dc91a", size = 258425, upload-time = "2026-08-12T14:32:20.207Z" }, + { url = "https://files.pythonhosted.org/packages/72/b2/c1b1c27f6f0ef35b21a8bdb592854bbf7f219629db3477f74c0b1380e0ed/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:6e44bc2780516b3df986d6fe33103c7080cd9dcd5576fe3cb4b0f64309c8f22b", size = 277437, upload-time = "2026-08-12T14:32:21.345Z" }, + { url = "https://files.pythonhosted.org/packages/b2/bd/a1b7d959a37847675adb2f5978f81703306dd78df4fefb82ee5a1cc5e37f/charset_normalizer-3.5.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:7faa47b56070b3dd6f4898ed28528843ab130d53266cb9948d9b1f3bb1a5c5e8", size = 262947, upload-time = "2026-08-12T14:32:22.613Z" }, + { url = "https://files.pythonhosted.org/packages/c1/fd/a76a2d7e639bfc6a9c869371e34f28231eaca7d7126ec19895aa38eedb51/charset_normalizer-3.5.0-cp311-cp311-win32.whl", hash = "sha256:830c04a49998b5ed58c8b642c65b7b26419397f52392a64121ba9fd0e95e7f9f", size = 181313, upload-time = "2026-08-12T14:32:23.736Z" }, + { url = "https://files.pythonhosted.org/packages/97/84/6fc03e802578df41a2ab9b6a1f26657fb92e32285a603ad5852a4a4f68c1/charset_normalizer-3.5.0-cp311-cp311-win_amd64.whl", hash = "sha256:8cb9b6892b53bd6d11fa4cde3dbee020b1f0b6656be1fbaa1ec0d4324a7839db", size = 206197, upload-time = "2026-08-12T14:32:24.938Z" }, + { url = "https://files.pythonhosted.org/packages/0a/27/7208360ff1901607359869fcc45ca0989d597f3e30777ba30c7254170587/charset_normalizer-3.5.0-cp311-cp311-win_arm64.whl", hash = "sha256:2403b489c103e9a18c835863fc6dd54361355c8291d4cafdb37492b683440b9b", size = 184925, upload-time = "2026-08-12T14:32:26.123Z" }, + { url = "https://files.pythonhosted.org/packages/6d/3c/045ea64ea5a550870dd8ab60b2242870328d53f17d2be593b4f9f3121474/charset_normalizer-3.5.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:98820e1ceb25c6df7a80c4fd8efa59cb121f99bc7c4c1693ad94a2caff5b311d", size = 343861, upload-time = "2026-08-12T14:32:27.254Z" }, + { url = "https://files.pythonhosted.org/packages/fc/1b/7502be709db899d5b4801509829188b3a5a10969411da9c846115a5f1b70/charset_normalizer-3.5.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:608553f476fca509537e804c4a71f5eb166ce63b75141f89c2c686ce1aa36956", size = 237550, upload-time = "2026-08-12T14:32:28.385Z" }, + { url = "https://files.pythonhosted.org/packages/ca/ce/66392661375148d9455c17bee25509a54e28c39969e34befa48ec8777936/charset_normalizer-3.5.0-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6753de11eef42f1c321b26d682957d92c7f7bbce6530f34bbe0f9291dd37cc6f", size = 229673, upload-time = "2026-08-12T14:32:29.668Z" }, + { url = "https://files.pythonhosted.org/packages/6a/64/f58c32a8d4ecf55b82ee61ee9aa6a664d4afcd36c72feb4c926fd6fe9af8/charset_normalizer-3.5.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0f76dc0a47f94cb9b69d86f01e477f4b0371ca70208b9ccea7e063c41eed9046", size = 260768, upload-time = "2026-08-12T14:32:30.891Z" }, + { url = "https://files.pythonhosted.org/packages/b1/ea/36c90e59a96386174377e855479ec154221ef001e96637e0b23be92489c4/charset_normalizer-3.5.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c387c6bf91b4774e359a48a179e2872b8e8bf741e4fde06ba8d1665eb9a4760a", size = 257880, upload-time = "2026-08-12T14:32:32.175Z" }, + { url = "https://files.pythonhosted.org/packages/23/35/5b85772eb82528ef22ba29487ad544a7049dfd27f35b1a5a55dbc0843048/charset_normalizer-3.5.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:14f6904a3cf870abf044df3a8c4924ac6c8ef77e9896586fd37e73ae96cff2af", size = 247547, upload-time = "2026-08-12T14:32:33.495Z" }, + { url = "https://files.pythonhosted.org/packages/b6/14/ba11a99c2a22ab04c2d5383a700b378cb463a78ab15f36444cabc10cd671/charset_normalizer-3.5.0-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0cce46dd29d73e135e8087b96eb62a4aca6d69391b7f97808c6588ebed3178f3", size = 243326, upload-time = "2026-08-12T14:32:34.794Z" }, + { url = "https://files.pythonhosted.org/packages/4c/4a/cadba3f2400b45aa1d62a4ae0298bf58a3b30b1158baf15c338c7ce5b601/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b476cdb63df22da2b91837593380be3ddbe406f36c506c1c91d80e7196b66288", size = 238820, upload-time = "2026-08-12T14:32:35.984Z" }, + { url = "https://files.pythonhosted.org/packages/47/41/d5188b9342d75b72c2b05d3ee373f01a691397e770f001ee05e3b37925f5/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:1f56ce84b317ef2a59d7d3461891c7597c79247d2192bb8114c68a1a1debfcc0", size = 231661, upload-time = "2026-08-12T14:32:37.191Z" }, + { url = "https://files.pythonhosted.org/packages/13/5f/df38fa972c4e945c3d8cee2bc4e610613af522fd359c7dc7a74c419f0278/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:9ce0f885239357379d92fd9a5fddbe20f0e30e0527c29ba69f8e99eeb1304a76", size = 261459, upload-time = "2026-08-12T14:32:38.369Z" }, + { url = "https://files.pythonhosted.org/packages/a5/d7/043ff7720067a3beee05523465ac9c1c846c68b7884930dd483f72ee5ab6/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:96ae7ab5d8155fde927aa0864fbc8ba3cc4fde6d41ab0c7cea9d6012b4978603", size = 242300, upload-time = "2026-08-12T14:32:39.505Z" }, + { url = "https://files.pythonhosted.org/packages/19/f6/33980b7b802a048e546a6d9ad2ea783a6cf6b10a86aaccd10db462d8b913/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:bf91921009025e96ce57a03ced6d14604fc3baf0530351638e9504a55da6fa3b", size = 259101, upload-time = "2026-08-12T14:32:41.02Z" }, + { url = "https://files.pythonhosted.org/packages/a6/b7/0d19bde844bff9165377c1da9ef3c4792a4c24bd49b5b7094d9e6f6ab58b/charset_normalizer-3.5.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0b2e44e6d42d1a4ff78ccc219a93c5449105d10b16198d1aea581080df8073f9", size = 249246, upload-time = "2026-08-12T14:32:42.47Z" }, + { url = "https://files.pythonhosted.org/packages/1c/cc/2c34fdfaacdf0e96e880ef562cbf80a9b5f8ea97e0dd9e57ba348a9c65cf/charset_normalizer-3.5.0-cp312-cp312-win32.whl", hash = "sha256:deb99535e9bf0bea8e274c6413eb939a21be35a3f492678dba4d5b1f4d70f142", size = 178025, upload-time = "2026-08-12T14:32:43.909Z" }, + { url = "https://files.pythonhosted.org/packages/76/d0/c34dbd1df23bcbdc1b5d2f48256340d72fa747f1eb03924a9d2fa35ed85b/charset_normalizer-3.5.0-cp312-cp312-win_amd64.whl", hash = "sha256:e54dd1a66fa4bce0ccaf0db9dde336e49b3eec646dc4c1c0991279369d373a14", size = 200143, upload-time = "2026-08-12T14:32:45.254Z" }, + { url = "https://files.pythonhosted.org/packages/c0/ea/4e6bf1465d60c3d8f488d5bde140d0cb91cd23ccab0dc2895cc6c6982047/charset_normalizer-3.5.0-cp312-cp312-win_arm64.whl", hash = "sha256:b8ea208b304587d47931b36481342d20336e0d338ab052f8b4305926482598d6", size = 180046, upload-time = "2026-08-12T14:32:46.545Z" }, + { url = "https://files.pythonhosted.org/packages/1d/be/cc7b7b6fc41984902c0d31b06f5d9297e67705c1dae9352608e5540fad09/charset_normalizer-3.5.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:5c23fa4f6eccdd601949cb00f3988c01d64e671d8faba356397971077022e144", size = 211050, upload-time = "2026-08-12T14:32:47.81Z" }, + { url = "https://files.pythonhosted.org/packages/d3/ae/e3ec8f17313609f43f7b323012fdb1ee37b83432277ca4eceba83e00366c/charset_normalizer-3.5.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:07f6f42b5a6325df35b458004fb5f9f29bf502d89287a33c7cdef3590e31de0f", size = 222768, upload-time = "2026-08-12T14:32:49.027Z" }, + { url = "https://files.pythonhosted.org/packages/c5/50/9f9c0d7ccc1512d49e27a0e7c12c58ec71dfe91698fa4326f058c33e1f1b/charset_normalizer-3.5.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:8efc3f1563ed431882dd0dc0411b5f8ace1b1b89074981deaf6bd8af77dbe1bc", size = 193907, upload-time = "2026-08-12T14:32:50.414Z" }, + { url = "https://files.pythonhosted.org/packages/fd/1d/cfe7b745ef7f4c3b7214581955b5a0869ba2ac551a58fc11036281ae167c/charset_normalizer-3.5.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:368eb2fc9482158b3a3386e8f01fa61f479c968e9a19ceab8f0188b86b312991", size = 197135, upload-time = "2026-08-12T14:32:51.605Z" }, + { url = "https://files.pythonhosted.org/packages/28/55/30fafdcfca9ba616bc394240545e4cd52f4f66dea43ded81b7d2d5274fde/charset_normalizer-3.5.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:826a295a039178479a325be1ae60eded1f0b10f7dda749df59e2440de8f61d64", size = 339892, upload-time = "2026-08-12T14:32:52.82Z" }, + { url = "https://files.pythonhosted.org/packages/f4/08/bdca5fc2bdc36ee443673dc7d12b23885a5a7b282bef85a1a4c3b325b40e/charset_normalizer-3.5.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:70ff1c16eb0eb5ee6bb12739292347f981a5ba764cc4df1bc2e69b0405d4ac3b", size = 239439, upload-time = "2026-08-12T14:32:54.058Z" }, + { url = "https://files.pythonhosted.org/packages/d1/9e/506c8d7a7722bba7c8cdd78c1b5ef23bda92bfbe0b3e28ea84673d519a0f/charset_normalizer-3.5.0-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3cfdab178a4add5483e26a9bb1c16d8018ccf39b4be7a3aea6c3979e6828f2ee", size = 227896, upload-time = "2026-08-12T14:32:55.326Z" }, + { url = "https://files.pythonhosted.org/packages/0d/1a/dd828f2b1d6f4bf10821b9a74d866be05ffcdbfcddfc501d6fe6428762a7/charset_normalizer-3.5.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6083d10a846218502d664375b9448508d9fa580bd834567423156c6abfbe899d", size = 262548, upload-time = "2026-08-12T14:32:56.484Z" }, + { url = "https://files.pythonhosted.org/packages/be/81/196d26f6bd78b93e0d451b69082a71027ceeddd4b0be9170b81bb038f824/charset_normalizer-3.5.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0f211c21aa316cb6e2662e54a1194633a79d98a50a876addacfce7ba5b34b09f", size = 259986, upload-time = "2026-08-12T14:32:57.661Z" }, + { url = "https://files.pythonhosted.org/packages/3a/6a/5b964a1eb0f9075ecd45083eeb21aaec215334f98bac3d400302ea73875d/charset_normalizer-3.5.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3b08ebf9488c7ff5eff038e48e6ea938178dfd9dcc8598b5ca941e4ae27b20be", size = 249853, upload-time = "2026-08-12T14:32:59.123Z" }, + { url = "https://files.pythonhosted.org/packages/c8/b4/aee3a9d82edd0e931091ef3e9f03e46491ae3590e96e998d0975dadbe17c/charset_normalizer-3.5.0-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6c95450fce59f00c6d08eff6572ec2e736e5054c9450253afd5748f8416f2eb9", size = 244217, upload-time = "2026-08-12T14:33:00.341Z" }, + { url = "https://files.pythonhosted.org/packages/22/3e/33f72ca11c1b619b220fd9f35905ebd171cbd0e0470f2357e467b9e861ee/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:5780a29823e1d2bec69b7a104ead4195a43f3e97782efaedbf1f79a0157af715", size = 241307, upload-time = "2026-08-12T14:33:01.589Z" }, + { url = "https://files.pythonhosted.org/packages/78/27/6029dccba958621c7f3a65136f87c5512d712aef9e890f09512cc171bd03/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:054420b5db984971d886e5e4e2c37c760ae6682aedbd066687ff0949d9ed5f08", size = 233305, upload-time = "2026-08-12T14:33:02.866Z" }, + { url = "https://files.pythonhosted.org/packages/18/d7/691c967be459153fe9faf49bf78bc95639ef8bf6dd008f38cc6389a349eb/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:d016dc857136c726958102c3b8a3986acdc65ace6fbf12cfdc09cc4bfa2935b2", size = 263465, upload-time = "2026-08-12T14:33:04.166Z" }, + { url = "https://files.pythonhosted.org/packages/7c/2d/9202221be5c90b2a835924191e362690ed8dc8c7d6606100c2bd03fe0f8c/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f2ce3d39fb4a9d674e6639dd5d3146b2e273475d2260f10163228d66fc04433d", size = 245060, upload-time = "2026-08-12T14:33:05.325Z" }, + { url = "https://files.pythonhosted.org/packages/b3/9a/298772fd0a0cbccadf36451a1cd7eef4b66a11e99b4a7f6fafc47cc62c75/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:a5613a3a82c974227bde18f03409e30c467f8065cb56d822e3eb83708a5f223d", size = 261091, upload-time = "2026-08-12T14:33:06.475Z" }, + { url = "https://files.pythonhosted.org/packages/82/3b/1a11fe66e555dbe2f5714ade6ba74fa29edc9155d9cf1001d4d6ed096aa7/charset_normalizer-3.5.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:fded2e82ff082e5d8e017e2ddcc1411bd8cb83b8585097fc401ef574f756b888", size = 251857, upload-time = "2026-08-12T14:33:07.784Z" }, + { url = "https://files.pythonhosted.org/packages/17/fc/73b817e8af3f1d25ec5cf458d405abba5a144cf9812238a61530f5eac186/charset_normalizer-3.5.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:8f006866047c6ec4b627ec144b1e0bbc7427cb31fd7c08d19897d0ac9032af3d", size = 139745, upload-time = "2026-08-12T14:33:09.123Z" }, + { url = "https://files.pythonhosted.org/packages/b3/fb/ddb66303c86f7dc5043a457dad9fa82b4d6d0cb97094f9bcdde21693fd58/charset_normalizer-3.5.0-cp313-cp313-win32.whl", hash = "sha256:196e270c4e80827b5072eed7d6aa661d133afada94fe366669f9609e718d305e", size = 177217, upload-time = "2026-08-12T14:33:10.305Z" }, + { url = "https://files.pythonhosted.org/packages/fb/88/6018cc8d76ea2b7cb02918f37e23e86c261d1a102713d7e88d2cfb8b211c/charset_normalizer-3.5.0-cp313-cp313-win_amd64.whl", hash = "sha256:72982d9958a42f8132bf2d6b90214ed66477295ef1188731f98ae3511c6eeb5a", size = 198896, upload-time = "2026-08-12T14:33:11.51Z" }, + { url = "https://files.pythonhosted.org/packages/20/2e/04c0bbfc8d9abf91959f7a3d207d45cbf63a8116984caae2381890019bb5/charset_normalizer-3.5.0-cp313-cp313-win_arm64.whl", hash = "sha256:0b373bab0b867b68b8eb249da9478cab9181a42993437cd2f5dba5fb0b4fbd1b", size = 179193, upload-time = "2026-08-12T14:33:12.726Z" }, + { url = "https://files.pythonhosted.org/packages/43/14/d098868dac5ff27e0258f548b1c74c6484be528384965d8fcf8fc6a4011d/charset_normalizer-3.5.0-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:d95244906ed69d0f79f190893c65e336c15959003e21449256dc05c001b52ea2", size = 211664, upload-time = "2026-08-12T14:33:14.153Z" }, + { url = "https://files.pythonhosted.org/packages/e7/da/a944b32a46601ae5a4c3499e8d64ecd14fe82313f00da74dcdf00273a0b4/charset_normalizer-3.5.0-cp314-cp314-android_24_x86_64.whl", hash = "sha256:d788e2ded0c4c47efa4d73cfe59eaf975ee32f425219873d2cb3e3fbaa00f636", size = 224375, upload-time = "2026-08-12T14:33:15.472Z" }, + { url = "https://files.pythonhosted.org/packages/f7/db/eabb5996be2f529744755e7b2fc9396eff4a64961f034e7fd49d54b9afb2/charset_normalizer-3.5.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:f9f91d3e8382900f3a68fa0ce94294479de9cd2de6bc0c70acd0f0dfd511836b", size = 194364, upload-time = "2026-08-12T14:33:16.607Z" }, + { url = "https://files.pythonhosted.org/packages/78/65/4ad3c5be108930310d8003f5602861d5b89f728293b9f09c3a4837f7ba10/charset_normalizer-3.5.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:d54625cbf4e6b60bf0639728cb8b4cb541e340f6d7cafae5806051a40ddf4c45", size = 197643, upload-time = "2026-08-12T14:33:17.88Z" }, + { url = "https://files.pythonhosted.org/packages/3d/39/8fee3201b98d52289be60a775797d69be05a04fb6cfb48c1587dad33e649/charset_normalizer-3.5.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:1f99a8c3a1da5d955edbad18208b3d627bdd54c48a6e739fa877bdca98c686d6", size = 341384, upload-time = "2026-08-12T14:33:19.239Z" }, + { url = "https://files.pythonhosted.org/packages/a7/dd/9e757101d1f76c35c0643684ba499ac3a181fb2b264c68174bf727d627e8/charset_normalizer-3.5.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bf1e75dc07a3850b53d1e5f75e04d3ae12afe56284be7821771eaa2466350c73", size = 241637, upload-time = "2026-08-12T14:33:20.619Z" }, + { url = "https://files.pythonhosted.org/packages/eb/e4/7857023015400bc4aa0a82fbcca29fa2dc7ec25f971a130764cb2dc7a589/charset_normalizer-3.5.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:ac5a9cc079c67d75f4ddf343276031879eadbb333d1bb231cce297b8d7b9aae8", size = 226170, upload-time = "2026-08-12T14:33:21.773Z" }, + { url = "https://files.pythonhosted.org/packages/7d/ae/8b52935b304f7b6bbf33151ed2b75266b09aa4b6f8f04230d948885b2577/charset_normalizer-3.5.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0dfe83c1b4d00abbf433998117a14f56a5c2bc68226c0d331709eed0d1ce539b", size = 265093, upload-time = "2026-08-12T14:33:22.999Z" }, + { url = "https://files.pythonhosted.org/packages/dc/78/6e838f6bb059f2c0afc60a4e7f294252f043c254656ad4114c50302cae4d/charset_normalizer-3.5.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:e4e8fa586df2208ef040684751345f10f503834a757c9a74ecd19c1a2f9b1ccd", size = 262789, upload-time = "2026-08-12T14:33:24.214Z" }, + { url = "https://files.pythonhosted.org/packages/c2/08/189b27e51fddc9d6b3695331da0e31792c1d88b953ad854e57f06e9b2cc8/charset_normalizer-3.5.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:82cc5835997ec78afe293a192e385099355770a7db94b2fb1239d36b32796f1c", size = 250580, upload-time = "2026-08-12T14:33:25.707Z" }, + { url = "https://files.pythonhosted.org/packages/ac/55/64854e99b25841f83e8e37d9df2f3d1f96f693439f80e5fabd542a7e47ab/charset_normalizer-3.5.0-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:19e52bda45086df8a4be4bb5910af6f5d9d3b538c78712c8ae09ef10b85bf458", size = 245008, upload-time = "2026-08-12T14:33:26.971Z" }, + { url = "https://files.pythonhosted.org/packages/4f/01/7720c904fa635d4260b4dced6029cf3d298c57b26741365d5a8d28c54043/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:3418edd0ecb72a0a3861cf72f31be0ad9b7fe338ce2b58fb5cc80b9aeb792700", size = 243892, upload-time = "2026-08-12T14:33:28.237Z" }, + { url = "https://files.pythonhosted.org/packages/70/50/7bfcb327631d4870c720872b548745f6ec8baa044d51c21b5d1d32ac4e3a/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:125ee619611019471b177c70bc3e9d4cda9fad7e01d93523501d3b188df0193a", size = 230996, upload-time = "2026-08-12T14:33:29.511Z" }, + { url = "https://files.pythonhosted.org/packages/24/51/40c45d6d940c04005ed721aa54bdebf1ebb2930f8a2ae537e8d60484fb27/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:a3ad0e3da22852533858663848608f3f24c0d35e5cde415a4903476f2b4c88ec", size = 265834, upload-time = "2026-08-12T14:33:30.689Z" }, + { url = "https://files.pythonhosted.org/packages/eb/d4/ef7a227ef89d215b47f9df79c3966610b17faa13bb2f236989207a631622/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:4ebebb410bc517e1d284c52a123e82704b21e4e7e26a21ebecf7439d0647b8a3", size = 245544, upload-time = "2026-08-12T14:33:31.859Z" }, + { url = "https://files.pythonhosted.org/packages/37/a9/a4ca9156964ded61c7718eba410ce11be2fd2b263fda4bcf08367b6578cd/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:91f9f7c151e772acebe489eaec96e96a2877202d7dd144e3f96b8676881715a0", size = 264110, upload-time = "2026-08-12T14:33:33.13Z" }, + { url = "https://files.pythonhosted.org/packages/38/6a/838364bb8702229c6e5f8b23f80ff0f052a12dfaf3113a12fd6acbe92a44/charset_normalizer-3.5.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:401ea6e7af9e7852ed818f64714b579c1935482049670847ca3bd7ba45dc63fb", size = 252303, upload-time = "2026-08-12T14:33:34.98Z" }, + { url = "https://files.pythonhosted.org/packages/9c/5f/d88032edce951f499a2321cf7ae0d35a043c74be12bc22d81084cc7afbcc/charset_normalizer-3.5.0-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:f7496aed56b06325a1ad419c5bf23c6dd042558e874f71dd1b958f3e255f3053", size = 139964, upload-time = "2026-08-12T14:33:36.195Z" }, + { url = "https://files.pythonhosted.org/packages/37/ae/1c4a46b6b00d1c34d2ee355ef99ad6173674166800d1af0f05f85028d513/charset_normalizer-3.5.0-cp314-cp314-win32.whl", hash = "sha256:606a86c1c3196f3738de39a67a7490bbd61cb31c0e0436070bd0c6a48170b38e", size = 179790, upload-time = "2026-08-12T14:33:37.356Z" }, + { url = "https://files.pythonhosted.org/packages/01/51/f94dcf34fa8eba48c1fb89b6490a5f1426e19488fe5f38aac6c648c99057/charset_normalizer-3.5.0-cp314-cp314-win_amd64.whl", hash = "sha256:ec6c464cf45867f66a2273e2214d9199a8fbad5cb95ca0fd45f6a2fe1d9d2cf4", size = 203723, upload-time = "2026-08-12T14:33:38.639Z" }, + { url = "https://files.pythonhosted.org/packages/9f/ba/91d386870b5d9e4b0d8c4034f63877cc2e47b99c81ef05f3e6d42bf9a53f/charset_normalizer-3.5.0-cp314-cp314-win_arm64.whl", hash = "sha256:dc28949de1bb5f7f30a46f15d74ce7ac5aaa63e03c5de04d68f571c7423af834", size = 183423, upload-time = "2026-08-12T14:33:39.899Z" }, + { url = "https://files.pythonhosted.org/packages/f1/c9/534ecb17b7fb95f9052c4a44cf316316a27d4a8f73e8475ff55e778dcdd7/charset_normalizer-3.5.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:68b7e84ae8239a94f8d2c8f3f3a3a81bcde54805ec8f42a34de927d155688ec6", size = 368967, upload-time = "2026-08-12T14:33:41.093Z" }, + { url = "https://files.pythonhosted.org/packages/3c/b2/ad7c3242d7fe55cd55126c22c65cb1b49779782cdf8932fd01d12232d86a/charset_normalizer-3.5.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:58ca5dc0a0ef99f2801ec0574214c978e9574055bc783830bbb6e7433218609f", size = 239478, upload-time = "2026-08-12T14:33:42.428Z" }, + { url = "https://files.pythonhosted.org/packages/7e/62/77f0b850048e430fc350ec58876b0c020f5c8d0d3956fd1a4d6ae2fa292f/charset_normalizer-3.5.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6c57af4084c10cb3286688d65e4c654190ff5edcbc2411d08cdca0a8a44c59a1", size = 227036, upload-time = "2026-08-12T14:33:43.635Z" }, + { url = "https://files.pythonhosted.org/packages/f0/ba/47d951e1a51dddbaad0a1410baf49fb1d897ceb00281568f1183b79bce9a/charset_normalizer-3.5.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f1619a3cc174a7e3963dd34348e6fceb6e50db0ddeb0031bd7c73a58286454fa", size = 260772, upload-time = "2026-08-12T14:33:44.96Z" }, + { url = "https://files.pythonhosted.org/packages/b0/61/8c7ff4c81b2a88271126acf4b83ab3e31f6d63868b0f01d331eaa0f9cb67/charset_normalizer-3.5.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1328cc57dd4372be1265f68232cee890e087416e3e6e93e6ffb32c2bad4d36a4", size = 259273, upload-time = "2026-08-12T14:33:46.185Z" }, + { url = "https://files.pythonhosted.org/packages/e1/fd/36129689be08dc287b951306946657ff70d76e287dd57018861f86d0e474/charset_normalizer-3.5.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:06f4fb62a9139bef056b8b2da6773c94c2f259f90e4b8e53b166f3d0372d7cf6", size = 248086, upload-time = "2026-08-12T14:33:47.54Z" }, + { url = "https://files.pythonhosted.org/packages/cf/f8/bcae67f994c8fd31dda445e5ebf84045823c31443fe46f0e9ee6aca99aa0/charset_normalizer-3.5.0-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:478650a70a750d75d5add401606c77f77069c32e4ba2c9131dc6cee566962ca0", size = 242671, upload-time = "2026-08-12T14:33:48.746Z" }, + { url = "https://files.pythonhosted.org/packages/61/92/0472cdad1061c2f0e4d3aee29973eb6e81bb8fe256ff2860cf115b15f1c9/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:48920bf6fe83eb2226756ac623fa54940487154eb18f80889d5735cf234965c0", size = 241311, upload-time = "2026-08-12T14:33:50.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/89/04a03de5d27c77c624d9fcf6287073754bd438df1b58cb7d030c57c2824d/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:168a0cb536b5123a77bc42ecf5e0bf6f923d0d9ae43c42a14eb0677c19ac6c19", size = 229898, upload-time = "2026-08-12T14:33:51.523Z" }, + { url = "https://files.pythonhosted.org/packages/42/a2/639c4278adcb7ed1f4db608dd9ac19b6774fa2285a96b1c0bdb9c124ccbd/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:f278e131afa96a3622cef9211c406ea2ad1b68eb06f8837cd443684a40e0ae50", size = 262852, upload-time = "2026-08-12T14:33:52.924Z" }, + { url = "https://files.pythonhosted.org/packages/9d/95/02e34c97bedfd0c5574efb9179c850591acc7f967ba039ed8dd29d332b73/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:d6100f877d2ed95f0856a3fde25334153add94bf2224c43f45f88e7039262aaa", size = 242913, upload-time = "2026-08-12T14:33:54.18Z" }, + { url = "https://files.pythonhosted.org/packages/a3/64/0946aeab6462dad9f160a50dfb4704d3f58a5ee708f085abc2105fbbff0c/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:4c440122e1ea68b1f8b44a631ebf49c39180f6869b1da22d76e8a724208ec6e9", size = 257938, upload-time = "2026-08-12T14:33:55.802Z" }, + { url = "https://files.pythonhosted.org/packages/79/77/36787d41ead124746506a4425c729f4f17c68280af8a6a5baa0a598cae86/charset_normalizer-3.5.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:e5f834965c2fe589837bac1002e07e25734ff70381903ccd95b3d649e22bfa40", size = 249467, upload-time = "2026-08-12T14:33:57.081Z" }, + { url = "https://files.pythonhosted.org/packages/65/10/d9f6c5589cd24198d4ce6cd2948191c18e657272f433e5a00d258d9f5c22/charset_normalizer-3.5.0-cp314-cp314t-win32.whl", hash = "sha256:076cf9d3f3c7e410295c09d96355cf3b1bcae74990034d80e4371e20fe1ba4c6", size = 190624, upload-time = "2026-08-12T14:33:58.449Z" }, + { url = "https://files.pythonhosted.org/packages/6c/81/43e0584a802051a22c725795ebe1df78263abc7de858eef6cdc9b36637e9/charset_normalizer-3.5.0-cp314-cp314t-win_amd64.whl", hash = "sha256:3288a560dc3114d5d2ebe309b1ef43f8af355eafe25856832415c2a8196c9db3", size = 215902, upload-time = "2026-08-12T14:33:59.753Z" }, + { url = "https://files.pythonhosted.org/packages/30/f3/af6a1160fef0eac4510d035241e11eccf78e5350e4cd4de79e79fe02a5e5/charset_normalizer-3.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:a284c36b9c6616bf0a8aa4aabba668a0c75ba65ccf40a79868aeaa69ad996897", size = 193452, upload-time = "2026-08-12T14:34:01.017Z" }, + { url = "https://files.pythonhosted.org/packages/42/a4/dee470afb7a55c4f78b6fef37306c51fed17ebf94dbe530798c91d394350/charset_normalizer-3.5.0-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:c38d1e9bc2073b0984d2099ea647fd7f6c0d8f83a1e14e0cd32926f16e4c44ce", size = 341595, upload-time = "2026-08-12T14:34:02.4Z" }, + { url = "https://files.pythonhosted.org/packages/6a/32/9c3126dc429c6d9d7f79c52681a7c4453ed20a26267c9a8275d7ab620aba/charset_normalizer-3.5.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c9f45186390aee4d1f26f723c615b67df346766c3b16df000d84d6e374f06757", size = 242177, upload-time = "2026-08-12T14:34:03.741Z" }, + { url = "https://files.pythonhosted.org/packages/4b/9d/5b616a887301ff4cc0916b39ba44257390d3da80deeed6e8b6f2f26b14a8/charset_normalizer-3.5.0-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:f0fde5e5100c735b2274ab898f0742a5dcde492796296cfbe7e0ad6a4cd1a396", size = 236730, upload-time = "2026-08-12T14:34:04.991Z" }, + { url = "https://files.pythonhosted.org/packages/be/b4/d6d3e70be93ebe5fabef65e4c7ac113e1d1705cbaeb5fb72467e713aca17/charset_normalizer-3.5.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3d00e18e7bbf47e332ab63903d18bae31efc701b1d8cca0382b97784a621fc44", size = 265158, upload-time = "2026-08-12T14:34:06.235Z" }, + { url = "https://files.pythonhosted.org/packages/30/e7/3f1fafa87e2643257474f9c4eec609f2193a61d907dce7dd4f3f2390ebd5/charset_normalizer-3.5.0-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5b81980668800dd1c69faad8aea6e85a8cee0e13bcd3bba7671695ff16260293", size = 262931, upload-time = "2026-08-12T14:34:07.511Z" }, + { url = "https://files.pythonhosted.org/packages/0a/df/ebeb224a949d91829e5e114c6b64372a3c792b00762a9e951ce416f3a32d/charset_normalizer-3.5.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9bb3e0d1345b9c0fe73673ea656375f38a78ec679c2edeae0c24800f04798a85", size = 251388, upload-time = "2026-08-12T14:34:08.949Z" }, + { url = "https://files.pythonhosted.org/packages/a0/64/9a6ce2e7acc5cf1b4636f78f82e89ff581e06a0216a40678b28bd4d832c4/charset_normalizer-3.5.0-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:2401f7671242e921e604f609d429f6b282ea4ca787a6ffd22ed7372011ddb9d1", size = 251821, upload-time = "2026-08-12T14:34:10.138Z" }, + { url = "https://files.pythonhosted.org/packages/f1/b1/6e69b8056f615e5ccff6b91ca16db2d47922251f016821a300c115267fef/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:96720f2aeed3434bc48f4d52fbad64ecc820cfed88915d664780ed9ba09ede78", size = 244507, upload-time = "2026-08-12T14:34:11.488Z" }, + { url = "https://files.pythonhosted.org/packages/0f/34/02c15d6a0aa6b934dcdc136b111da63ae857b9fd51cf5505b0736337c2eb/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:c455829625df983f716cbaecbba77f2d1dc2e0e0ed1638c059cece15a279344b", size = 240951, upload-time = "2026-08-12T14:34:12.991Z" }, + { url = "https://files.pythonhosted.org/packages/ae/15/0fe893d3e1c7d111280bd6c4bd4c1e431487a1124a1bcbce78dfeda3a3a8/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:c41b067eddcfa5ee6b1169c287605be7fb6b0ea22bba6474c5bb978a668def4f", size = 266162, upload-time = "2026-08-12T14:34:14.232Z" }, + { url = "https://files.pythonhosted.org/packages/55/ea/eca03527307670f5d102c295671a800c404ca958cf94fefd10fc963a72f0/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:e31786a947b136329bfdc458c82c06d4ec539b4a4436b7da4df4aafc9902ee80", size = 251835, upload-time = "2026-08-12T14:34:15.48Z" }, + { url = "https://files.pythonhosted.org/packages/03/a8/fee5633081e595fe9e191df6f215106c791ad596eddf5e41e39b8ea0f2e2/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:c5c6d47a865147e0ae3322ce92e7fb52ba3169d94b447deda56897ea2aa6fac9", size = 264314, upload-time = "2026-08-12T14:34:16.679Z" }, + { url = "https://files.pythonhosted.org/packages/cf/fb/17f47ae6ca35b562fb6e6f4b05f7aec6034217353eb4a23aaa3566dc7340/charset_normalizer-3.5.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:c75191e3c8052045179646cb40e280800a4e0bdfda34d9c949c2f268d44e80e4", size = 253194, upload-time = "2026-08-12T14:34:17.975Z" }, + { url = "https://files.pythonhosted.org/packages/59/88/f2b0f7ebb92493e925889ff29239b3b0073ffafd91230dbfc69e5cf9389c/charset_normalizer-3.5.0-cp315-cp315-win32.whl", hash = "sha256:83b62410bd36bb1178a7d563e2ee0cf21eb1c980c912ab99c2c78f06227f1731", size = 179800, upload-time = "2026-08-12T14:34:19.409Z" }, + { url = "https://files.pythonhosted.org/packages/e2/f0/afb5bfdea52fd943b1960403847a276b8e900c6e4cd6a38752321b4eda64/charset_normalizer-3.5.0-cp315-cp315-win_amd64.whl", hash = "sha256:e3b9eaa99a6d8c9ace4cd303915947ef55088d4cd87c6676874f98c5c03aa040", size = 203726, upload-time = "2026-08-12T14:34:20.656Z" }, + { url = "https://files.pythonhosted.org/packages/fc/71/219783eb691aa2ec879c0e521afdfe2b826f9678eed51b9c039d03e0db2b/charset_normalizer-3.5.0-cp315-cp315-win_arm64.whl", hash = "sha256:fec352b793cdc183cc9e7e0b6c10fd7bff38ec54ba44cc43599b9b56f7f3db2e", size = 183428, upload-time = "2026-08-12T14:34:21.975Z" }, + { url = "https://files.pythonhosted.org/packages/d1/d0/14aef3b9f80f2593c039d897e89034635b9eb0eb44b6ce5173bbd79ff338/charset_normalizer-3.5.0-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:c9bde7a960720c8b8e1b5ef7afaa0c9a2f3b55c44abd635b2b29dd066b298e3a", size = 368728, upload-time = "2026-08-12T14:34:23.221Z" }, + { url = "https://files.pythonhosted.org/packages/10/fc/b249466ddbbeffa448b6597631e9091d1f01b5132ff8e7a0e21a6eb72b63/charset_normalizer-3.5.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f8cd1283a9fe6c2065c807e9d5da81afe5e1e004caef39adc0d8ae86dd883698", size = 240925, upload-time = "2026-08-12T14:34:24.504Z" }, + { url = "https://files.pythonhosted.org/packages/2b/b9/c17e72aaa1b3e1ca6c184e8025cf138ed492d01a54f85286ff7d31253a4b/charset_normalizer-3.5.0-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:1c010dd86d3f4c4433c9634d33ce8147393b270dfa54f217f965540b8ae8e075", size = 234932, upload-time = "2026-08-12T14:34:25.822Z" }, + { url = "https://files.pythonhosted.org/packages/18/d7/f84ef0966bbe216f71029e34e7fa425a16b1682e2a40265e679dedf2b655/charset_normalizer-3.5.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5e68229977b2dea28e7061c0c0630a23f2f9f6e9c6fb38d77d3d6dbfe3768b74", size = 261733, upload-time = "2026-08-12T14:34:27.112Z" }, + { url = "https://files.pythonhosted.org/packages/3b/73/3e887fa0781a395339355ed934ab6561ceb5bb52574160f070224039c630/charset_normalizer-3.5.0-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a60773eb5fda796e6e6f76b9c152d270fe59f9788a51a6ff8ba44082d8548ae4", size = 258460, upload-time = "2026-08-12T14:34:28.431Z" }, + { url = "https://files.pythonhosted.org/packages/b3/81/52ebd9849bf9e35d0b21fff115cb6543162a8e1f2f564e8f87121a336b8c/charset_normalizer-3.5.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d9419f44e568f7fafcdc0b3b5c766a2364e705a9b34fb8a56b431e0d1f3f4258", size = 249894, upload-time = "2026-08-12T14:34:29.698Z" }, + { url = "https://files.pythonhosted.org/packages/85/f3/9366492b8a5fe0187de282e001d61345740cf79eb4a5f20181d769be02b5/charset_normalizer-3.5.0-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:75e243abbb528c1a774390ed71e3f868a9f37b1373442e4bbadd401cfc505ff4", size = 249540, upload-time = "2026-08-12T14:34:30.96Z" }, + { url = "https://files.pythonhosted.org/packages/0b/af/28bb5e5dbd3e67cb9196a62781ac2b6d79492f4fc7a069b6ca7d6d6c8d58/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:54c963ce6404e52255b737e8a06d356fc762d59096ae566203a67cf2b7d050f2", size = 242734, upload-time = "2026-08-12T14:34:32.481Z" }, + { url = "https://files.pythonhosted.org/packages/26/d6/7ccfa62b53b40fc06b2d3504825aa400764740bd10cf248fdc4272441b93/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:63ea0cc840c66670183578c2630d138c0e944aeadfc33f25173ee240f5db780d", size = 239580, upload-time = "2026-08-12T14:34:33.916Z" }, + { url = "https://files.pythonhosted.org/packages/0b/82/71c0c9b046697b8da66b3acefa8d5f92d00a9ef433ad7c3522b971d0369a/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:6c06875a1d4a7537bef70f659b55c6b55b9a47ec3ba8f2db610350c2d9915e6e", size = 263281, upload-time = "2026-08-12T14:34:35.333Z" }, + { url = "https://files.pythonhosted.org/packages/d6/01/d027583c869f40ba980c1c76994adbd522c360a6327e72beb44d7c267385/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:4253da1b4456b633651a8d59eb1dc7a8a8fa38241014dd7c217b353e547ae394", size = 250027, upload-time = "2026-08-12T14:34:36.991Z" }, + { url = "https://files.pythonhosted.org/packages/2a/e9/6475d739e0ec8bb1236e06263dc3affaffdf947d8114ad27024932f325da/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:f044cb1cf44012184715f46584658993b5fee9344d71c4b0c455a17a299730c0", size = 257547, upload-time = "2026-08-12T14:34:38.277Z" }, + { url = "https://files.pythonhosted.org/packages/a5/60/d1f502fcaa048a2aca3ab80bfef8407659c131e4f1792fa805fec14b4960/charset_normalizer-3.5.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:a17864853f7c518ae7d4b368af98f427f9396805476af40af8698560f09d7d97", size = 251718, upload-time = "2026-08-12T14:34:39.562Z" }, + { url = "https://files.pythonhosted.org/packages/c3/69/76343dcf4381a698807ff8a20d89f66bbdd9f6222b0b17740f77ab764335/charset_normalizer-3.5.0-cp315-cp315t-win32.whl", hash = "sha256:9e726478d7a213847860219d74665a6892a643ac93b8f76580f6cf9ed39996b7", size = 190757, upload-time = "2026-08-12T14:34:41.053Z" }, + { url = "https://files.pythonhosted.org/packages/e1/ea/d18147626a1667cc773c42104ab155a4ca5d6d4d174b7a35e01062213ea5/charset_normalizer-3.5.0-cp315-cp315t-win_amd64.whl", hash = "sha256:d7229a99120c6c2792d96f4857c2648ce5530e93667a2c2388c5ef69a6b84775", size = 215431, upload-time = "2026-08-12T14:34:42.501Z" }, + { url = "https://files.pythonhosted.org/packages/47/21/4869598aae0872d94faa5933918a4fe37ab2c5af9d095786e241f9506fed/charset_normalizer-3.5.0-cp315-cp315t-win_arm64.whl", hash = "sha256:527e28a5e751d9e11369b9c5f9ab35c748eb9c109101920c7deb40d6eadf8d03", size = 193205, upload-time = "2026-08-12T14:34:43.781Z" }, + { url = "https://files.pythonhosted.org/packages/5b/f3/7b523d807cb5e73562ef8acf21d39cdb9d704955327362c781bc3478a73d/charset_normalizer-3.5.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:5a4ee37248dfac25107c758bda99d545ce73e60b44d2dd39e4a2bb9f2831e9f5", size = 330840, upload-time = "2026-08-12T14:34:45.06Z" }, + { url = "https://files.pythonhosted.org/packages/f0/de/fc68978fe78ca97063c96d764e41ff92ca639948f319271e0ff450e577a2/charset_normalizer-3.5.0-cp37-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a864bdcacd8bff58bb4845304e031f821a3ec64b2b7259f2d409cd49c9e59ca3", size = 251862, upload-time = "2026-08-12T14:34:46.58Z" }, + { url = "https://files.pythonhosted.org/packages/a9/cb/82b41a0ab7fb1a88065f1d78ad32696ad88ea3fe8e25b8189d08833938de/charset_normalizer-3.5.0-cp37-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:84b736e3b391601bc47b86da381c749c0f894e9191aaca9f31f30c2632206df3", size = 239484, upload-time = "2026-08-12T14:34:47.869Z" }, + { url = "https://files.pythonhosted.org/packages/e8/0c/19608b631f4538f908098d4a2d56a8f79a665e27cc58e9d90479761a9227/charset_normalizer-3.5.0-cp37-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6abb1f356fb865baeb6ebc3fadd843e9a96fbf49b9adcca55037f3cceccb7438", size = 230602, upload-time = "2026-08-12T14:34:49.265Z" }, + { url = "https://files.pythonhosted.org/packages/29/db/f648eb30e14eba301aed61e11672156f137905c1bdbb530151abe8065943/charset_normalizer-3.5.0-cp37-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1d366548d2ee28a8cfdcc4296363978cc644a728333be9824d2de4652e83df0a", size = 259208, upload-time = "2026-08-12T14:34:50.632Z" }, + { url = "https://files.pythonhosted.org/packages/d3/e0/ed2c8bdbac484d69614d6993143aeb6cb0f4dd1561c883402517b623c8ef/charset_normalizer-3.5.0-cp37-abi3-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d672f329ae504ee240eb39b6effb3318aa8e7e8924c0ce8eee5760b3fad98539", size = 253659, upload-time = "2026-08-12T14:34:52.11Z" }, + { url = "https://files.pythonhosted.org/packages/32/08/b4907cb9ec5b521d9d024ced13611240b86ef065c2eb15b3ad2334dc9940/charset_normalizer-3.5.0-cp37-abi3-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c54036a518748b6c02e666f6d46c3817561998fb904c3be25b56fb4fe3dc5706", size = 248821, upload-time = "2026-08-12T14:34:53.399Z" }, + { url = "https://files.pythonhosted.org/packages/12/b2/e2d1abcfbc05822f0030869efb4e9f8a3658e13b4821796d4b62da917327/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:22a1889f1c9b752c63c36758a0c2145458e3cadb20fced7a0790002e9dd12b26", size = 240271, upload-time = "2026-08-12T14:34:55.09Z" }, + { url = "https://files.pythonhosted.org/packages/dc/f9/4ba127ad610542fa3eabfa41c45bf12d357860a815b3566374ec0188e213/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:b7eb3eab5c646d3de7dcb14a7c9caebace5249c5767da39e1761cb1576e521a3", size = 232155, upload-time = "2026-08-12T14:34:56.543Z" }, + { url = "https://files.pythonhosted.org/packages/01/68/40613182366d00bd6dbd5f6c84a926cbd120960e038a8269e9ae7d782762/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:2080aa129a28267984cdc902898993d788c995c384e285d0d19199f56760d52e", size = 259674, upload-time = "2026-08-12T14:34:57.815Z" }, + { url = "https://files.pythonhosted.org/packages/0a/53/4574a14fa4c9de4a6c9f31725354bfa40b67f653e6d594ce1654f9a41b32/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:fd68c825548a611158230e2f9222e210ceb2e3391995c0aa5865cbdf3ab4bd49", size = 246122, upload-time = "2026-08-12T14:34:59.337Z" }, + { url = "https://files.pythonhosted.org/packages/5a/02/bd8030d13d92c058ca7b2b9615bbb3169569e144db64d65c149cd45abf5e/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_s390x.whl", hash = "sha256:d8a9316f4da85e937242642b537c6d55d7e9287dd38e5634732f8233932aff45", size = 255221, upload-time = "2026-08-12T14:35:00.71Z" }, + { url = "https://files.pythonhosted.org/packages/77/9d/10ecd3bcbe2666b3d4d4026c97b48f73990682815db516052a1e8f4a31c5/charset_normalizer-3.5.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:d90254c8f609338c53ec180fcd4c4f9c16502e238e3fc88ca7fd4c2f38d445b8", size = 253450, upload-time = "2026-08-12T14:35:02.217Z" }, + { url = "https://files.pythonhosted.org/packages/e4/0f/d044c4872c0938a84f87b5027a698c0e61bacfc5c3551a4e749ca9b7bc5c/charset_normalizer-3.5.0-cp37-abi3-win32.whl", hash = "sha256:8b3e9e29b8b07cc461b9ce7768db7693a93979d0dadf22046f6f3555ded2f516", size = 173594, upload-time = "2026-08-12T14:35:03.845Z" }, + { url = "https://files.pythonhosted.org/packages/10/6b/6046773901f1944b9a89436351529811ee958afc7b774563be9d74a6f0c3/charset_normalizer-3.5.0-cp37-abi3-win_amd64.whl", hash = "sha256:0c8953d9d1617794cfc40d81179571c9ba3805dd029623a15c93f1fb70e60a74", size = 198959, upload-time = "2026-08-12T14:35:05.187Z" }, + { url = "https://files.pythonhosted.org/packages/ab/a6/b57708ac92aefc8e8389d51d5178129b81f03196da61ee2c23e687b8178a/charset_normalizer-3.5.0-cp37-abi3-win_arm64.whl", hash = "sha256:562d24ca7797c1af8852994950c2e623a907b201fc4b0ed29e92af173d3828ca", size = 267055, upload-time = "2026-08-12T14:35:06.533Z" }, + { url = "https://files.pythonhosted.org/packages/22/c7/754d09943a616937df61e4ba367c409ded2a987e872972098d51a6fcf73b/charset_normalizer-3.5.0-py3-none-any.whl", hash = "sha256:993dfcbe75a85a3784abb5084f2c41b915767c90546fcc92803cffa28611baea", size = 67943, upload-time = "2026-08-12T14:35:30.363Z" }, +] + +[[package]] +name = "click" +version = "8.4.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/76/d4/81420972a676e8ffea40450d8c8c92943e7218a78fe9b64359836cc9876b/click-8.4.2.tar.gz", hash = "sha256:9a6cea6e60b17ebe0a44c5cc636d94f09bd66142c1cd7d8b4cd731c4917a15f6", size = 338000, upload-time = "2026-06-24T17:45:15.148Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fb/e2/79c688af8b210d232694e31e59da9f6ec747bae31c3f5946e4e9b98860d5/click-8.4.2-py3-none-any.whl", hash = "sha256:e6f9f66136c816745b9d65817da91d61d957fb16e02e4dcd0552553c5a197b76", size = 119243, upload-time = "2026-06-24T17:45:13.73Z" }, +] + +[[package]] +name = "cobble" +version = "0.1.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/54/7a/a507c709be2c96e1bb6102eb7b7f4026c5e5e223ef7d745a17d239e9d844/cobble-0.1.4.tar.gz", hash = "sha256:de38be1539992c8a06e569630717c485a5f91be2192c461ea2b220607dfa78aa", size = 3805, upload-time = "2024-06-01T18:11:09.528Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d5/e1/3714a2f371985215c219c2a70953d38e3eed81ef165aed061d21de0e998b/cobble-0.1.4-py3-none-any.whl", hash = "sha256:36c91b1655e599fd428e2b95fdd5f0da1ca2e9f1abb0bc871dec21a0e78a2b44", size = 3984, upload-time = "2024-06-01T18:11:07.911Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "cryptography" +version = "50.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "platform_python_implementation != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/41/6cbdcf9142d00fe82836fbb51e503e58088575cf7a0fe1dbff6695bf0840/cryptography-50.0.0.tar.gz", hash = "sha256:eeac2acb5a20ed25e0ad6d1df9891a520b78b404266b6d11778f25d5d691a6c9", size = 880201, upload-time = "2026-07-31T14:25:10.11Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c5/5c/59086b4aac5e879d38ddbcf74e4be7ade89cebc3eb199a55da998c3bb46a/cryptography-50.0.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:031e2d5dd4bb9caa3ca9c82e5a197fd8ae680232cee62603d1a813f3f07e3d03", size = 4001252, upload-time = "2026-07-31T14:23:33.331Z" }, + { url = "https://files.pythonhosted.org/packages/57/ef/8f2df13c7216bcad3e1c74e07f6e193d93e998e114f524a53877c9af27ad/cryptography-50.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:fd9192b7b70c573d7f214eb1ae35e00d359f6f5e4b27c7e21e30de1fc6204645", size = 4719554, upload-time = "2026-07-31T14:23:35.611Z" }, + { url = "https://files.pythonhosted.org/packages/d9/41/029086c34d91052fc3b88bcc8056f709a7c915c7a23b235a54eb800b1c97/cryptography-50.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:06a32a980526a6ab9a4b9bf8f7385800791e2bb960903cb6b530e4817509a3b7", size = 4702130, upload-time = "2026-07-31T14:23:37.635Z" }, + { url = "https://files.pythonhosted.org/packages/7d/ff/b6ce0954962e7f7b969f850a883744197bb3910bdfd7b6da162eab7d9f68/cryptography-50.0.0-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:a1b30560f2acc95aa8b2e06e716a13dbfc97314747b80d9707e307f77b40d6b3", size = 4725244, upload-time = "2026-07-31T14:23:39.471Z" }, + { url = "https://files.pythonhosted.org/packages/06/1e/63a1027cb7fec360a182208e1b7767d5aa1fe57be3d6aa856e69a321edc0/cryptography-50.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:8d89f3976b10b4ce31118de72329025f70d2c6ead14a8217c5514dd2c6d5a78f", size = 5342265, upload-time = "2026-07-31T14:23:41.286Z" }, + { url = "https://files.pythonhosted.org/packages/6b/72/a1116d683a6d7ece94590013882515de087edf9ef0e6292aae615a44df73/cryptography-50.0.0-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:b42a28c1844fd9de8f3f7d540e36b66f3a9c83fceac7170ebc7a6a19edd9dcae", size = 4734609, upload-time = "2026-07-31T14:23:43.139Z" }, + { url = "https://files.pythonhosted.org/packages/15/37/36a9c479bbe49acea2636c7fd3360d20f7b7e079c300352011c44850b181/cryptography-50.0.0-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:900131fafd8aead39ac7dd3a7e833be754c17a95cfd91221636949fe4eb0aa8a", size = 4356517, upload-time = "2026-07-31T14:23:44.939Z" }, + { url = "https://files.pythonhosted.org/packages/32/98/8a151d64367204cbc63ec65d37502f1d9c53cf4bfc6ec3c532614dbec60d/cryptography-50.0.0-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:07949c449a1abcf60d1ee6e88956d89404c7df3c8258f46589e912988e551987", size = 4724529, upload-time = "2026-07-31T14:23:46.93Z" }, + { url = "https://files.pythonhosted.org/packages/22/f6/ec13b470172126464a86bf54d2294a46d29837fc51ba3e45d4047946fb5e/cryptography-50.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:f89831ef99dd7dd169ab06d63a831adb9e20a87aac6d380266bbda5823349169", size = 5299852, upload-time = "2026-07-31T14:23:48.851Z" }, + { url = "https://files.pythonhosted.org/packages/da/3a/f05e32c99d440c9bb891ea0e36c9091891e36be5a9a87ab2ee6ea20729f6/cryptography-50.0.0-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:82148ec5bddac30b51a5b3c1945075f896fa022cb93f8e4a01e9f6ee95292c5f", size = 4734462, upload-time = "2026-07-31T14:23:50.861Z" }, + { url = "https://files.pythonhosted.org/packages/ca/dc/bd72b26be8953f80625f63151efd38eee71c76ca6cf591c08ff34615a79e/cryptography-50.0.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1489e263a8048bb8b6a8bac662eb2d402ea5d2b7b4699b72f385f1e2772db105", size = 4852708, upload-time = "2026-07-31T14:23:52.715Z" }, + { url = "https://files.pythonhosted.org/packages/27/20/c930314a2ab476d15dec966ec87e2e9637bb02b06106b12c0396c57bb603/cryptography-50.0.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:7cec5b856506da6defb290f30c9ee687d5f5e8cb0bd3f6459dde43b0b4fa40ef", size = 5004179, upload-time = "2026-07-31T14:23:54.887Z" }, + { url = "https://files.pythonhosted.org/packages/32/2e/c9db68a0c4bfa28e310707527c0ee3a2bd254104d2e02e68f368e197aa4c/cryptography-50.0.0-cp311-abi3-win_amd64.whl", hash = "sha256:bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30", size = 3840395, upload-time = "2026-07-31T14:23:56.677Z" }, + { url = "https://files.pythonhosted.org/packages/c3/fb/951032a3bf22a5697c83183fb6294a4843772947a70e616c57b3ff5f522e/cryptography-50.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:49e7d93abdbd2990caced757e5fade25302f719c3c8fb6e6fff2dde98999fc41", size = 3989258, upload-time = "2026-07-31T14:23:58.881Z" }, + { url = "https://files.pythonhosted.org/packages/d4/67/91eb047e69c5e845f2f14b8a2e4a1aab0f283cb885531e9e22c8adb176bc/cryptography-50.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:19736989797678c6af1e55cd49055cdbcb55d8f6b5583ac5335f933aba9101dc", size = 4700648, upload-time = "2026-07-31T14:24:00.702Z" }, + { url = "https://files.pythonhosted.org/packages/30/82/85f0f7425c856b9f96459411eb12e74ef72df9caf6f8f15bf23a33ff131f/cryptography-50.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:80b63928fa35083b33966ce1efb70e5b9607181e49dcd1c22c8c005e319f667f", size = 4682442, upload-time = "2026-07-31T14:24:02.538Z" }, + { url = "https://files.pythonhosted.org/packages/1a/28/b555a365adff1cca2fbe7b9e487d68a40de6bc67ff2cb587473eb43de0e7/cryptography-50.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:d58c3db7cd6eed54e6c06744db55456b65ebd7492ddeae9c1e93cfca7aa857d3", size = 4707596, upload-time = "2026-07-31T14:24:04.394Z" }, + { url = "https://files.pythonhosted.org/packages/72/d8/f52538140cc719df62a01cf87d1c7142318d235817109d6f4054d7c352d6/cryptography-50.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:df2a58a472f332225671c35b0a830208b86d004f82baa8530fa3782c85646533", size = 5314552, upload-time = "2026-07-31T14:24:06.31Z" }, + { url = "https://files.pythonhosted.org/packages/38/14/6120e5bd7c5aa022ad15424ba4d5c5269d0d9448ed4d55e492ea91e3c1c4/cryptography-50.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:11b74db56cdbe3cdee6e3f6982ecb70334fa10dce99ed58bf7894aaaa3b2a037", size = 4717113, upload-time = "2026-07-31T14:24:08.349Z" }, + { url = "https://files.pythonhosted.org/packages/fa/71/190bf38c3ee2e0f8efc9860ae100c9df4169742eef274b91e7aa1cb133b9/cryptography-50.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f59e38625469987d7ef6d495323c55e7db6c212eaf6112267e0d3b565a2e9c9f", size = 4338580, upload-time = "2026-07-31T14:24:10.227Z" }, + { url = "https://files.pythonhosted.org/packages/3a/63/504ccfbbe61fd8aa983f7f146399cdf034c72c2fc55f5b2dfdcdcdb20c99/cryptography-50.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:ecfed7367f965a0328cfbdd70da860f15441f002f613185668c6e6ebf5a0ac11", size = 4707038, upload-time = "2026-07-31T14:24:12.169Z" }, + { url = "https://files.pythonhosted.org/packages/01/77/2cf79bbfc4d12ca106437a6e170d6aaa01a373e93093118aaaef0e801bd4/cryptography-50.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:9aa87839c383bdbab6ef865787a1fb877af8dd03464c4400322726feaaadfc6d", size = 5273110, upload-time = "2026-07-31T14:24:14.38Z" }, + { url = "https://files.pythonhosted.org/packages/e5/45/8aae2972c520145377ea3559a605a899bebe227bf070b33cdb445929a9b9/cryptography-50.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:6ba6a53445bd3cfa809ef3ef5f1589aa6ba08784a1d962bf47d0940e871dab1c", size = 4716439, upload-time = "2026-07-31T14:24:16.415Z" }, + { url = "https://files.pythonhosted.org/packages/7b/20/4fe50b619a48c2525cc46e2dbc1ac490708d704be5d467bdaac6dc955682/cryptography-50.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3f5735ffe4996d28b809371756219f5354864902a3b9e7c0b9ee87041209fc9c", size = 4837383, upload-time = "2026-07-31T14:24:18.553Z" }, + { url = "https://files.pythonhosted.org/packages/92/91/3a31366e183343d3703f8995c095f5734676bd6938118047e50fcf279eb4/cryptography-50.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:1b4a266766514614f8aa60416e71f2fc6e575d36e7bdc90f644fadb2f4b75b95", size = 4985772, upload-time = "2026-07-31T14:24:20.385Z" }, + { url = "https://files.pythonhosted.org/packages/74/9a/02ffe35b2853d121689871eb5dce862092562b3a1ed5cc98f1aaed441506/cryptography-50.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:12b9c6996425c76ea6c457ace4f3073e715b8c545add07cd1a8f3a4f90691269", size = 3816291, upload-time = "2026-07-31T14:24:22.125Z" }, + { url = "https://files.pythonhosted.org/packages/03/37/73d005be173aff344af30e9fd2a576575cb2391a7101d9cd3842e1fa8cce/cryptography-50.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:ccdc4a71a4dabae05de219404f9f4abc38e3b58422177ff93d0da05967dafa07", size = 4036009, upload-time = "2026-07-31T14:24:24.122Z" }, + { url = "https://files.pythonhosted.org/packages/ff/c6/7a6202a534e32103a285b7834a120869557fe198d51d7cfe59754c8bda9c/cryptography-50.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:910e1d2668e7de9648f2bcee30e180db2a6b15c30f887d7c4c93ddf96e3992e3", size = 4745252, upload-time = "2026-07-31T14:24:26.118Z" }, + { url = "https://files.pythonhosted.org/packages/85/4f/0fa8c2f4428198f15d9ff8d63400e27afbf94ce833f6108da1eb3753f945/cryptography-50.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a91296cb61e8df6f86d0c19cc4068228da256bf59bf86049fbd821084565327f", size = 4728939, upload-time = "2026-07-31T14:24:27.994Z" }, + { url = "https://files.pythonhosted.org/packages/d1/63/54dd723490ba2dc09b299682c10b38db38f159728bcaae8c591b8af2f22d/cryptography-50.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:e722f16708d854fe924790e051061f6704a472c3bac347b6fd88033ea8dd0dc5", size = 4748483, upload-time = "2026-07-31T14:24:30.254Z" }, + { url = "https://files.pythonhosted.org/packages/1d/dd/7c77d26285cc7f6991efce64a0f5b4f9383bfa5dd8c5033003eaf7db4cdb/cryptography-50.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:d764dcf130c428ef66786f866dd750f53182bc608813489915e9fc106bb0c82f", size = 5367599, upload-time = "2026-07-31T14:24:32.457Z" }, + { url = "https://files.pythonhosted.org/packages/46/c9/f60aed34c013f317f92817b6c171c2d22a78270fa41109bd4b08af26b194/cryptography-50.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:105110f43a471dbd0060b9c9516cb8a6a79233631a04cc2ba16f28323ac6e025", size = 4762647, upload-time = "2026-07-31T14:24:34.599Z" }, + { url = "https://files.pythonhosted.org/packages/be/f3/f9a0173b139372c3a48ed98154b45cc6b9de17c789d5ab552e621c293609/cryptography-50.0.0-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:828743d939e9629bc267b8e2d08d8bb67cd4319c771a33d4b18b22dd8fb7440a", size = 4385197, upload-time = "2026-07-31T14:24:36.647Z" }, + { url = "https://files.pythonhosted.org/packages/d8/36/83bb81f6e569bc38e1e4a7bc80f29b46bb9601920bc455fc8e888f5d5742/cryptography-50.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:2a8183b489dc1f7f80f135780fadc1108f14b31b8a40411c7a5b17425f65f28b", size = 4748095, upload-time = "2026-07-31T14:24:39.493Z" }, + { url = "https://files.pythonhosted.org/packages/6b/16/d3008eff98c764979865834c3d386d4fd041b5f52e7f34fc29ac1a5eb515/cryptography-50.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:6e7d61120573a7f2cd94cc095f9e81f6967c61ccdf194285aa143ecec8e0b708", size = 5325948, upload-time = "2026-07-31T14:24:41.556Z" }, + { url = "https://files.pythonhosted.org/packages/9c/f8/d97f9603efda3888187bfdb893f26c41be4735c10631d05d284ee6b047c4/cryptography-50.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:37fdb0d0111f1e2ff07139dfb79f1b49531f8e213c46f1163dd7642979b58c47", size = 4762400, upload-time = "2026-07-31T14:24:43.636Z" }, + { url = "https://files.pythonhosted.org/packages/64/a2/4615c8f7d81a00b1d6e6afe19f694e1543582349fb5f4076f6cb5dc36485/cryptography-50.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:c87f62a3d3b9888ed0fdde100ec06aa61ca9cd44bad9057d1dff9a516b5f5bb9", size = 4878208, upload-time = "2026-07-31T14:24:45.522Z" }, + { url = "https://files.pythonhosted.org/packages/d2/1a/efcfb02f91407149a0dacffffab791f7e19bf6385f63b3666dc8b5e5c9c8/cryptography-50.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:65c2c3add92b45fd0709db8594536aea39c2a67af0e27ffcf049c498501140b7", size = 5037050, upload-time = "2026-07-31T14:24:47.697Z" }, + { url = "https://files.pythonhosted.org/packages/57/30/4a22984d4f1bdfb8c054f07a92bc176b97a3134cc1d6c4b3bffb1f3688b4/cryptography-50.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:d24fead1d4d076e1bfb006dcec392074a3cd8d7b4fc8a595aa64073b2b7a96ba", size = 3874135, upload-time = "2026-07-31T14:24:50.085Z" }, + { url = "https://files.pythonhosted.org/packages/9d/3e/e54cde8c01631a5a8226ccd617eab9e57fd5cfdad90f1a9e6bb570794631/cryptography-50.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:5e34edd123674534acd70147f0ca331eaa2c74e6325fb2028c886aa26ba0b68c", size = 3963170, upload-time = "2026-07-31T14:24:51.968Z" }, + { url = "https://files.pythonhosted.org/packages/01/b6/0b9e125e90f3d2dcf599a218a899cda7326a3158cfa258723f0b398b08f6/cryptography-50.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:8eb5e1172eb569ea8a872796576e6a67c276351728b6455d5beb01242b027c6a", size = 4692441, upload-time = "2026-07-31T14:24:53.743Z" }, + { url = "https://files.pythonhosted.org/packages/53/c9/a5151588710785a96d7bc4de27d4cd62f263bbbcb203cfe29df537eb6505/cryptography-50.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:910d11e1a385c654bf738bf3e6b8e6ed5de0f5610fcae2be9e5b398d8081d20e", size = 4699810, upload-time = "2026-07-31T14:24:55.746Z" }, + { url = "https://files.pythonhosted.org/packages/c7/1a/15b92b25eb6ce3089cd49377ae990a0f3ad485a510f968aed1f19dbdcdf2/cryptography-50.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:62598a8a57f815db4c6259a4e97d857dab56697e7de8e8ab02352ab74da1995d", size = 4691924, upload-time = "2026-07-31T14:24:58.082Z" }, + { url = "https://files.pythonhosted.org/packages/62/15/219075012ab13e8905f3cd572204f4acb4b111df787104346b9bc0cea789/cryptography-50.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:07479a1cb08219ab719147e742e76090c9c773321959bb94946fffdd397a6437", size = 4699593, upload-time = "2026-07-31T14:24:59.951Z" }, + { url = "https://files.pythonhosted.org/packages/8e/b5/c2c5fce26f0ee40d21bafe7f191d29a34b35a65ac4fe8a1191d1983612e9/cryptography-50.0.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:c99c003e088647b8a5b7c145d6f78c335f6348332b62e142d411c4b63d1460b9", size = 3813796, upload-time = "2026-07-31T14:25:02.298Z" }, +] + +[[package]] +name = "distro" +version = "1.9.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fc/f8/98eea607f65de6527f8a2e8885fc8015d3e6f5775df186e443e0964a11c3/distro-1.9.0.tar.gz", hash = "sha256:2fa77c6fd8940f116ee1d6b94a2f90b13b5ea8d019b98bc8bafdcabcdd9bdbed", size = 60722, upload-time = "2023-12-24T09:54:32.31Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" }, +] + +[[package]] +name = "et-xmlfile" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d3/38/af70d7ab1ae9d4da450eeec1fa3918940a5fafb9055e934af8d6eb0c2313/et_xmlfile-2.0.0.tar.gz", hash = "sha256:dab3f4764309081ce75662649be815c4c9081e88f0837825f90fd28317d4da54", size = 17234, upload-time = "2024-10-25T17:25:40.039Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/8b/5fe2cc11fee489817272089c4203e679c63b570a5aaeb18d852ae3cbba6a/et_xmlfile-2.0.0-py3-none-any.whl", hash = "sha256:7a91720bc756843502c3b7504c77b8fe44217c85c537d85037f0f536151b2caa", size = 18059, upload-time = "2024-10-25T17:25:39.051Z" }, +] + +[[package]] +name = "filelock" +version = "3.32.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/64/a02e6765de08964ed371eca577870593245afc9dfac16d037de7c10d18e6/filelock-3.32.3.tar.gz", hash = "sha256:0ffa185a3540854c95caa7fa76b76cb219d907415e2c5dc9af25fd970563487f", size = 218135, upload-time = "2026-08-13T16:00:05.577Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/8e/50f46a9c0ce8d2861a394c1347caae037ea0431d2f67d7feb151cbc4649a/filelock-3.32.3-py3-none-any.whl", hash = "sha256:7f0ca4bcc0e181c60dbbd8aa9ab5b120ebb99e4e064e83636340056f833a1f09", size = 98901, upload-time = "2026-08-13T16:00:03.974Z" }, +] + +[[package]] +name = "fsspec" +version = "2026.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/00/78/f34251dadb8f3921264a1d9b8946f5e542014ee2614b285261b4e40e6775/fsspec-2026.7.0.tar.gz", hash = "sha256:c803c40f4cf860b49dea58ee3e1c33cb9c790520e233537e1340049f89b82a88", size = 317040, upload-time = "2026-07-28T16:34:51.052Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fd/3c/6a2bf344106328fd04963664a60b9bb6496fc25df8e962fcdc1367285fb9/fsspec-2026.7.0-py3-none-any.whl", hash = "sha256:b57ddbafedfaef7018c1ecab32aa200a9d7ca26b77965f64e48b70061249d279", size = 206583, upload-time = "2026-07-28T16:34:49.538Z" }, +] + +[[package]] +name = "h11" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" }, +] + +[[package]] +name = "hf-xet" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1b/ab/522a2ab67f27971a9d48ca666d4fca85ef7d5282d142e31fd087e27b1bbe/hf_xet-1.6.0.tar.gz", hash = "sha256:2e58454a340b3556dfa4972d5451aff4fba8dd42a236600ba1a1d2b1514f0fef", size = 920527, upload-time = "2026-08-03T22:33:13.243Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/41/62/3c062f593bd92ef4e77a0ef39541e3d82a0a1d3947c8a777a02a13a27828/hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:70cbb9c896901600128cb9b6f06e132954fbede1db30f31f7c6c63f84cb7c31d", size = 4074584, upload-time = "2026-08-03T22:32:47.364Z" }, + { url = "https://files.pythonhosted.org/packages/bb/1e/c0ad437dd267a8e435bef594acf781bbc3874ff0b6435b4962d03ecf7cc4/hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:23379c2f9ec8696d952b16414a2bae72cad86a52df869b050698ba60f538c675", size = 3867381, upload-time = "2026-08-03T22:32:49.049Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ee/7c0d7b6ab336167531b1c30af2af003f054af4c749becbd7209ae33a77c3/hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f2f7278c05c22fd60cb436cda1269649b3e81db65ecdc8496e5e164aa4143e7b", size = 4453982, upload-time = "2026-08-03T22:32:50.568Z" }, + { url = "https://files.pythonhosted.org/packages/63/06/ad8eab1c9525246650cbaa821caa3cdbaca734ab1a5b8c91bea09cbd8d69/hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:948f15d3a9545cfe5932f6bd8b440f6ae630aee108f14b7bd6c561f7c2dcc522", size = 4249445, upload-time = "2026-08-03T22:32:52.391Z" }, + { url = "https://files.pythonhosted.org/packages/d8/26/1eee8aedb0dafc1ab9717dc9ac602cde33361b232dc06803f1f6ed18b58c/hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5153e6bb103ad49d6ea9f1b2e230db5a2ea32551ad09a706d2f61d7c7c80d80e", size = 4451099, upload-time = "2026-08-03T22:32:54.114Z" }, + { url = "https://files.pythonhosted.org/packages/67/57/0b88af1f194ab6c9c650547d9cc06bfeaab836ae4dcdb331676bfb8be95a/hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:35cec30d75c6f9eb9c16a77cef68e85a103b72e24d4b473714ec9ff06428bab9", size = 4664712, upload-time = "2026-08-03T22:32:55.547Z" }, + { url = "https://files.pythonhosted.org/packages/53/a0/26b717a9d1840e8abf48dcec64b5ed8fbe472671d38ad28d30e147132b33/hf_xet-1.6.0-cp314-cp314t-win_amd64.whl", hash = "sha256:5789835d7c6bc9436962853192082374297fb72d7eff7e7762ec25ceb7e25338", size = 4025906, upload-time = "2026-08-03T22:32:57.391Z" }, + { url = "https://files.pythonhosted.org/packages/49/f6/4a9966633c6fef83af997e2cff68ec1963676d412bdfd096df2a93b8e185/hf_xet-1.6.0-cp314-cp314t-win_arm64.whl", hash = "sha256:75765820ce4700db3750c94acc8fe27c5fae4c9ec000a0dbac3ca082acf97765", size = 3849221, upload-time = "2026-08-03T22:32:59.123Z" }, + { url = "https://files.pythonhosted.org/packages/a2/50/7afa2c9c787405864fc47a0d1bbc02c62e9101947ed43c1f43899fc7d91d/hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl", hash = "sha256:633dc0cd71d32da58ab8c03ad38e2fac452c15c2b0a2866ebf6ededfe0a5061d", size = 4071729, upload-time = "2026-08-03T22:33:00.721Z" }, + { url = "https://files.pythonhosted.org/packages/4b/69/55b8dcf636142ae660fec1869fcac14c4da2e8412e14d6eee1523be77e9f/hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:f0906082d9932ae0c0057fa194041c22b4e2cdb46b2592ef3b91f020d62a081a", size = 3876287, upload-time = "2026-08-03T22:33:02.251Z" }, + { url = "https://files.pythonhosted.org/packages/67/4e/a28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08/hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d62671bb130879cef0ee4c9ebe47a14af6c66ec53e6d84dc15936e5ffdfac82f", size = 4464663, upload-time = "2026-08-03T22:33:03.802Z" }, + { url = "https://files.pythonhosted.org/packages/9a/69/1f0cbc2fb22ae6082d094f743d1b8945a3f36f6089cb95f42b7ee348cda7/hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:0e6e21fa3cdfcdcd76748564bf593870a5e013f47d97cf10aed63aa222cff5b7", size = 4262538, upload-time = "2026-08-03T22:33:05.287Z" }, + { url = "https://files.pythonhosted.org/packages/d1/3a/4f4f2301ade26e404462d3336fa11f7958d914cabbabdd6e03c3c5d5658c/hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:4fc74352a17015bd0ee90038bc9efe38db894cde45f268b6712b04fce8cd0acb", size = 4460520, upload-time = "2026-08-03T22:33:06.81Z" }, + { url = "https://files.pythonhosted.org/packages/ab/5f/311725e2a905534dfee2dcb5b08414f249147f1f12252bfc2bd24caa075c/hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8fb4f71cba6129110c3374a33f919001ff130488fc23553698e34cc1c2a1198c", size = 4675937, upload-time = "2026-08-03T22:33:08.616Z" }, + { url = "https://files.pythonhosted.org/packages/98/b7/8c59a66d15205024662f1d66968136f13893f96df1ddc5087e2e281fc95f/hf_xet-1.6.0-cp38-abi3-win_amd64.whl", hash = "sha256:fb4fadde1b2b70bf4c0c14a6dccbe7194b1c28947fefd5bbe3fed9d940676c3b", size = 4033128, upload-time = "2026-08-03T22:33:10.171Z" }, + { url = "https://files.pythonhosted.org/packages/73/63/ca511b6f802f28cf3489b280fe77475bcca8de85e81a6299d7916b5b5555/hf_xet-1.6.0-cp38-abi3-win_arm64.whl", hash = "sha256:3dc3e35441ba395006af5aaacc40ef2e603c51ef46c3530b9156185f00935ea3", size = 3859359, upload-time = "2026-08-03T22:33:11.725Z" }, +] + +[[package]] +name = "httpcore" +version = "1.0.9" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "h11" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" }, +] + +[[package]] +name = "httpcore2" +version = "2.10.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11" }, + { name = "truststore" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a9/83/a896fc59940fc5a6e2aff3a4be1d92fa890112936803b331cae75a993c34/httpcore2-2.10.0.tar.gz", hash = "sha256:13c0cc3d1919d4f28457f60cd2c2abe04113a8af184ccf1142811beba936f9dc", size = 67427, upload-time = "2026-08-09T09:11:32.123Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e5/4f/d149104195a35e2853a2fc203a8e3477747e58c80e17dda686dace174383/httpcore2-2.10.0-py3-none-any.whl", hash = "sha256:7df06cfb34070cae4f7c89be69dc1095eca138e9704ceffb98d25c1912ab6f01", size = 83000, upload-time = "2026-08-09T09:11:29.555Z" }, +] + +[[package]] +name = "httpx" +version = "0.28.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio" }, + { name = "certifi" }, + { name = "httpcore" }, + { name = "idna" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" }, +] + +[[package]] +name = "httpx2" +version = "2.10.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "sys_platform != 'emscripten'" }, + { name = "httpcore2", marker = "sys_platform != 'emscripten'" }, + { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" }, + { name = "idna" }, + { name = "truststore", marker = "sys_platform != 'emscripten'" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/bd/3d/f9a8c07a3884f3e5b26205e8436a18b3af61c5d53192c3bea235574dbbec/httpx2-2.10.0.tar.gz", hash = "sha256:8741d7329fe2c7885fc9ceb61c8217acfb87a85f75723714b89ebf7ad7196338", size = 98749, upload-time = "2026-08-09T09:11:33.24Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b9/6d/a637d52449d98a6892d9a4dc0262587afdb6a66f201871842dce5a97b1c1/httpx2-2.10.0-py3-none-any.whl", hash = "sha256:5e3194a432701e1cc6f69a8b1b2fa199ef907013fede8d9a09a2c5b7b8141a18", size = 94355, upload-time = "2026-08-09T09:11:30.882Z" }, +] + +[[package]] +name = "httpx2-jsfetch" +version = "1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" }, +] + +[[package]] +name = "huggingface-hub" +version = "1.27.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "click" }, + { name = "filelock" }, + { name = "fsspec" }, + { name = "hf-xet", marker = "platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'" }, + { name = "httpx" }, + { name = "packaging" }, + { name = "pyyaml" }, + { name = "tqdm" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/3e/9b/ddf3d02a8681f1b9ce52fda03d755dad6b74c4f8172304c4c8d2975450f9/huggingface_hub-1.27.0.tar.gz", hash = "sha256:c1fed40ea82a6b41b477f5243546549b792ae0a93abcea608cff66089bf8f8df", size = 942668, upload-time = "2026-08-07T12:48:05.161Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/de/d8/95b735e183957c1f26d94c52977f09d466d55119cbbc1558ea4975e4c216/huggingface_hub-1.27.0-py3-none-any.whl", hash = "sha256:7df6827c2f956c60fbaa64646e979e566db76f619dd0a9729dfb8c5a3eb4f68d", size = 784926, upload-time = "2026-08-07T12:48:02.905Z" }, +] + +[[package]] +name = "idna" +version = "3.18" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + +[[package]] +name = "jiter" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1d/1f/10936e16d8860c70698a1aa939a46aa0224813b782bce4e000e637da0b2d/jiter-0.16.0.tar.gz", hash = "sha256:7b24c3492c5f4f84a37946ad9cf504910cf6a782d6a4e0689b6673c5894b4a1c", size = 176431, upload-time = "2026-06-29T13:05:13.657Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4e/3f/fae6cc967d120ec89e31c5418a51176d8278b3087fbb384a9176754f353c/jiter-0.16.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:67fddeda1688f0cce2d2ae83ccf8a80f79936f2d2997d6cc2261f82fdb54a4d3", size = 309289, upload-time = "2026-06-29T13:02:52.301Z" }, + { url = "https://files.pythonhosted.org/packages/c8/e3/97c6c3562c077f6247d6e6ce5c82562500b6316c0d928e97e106b7a1321a/jiter-0.16.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c90c0f63df322be920eda6ce622e3083d8906ba267f8220fe7873213b8b4430e", size = 315181, upload-time = "2026-06-29T13:02:53.964Z" }, + { url = "https://files.pythonhosted.org/packages/7b/89/d8d073f8aa2667e46c6c0873f86fe4a512bba4293cc730f626a076211a62/jiter-0.16.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:64c0203212098470032aabcde9356fc168f377aade3e43def61dfe17e92f2037", size = 340939, upload-time = "2026-06-29T13:02:55.412Z" }, + { url = "https://files.pythonhosted.org/packages/87/c9/db4fda3ed73fb864139305e935e5b8b38a5a24692a5a9dd356c22f1b9c8d/jiter-0.16.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:12288303c9844e61e1651d02a9a6f6633e47d39f897d6991d1427161ce6b746e", size = 364932, upload-time = "2026-06-29T13:02:57.28Z" }, + { url = "https://files.pythonhosted.org/packages/a2/74/52b5e86241057f52ddd7c9a580f90effb51f9d06239f6fc612279b91a838/jiter-0.16.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5cf109d010b4b05a105afb3d43be36a21322d345ad3111e13d15f680afef0e5b", size = 461132, upload-time = "2026-06-29T13:02:58.994Z" }, + { url = "https://files.pythonhosted.org/packages/a9/87/544a700f7447c1f31c5d7833821a4daa5683165c2d5a094fbf5b5800c3dc/jiter-0.16.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:62c1b7fe1f77925acf5af68b6140b8810fa87dfd4dc0a9c8568ec2fa2a10429c", size = 374857, upload-time = "2026-06-29T13:03:00.455Z" }, + { url = "https://files.pythonhosted.org/packages/40/cd/0fcc3f7d39183674d5bfa9ec640faaeb506c60be7c8f94625dfba366e37c/jiter-0.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8597d23c87f59294f83bcb6229b9ed1fccee13dbba967b46930d2f1759466fee", size = 347053, upload-time = "2026-06-29T13:03:02.045Z" }, + { url = "https://files.pythonhosted.org/packages/5c/ae/c7e64e7932ad597fa395b61440b249ada6366716e25c6e08dd2afbd021e6/jiter-0.16.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:3126a5dbad56401989ac769aca0cb56005bfb3e2366eea0ca99d1a91c3c1ee03", size = 356153, upload-time = "2026-06-29T13:03:03.706Z" }, + { url = "https://files.pythonhosted.org/packages/d4/1c/1c719044f14da814e1a060191ab19b96f3e99207bc5b4bfc6d6be34b3f80/jiter-0.16.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c4b4717bdb35ae456f831a6b08d01880fff399887a6bbc526a583a406e484eea", size = 393956, upload-time = "2026-06-29T13:03:05.165Z" }, + { url = "https://files.pythonhosted.org/packages/3b/dc/7b2f303a2847207e265503853a2d964a55354cffd62a5f2936c155486798/jiter-0.16.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:adff21bc78edfe086c15eb495b900306076de378dc2337c132401fc39bd79c91", size = 521081, upload-time = "2026-06-29T13:03:06.886Z" }, + { url = "https://files.pythonhosted.org/packages/c2/5f/501cf6e1e09caeb420195179ffc6f62aca603f1220ec53fd80d0d70b3e56/jiter-0.16.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:dab907db06fc593645e73109acf4581ba5b548897d28b9348dc41ddc8343b2d3", size = 552085, upload-time = "2026-06-29T13:03:08.339Z" }, + { url = "https://files.pythonhosted.org/packages/79/54/aa5be86520113b79455c3877f3d1f07a348098df4083ba3688e9537e52dd/jiter-0.16.0-cp311-cp311-win32.whl", hash = "sha256:560b2cf3fb03240cd34f27409a238547488708f05b7c3924f571a60422251ec7", size = 206755, upload-time = "2026-06-29T13:03:09.653Z" }, + { url = "https://files.pythonhosted.org/packages/64/ec/2feb893eb330bd69b413866f4d5daada33c3962f1c6f270c91ca2d87fdf9/jiter-0.16.0-cp311-cp311-win_amd64.whl", hash = "sha256:e431cfc9caf44c1d5459ff77d4e64cbf85fddb6a35dad836a15c6a9ec23087c1", size = 199155, upload-time = "2026-06-29T13:03:10.979Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9c/ca040d94415048a3666fc237774df8151c96f8d2b661cbe3b184acc95876/jiter-0.16.0-cp311-cp311-win_arm64.whl", hash = "sha256:2a8e9e39cf083016137aa5cadafe3188adc2ba6ba1fbf1e5d18889ad3e9ad056", size = 194403, upload-time = "2026-06-29T13:03:12.341Z" }, + { url = "https://files.pythonhosted.org/packages/83/2b/52ace16ed031354f0539749a49e4bf33797d82bea5137910835fa4b09793/jiter-0.16.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:67c3bc1760f8c99d805dcab4e644027142a53b1d5d861f18780ebdbd5d40b72a", size = 306943, upload-time = "2026-06-29T13:03:14.035Z" }, + { url = "https://files.pythonhosted.org/packages/94/2e/34957c2c1b661c252ba9bcc60ae0bddc27e0f7202c6073326a13c5390eec/jiter-0.16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5af7780e4a26bd7d0d989592bf9ef12ebf806b74ab709223ecca37c749872ea9", size = 307779, upload-time = "2026-06-29T13:03:15.418Z" }, + { url = "https://files.pythonhosted.org/packages/88/6c/59bd309cab4460c54cf1079f3eb7fe7af6a4c895c5c957a53378693bad2b/jiter-0.16.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d5bf78d0e05e45cfdd66558893938d59afe3d1b1a824a202039b20e607d25a72", size = 335826, upload-time = "2026-06-29T13:03:17.11Z" }, + { url = "https://files.pythonhosted.org/packages/3b/8c/f5ef7b65f0df47afa16596969defb281ebb86e96df346d62be6fd853d620/jiter-0.16.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f4444a83f946605990c98f625cdd3d2725bfb818158760c5748c653170a20e0e", size = 362573, upload-time = "2026-06-29T13:03:18.781Z" }, + { url = "https://files.pythonhosted.org/packages/2b/0b/ace4354da061ee38844a0c27dc2c21eecd27aea119e8da324bea987522d0/jiter-0.16.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3a23f0e4f957e1be65752d2dfac9a5a06b1917af8dc85deb639c3b9d02e31290", size = 457979, upload-time = "2026-06-29T13:03:20.293Z" }, + { url = "https://files.pythonhosted.org/packages/55/40/c0253d3772eb9dcd8e6606ee9b2d53ec8e5b814589c47f140aa585f21eaa/jiter-0.16.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c22a488f7b9218e245a0025a9ba6b100e2e54700831cf4cf16833a27fba3ad01", size = 372302, upload-time = "2026-06-29T13:03:21.739Z" }, + { url = "https://files.pythonhosted.org/packages/a8/d2/4839422241aa12860ce597b20068727094ba0bc480723c74924ca5bad483/jiter-0.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:46add52f4ad47a08bfb1219f3e673da972191489a33016edefdb5ea55bfa8c48", size = 343805, upload-time = "2026-06-29T13:03:23.384Z" }, + { url = "https://files.pythonhosted.org/packages/e2/59/e196888a05befdda7dbe299b722d56f2f6eec65402bc34c0a3306d595feb/jiter-0.16.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:9c8a956fd72c2cf1e730d01ea080341f13aa0a97a4a33b51abebe725b7ae9ca9", size = 351107, upload-time = "2026-06-29T13:03:24.815Z" }, + { url = "https://files.pythonhosted.org/packages/ec/74/4cd9e0fca65232136400354b630fbfcd2de634e22ccbb96567725981b548/jiter-0.16.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:561926e0573ffe4a32498420a76d64b16c513e1ab413b9d28158a8764ac701e5", size = 388441, upload-time = "2026-06-29T13:03:26.266Z" }, + { url = "https://files.pythonhosted.org/packages/d9/8c/554691e48bc711299c0a293dd8a6179e24b2d66a54dc295421fcf64569c0/jiter-0.16.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:44d019fa8cdaf89bf29c71b39e3712143fdd0ac76725c6ef954f9957a5ea8730", size = 516354, upload-time = "2026-06-29T13:03:28.02Z" }, + { url = "https://files.pythonhosted.org/packages/a4/cb/01e9d69dc2cc6759d4f91e230b34489c4fdb2518992650633f9e20bece89/jiter-0.16.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0df91907609837f33341b8e6fe73b95991fdaa57caf1a0fbd343dffe826f386f", size = 547880, upload-time = "2026-06-29T13:03:29.534Z" }, + { url = "https://files.pythonhosted.org/packages/79/70/2953195f1c6ad00f49fa67e13df7e60acb3dd4f387101bc15abccddd905e/jiter-0.16.0-cp312-cp312-win32.whl", hash = "sha256:51d7b836acb0108d7c77df1742332cac2a1fa04a74d6dacec46e7091f0e91274", size = 203473, upload-time = "2026-06-29T13:03:31.025Z" }, + { url = "https://files.pythonhosted.org/packages/2d/05/2909a8b10699a4d560f8c502b6b2c5f3991b682b1922c1eedda242b225bd/jiter-0.16.0-cp312-cp312-win_amd64.whl", hash = "sha256:1878349266f8ee36ecb1375cc5ba2f115f35fd9f0a1a4119e725e379126647f7", size = 196905, upload-time = "2026-06-29T13:03:32.472Z" }, + { url = "https://files.pythonhosted.org/packages/e9/a9/6b82bb1c8d7790d602489b967b982a909e5d092875a6c2ade96444c8dfc5/jiter-0.16.0-cp312-cp312-win_arm64.whl", hash = "sha256:2ed5738ae4af18271a51a528b8811b0cbfa4a1858de9d83359e4169855d6a331", size = 190618, upload-time = "2026-06-29T13:03:34.672Z" }, + { url = "https://files.pythonhosted.org/packages/91/c0/555fc60473d30d66894ba825e63615e3be7524fac23858356afa7a38906c/jiter-0.16.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:41977aa5654023948c2dae2a81cbf9c43343954bef1cd59a154dd15a4d84c195", size = 306203, upload-time = "2026-06-29T13:03:36.243Z" }, + { url = "https://files.pythonhosted.org/packages/d0/2b/c3eaf16f5d7c9bad66ea32f40a95bd169b29a91217fcc7f081375157e99c/jiter-0.16.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d28bb3c26762358dadf3e5bf0bccd29ae987d65e6988d2e6f49829c76b003c09", size = 306489, upload-time = "2026-06-29T13:03:37.846Z" }, + { url = "https://files.pythonhosted.org/packages/96/3f/02fdfc6705cad96127d883af5c34e4867f554f29ec7705ec1a46156400a9/jiter-0.16.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0542a7189c26920778658fc8fcf2af8bae05bae9924577f71804acef37996536", size = 335453, upload-time = "2026-06-29T13:03:39.221Z" }, + { url = "https://files.pythonhosted.org/packages/b2/a6/e4bda5920d4b0d7c5dfb7174ce4a6b2e4d3e11c9162c452ef0eab4cdbdbd/jiter-0.16.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8fb8de1e23a0cb2a7f53c335049c7b72b6db41aa6227cdcc0972a1de5cb39450", size = 361625, upload-time = "2026-06-29T13:03:40.597Z" }, + { url = "https://files.pythonhosted.org/packages/b7/97/4e6b59b2c6e55cbb3e183595f81ad65dcfb21c915fee5e19e335df21bc55/jiter-0.16.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b72d0b2990ca754a9102779ac98d8597b7cb31678958562214a007f909eab78e", size = 456958, upload-time = "2026-06-29T13:03:42.074Z" }, + { url = "https://files.pythonhosted.org/packages/15/e0/97e9557686d2f94f4b93786eccb7eed28e9228ad132ea8237f44727314a7/jiter-0.16.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d5f91b1c27fc22a57993d5a5cb8a627cb8ed4b10502716fac1ffbfe1d19d84e8", size = 372017, upload-time = "2026-06-29T13:03:43.658Z" }, + { url = "https://files.pythonhosted.org/packages/0f/94/db768b6938e0df35c86beeba3dfbbb025c9ee5c19e1aa271f2396e50864d/jiter-0.16.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c682bea068a90b764577bdb78a60a4c1d1606daf9cd4c893832a37c7cc9d9026", size = 343320, upload-time = "2026-06-29T13:03:45.226Z" }, + { url = "https://files.pythonhosted.org/packages/c1/d6/5a59d938244a30735fe62d9433fd325f9021ea29d89780ea4596ea93bc89/jiter-0.16.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:8d031aabecc4f1b6276adfb42e3aabb77c89d468bf616600e8d3a11328929053", size = 350520, upload-time = "2026-06-29T13:03:46.671Z" }, + { url = "https://files.pythonhosted.org/packages/67/f8/c4a857f49c9af125f6bbcac7e3eee7f7978ed89682833062e2dbf62576b1/jiter-0.16.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:eab2cd170150e70153de16896a1774e3a1dca80154c56b54d7a812c479a7165e", size = 387550, upload-time = "2026-06-29T13:03:48.361Z" }, + { url = "https://files.pythonhosted.org/packages/8b/d6/5fbc2f7d6b67b754caa61a993a2e626e815dec47ffc2f9e35f01adfebec7/jiter-0.16.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:6edb63a46e65a82c26800a868e49b2cac30dd5a4218b88d74bc2c848c8ad60bb", size = 515424, upload-time = "2026-06-29T13:03:49.881Z" }, + { url = "https://files.pythonhosted.org/packages/ed/54/284f0164b64a5fed915fea6ba7e9ba9b3d8d37c67d59cf2e3bb99d45cdfe/jiter-0.16.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:659039cc50b5addcc35fcc87ae2c1833b7c0a8e5326ef631a75e4478447bcf84", size = 546981, upload-time = "2026-06-29T13:03:51.363Z" }, + { url = "https://files.pythonhosted.org/packages/13/c5/2a467585a576594384e1d2c43e1224deaafc085f24e243529cf98beef8e1/jiter-0.16.0-cp313-cp313-win32.whl", hash = "sha256:c9c53be232c2e206ef9cdbad81a48bfa74c3d3f08bcf8124630a8a748aad993e", size = 202853, upload-time = "2026-06-29T13:03:53.015Z" }, + { url = "https://files.pythonhosted.org/packages/88/6a/de61d04b9eec69c71719968d2f716532a3bc121170c44a39e14979c6be81/jiter-0.16.0-cp313-cp313-win_amd64.whl", hash = "sha256:baad945ed47f163ad833314f8e3288c396118934f94e7bbb9e243ce4b341a4fd", size = 196160, upload-time = "2026-06-29T13:03:54.447Z" }, + { url = "https://files.pythonhosted.org/packages/19/4b/b390ed59bafb3f31d008d1218578f10327714484b334439947f7e5b11e7f/jiter-0.16.0-cp313-cp313-win_arm64.whl", hash = "sha256:3c1fd2dbe1b0af19e987f03fe66c5f5bd105a2229c1aff4ab14890b24f41d21a", size = 189862, upload-time = "2026-06-29T13:03:55.754Z" }, + { url = "https://files.pythonhosted.org/packages/a7/89/bc4f1b57d5da938fd344a466396541e586d161320d70bffd929aaafcd8f4/jiter-0.16.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b2c61484666ad42726029af0c00ef4541f0f3b5cdc550221f56c2343208018ee", size = 308239, upload-time = "2026-06-29T13:03:57.205Z" }, + { url = "https://files.pythonhosted.org/packages/65/7a/c415453e5213001bf3b411ff65dec3d303b0e76a4a2cfea9768cd4960994/jiter-0.16.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:63efadc657488f45db1c676d81e704cac2abf3fdb892def1faea61db053127e2", size = 308928, upload-time = "2026-06-29T13:03:58.643Z" }, + { url = "https://files.pythonhosted.org/packages/11/fc/1f4fb7ebf9a724c7741994f4aae18fba1e2f3133df14521a79194952c34a/jiter-0.16.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cf0d73f50e7b6935677854f6e8e31d499ca7064dd24734f703e060f5b237d883", size = 336998, upload-time = "2026-06-29T13:04:00.071Z" }, + { url = "https://files.pythonhosted.org/packages/a0/8d/72cadaac05ccfa7cc3a0a2232862e6c72443ca40cf300ba8b57f9f18b69b/jiter-0.16.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bf3ea07d9bc8e7d03a9fbc051295462e6dbc295b894fd72457c3136e3e43d898", size = 362112, upload-time = "2026-06-29T13:04:01.52Z" }, + { url = "https://files.pythonhosted.org/packages/58/4a/c4b0d5f651fda90a24ffce9f8d56cde462a2e09d31ae3de3c68cef34c04e/jiter-0.16.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:26798522707abb47d767db536e4148ceac1b14446bf028ee85e579a2e043cfe5", size = 459807, upload-time = "2026-06-29T13:04:03.214Z" }, + { url = "https://files.pythonhosted.org/packages/80/58/ef77879ea9aa56b50824edc5a445e226422c7a8d211f3fd2a56bcb9493cf/jiter-0.16.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bc837c1b9631be10abfe0191537fe8009838204cec7e44827401ace390ddb567", size = 373181, upload-time = "2026-06-29T13:04:04.629Z" }, + { url = "https://files.pythonhosted.org/packages/49/2e/ffbc3f254e4d8a66da3062c624a7df4b7c2b2cf9e1fe43cf394b3e104041/jiter-0.16.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:49060fd70737fad59d33ba9dcc0d83247dc9e77187de26053a19c16c9f32bd69", size = 344927, upload-time = "2026-06-29T13:04:06.067Z" }, + { url = "https://files.pythonhosted.org/packages/9a/f6/0be5dc6d64a89f80aa8fec984f94dedb2973e251edcae55841d60786d578/jiter-0.16.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:adbb8edeadd431bc4477879d5d371ece7cb1334486584e0f252656dd7ffada29", size = 352754, upload-time = "2026-06-29T13:04:07.477Z" }, + { url = "https://files.pythonhosted.org/packages/da/6e/7d31243b3b91cd261dd19e9d3557fc3251a80883d3d8049c86174e7ab7af/jiter-0.16.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:31aaee5b80f672c1dc21272bcfb9cbdcfc1ea04ff50f00ed5af500b80c44fa93", size = 390553, upload-time = "2026-06-29T13:04:08.92Z" }, + { url = "https://files.pythonhosted.org/packages/25/33/51ae371fde3c88897520f62b4d5f8b27ad7103e2bb10812ff52195609853/jiter-0.16.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:6722bcef4ffc86c835574b1b2fac6b33b9fb4a889c781e67950e891591f3c55a", size = 516900, upload-time = "2026-06-29T13:04:10.407Z" }, + { url = "https://files.pythonhosted.org/packages/a0/45/6449b3d123ea439ba79507c657288f461d55049e7bcbdc2cf8eb8210f491/jiter-0.16.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:5ab4f50ff971b611d656554ea10b75f80097392c827bc32923c6eeb6386c8b00", size = 548754, upload-time = "2026-06-29T13:04:12.046Z" }, + { url = "https://files.pythonhosted.org/packages/9b/e7/fd2fb11ae3e2649333da3aa170d04d7b3000bbdc3b270f6513382fdf4e04/jiter-0.16.0-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:710cc51d4ebdcd3c1f70b232c1db1ea1344a075770422bbd4bede5708335acbe", size = 122381, upload-time = "2026-06-29T13:04:13.413Z" }, + { url = "https://files.pythonhosted.org/packages/26/80/f0b147a62c315a164ed2168908286ca302310824c218d3aae52b06c0c9a9/jiter-0.16.0-cp314-cp314-win32.whl", hash = "sha256:57b37fc887a32d44798e4d8ebfa7c9683ff3da1d5bf38f08d1bb3573ccb39106", size = 204578, upload-time = "2026-06-29T13:04:14.813Z" }, + { url = "https://files.pythonhosted.org/packages/5e/e6/4758a14304b4523a6f5adb2419340086aa3593bd4327c2b25b5948a90548/jiter-0.16.0-cp314-cp314-win_amd64.whl", hash = "sha256:cbd18dd5e2df96b580487b5745adf57ef64ad89ba2d9662fc3c19386acce7db8", size = 198154, upload-time = "2026-06-29T13:04:16.272Z" }, + { url = "https://files.pythonhosted.org/packages/26/be/41fa54a2e7ea41d6c99f1dc5b1f0fd4cb474680304b5d268dd518e81da3a/jiter-0.16.0-cp314-cp314-win_arm64.whl", hash = "sha256:a32d2027a9fa67f109ff245a3252ece3ccc32cc56703e1deab6cc846a59e0585", size = 191458, upload-time = "2026-06-29T13:04:17.707Z" }, + { url = "https://files.pythonhosted.org/packages/81/6b/59127338b86d9fe4d99418f5a15118bea778103ee0fe9d9dd7e0af174e95/jiter-0.16.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2577196f4474ef3fc4779a088a23b0897bbf86f9ea3679c372d45b8383b43207", size = 316739, upload-time = "2026-06-29T13:04:19.663Z" }, + { url = "https://files.pythonhosted.org/packages/2d/95/49461034d5388196d3dabf98748935f017b7785d8f3f5349f834bcc4ed0d/jiter-0.16.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:616e89e008a93c01104161c75b4988e58716b01d62307ebfe161e52a56d2a818", size = 340911, upload-time = "2026-06-29T13:04:21.257Z" }, + { url = "https://files.pythonhosted.org/packages/cd/97/a4369f2fb82cb3dda13b98622f31249b2e014b223fe64ee534413ad72294/jiter-0.16.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0e2e9efbe042210df657bade597f66d6d75723e3d8f45a12ea6d8167ff8bbce3", size = 361747, upload-time = "2026-06-29T13:04:22.677Z" }, + { url = "https://files.pythonhosted.org/packages/28/51/49b6ed456261646e1906016a6760367a28aacd3c24805e4e5fe64116c1db/jiter-0.16.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3f4d9e473a5ce7d27fef8b848df4dc16e283893d3f53b4a585e72c9595f3c284", size = 460225, upload-time = "2026-06-29T13:04:24.441Z" }, + { url = "https://files.pythonhosted.org/packages/33/b5/5689aff4f66c5b60be63106e591dbfcba2190df97d2c9c7cf052361ddb98/jiter-0.16.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8d30a4a1c87713060c8d1cc59a7b6c8fb6b8ef0a6900368014c76c87922a2929", size = 373169, upload-time = "2026-06-29T13:04:25.884Z" }, + { url = "https://files.pythonhosted.org/packages/a2/96/3ae1b85ee0d6d6cab254fb7f8da018272b932bbf2d69b07e98aa2a96c746/jiter-0.16.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bae96332410f866e5900d809298b1ed82735932986c672495f9701daacd80620", size = 350332, upload-time = "2026-06-29T13:04:27.302Z" }, + { url = "https://files.pythonhosted.org/packages/15/32/c99d7bafd78986556c95bf60ce84c6cc98786eac56066c12d7f828bb6747/jiter-0.16.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:da3d7ec75dc83bb18bca888b5edfae0656a26849056c59e05a7728badd17e7af", size = 353377, upload-time = "2026-06-29T13:04:28.731Z" }, + { url = "https://files.pythonhosted.org/packages/0e/4b/f99a8e571287c3dec766bcc18528bbe8e8fb5365522ab5e6d64c93e87066/jiter-0.16.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ee6162b77d49a9939229df666dfa8af3e656b6701b54c4c84966d740e189264e", size = 387746, upload-time = "2026-06-29T13:04:30.319Z" }, + { url = "https://files.pythonhosted.org/packages/75/69/c78a5b3f71040e34eb5917df26fb7ae9a2174cad1ccbf277512507c53a6e/jiter-0.16.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:63ffdbdae7d4499f4cda14eadc12ddcabef0fc0c081191bdc2247489cb698077", size = 517292, upload-time = "2026-06-29T13:04:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/c2/f7/095b38eda4c70d03651c403f29a5590f16d12ddc5d544aac9f9cddf72277/jiter-0.16.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:a111256a7193bea0759267b10385e5870949c239ed7b6ddbaaf57573edb38734", size = 549259, upload-time = "2026-06-29T13:04:33.721Z" }, + { url = "https://files.pythonhosted.org/packages/2e/c5/6a0207d90e5f656d95af98ebd0934f382d37674416f215aeda2ff8063e51/jiter-0.16.0-cp314-cp314t-win32.whl", hash = "sha256:de5ba8763e56b793561f43bed197c9ea55776daa5e9a6b91eed68a909bc9cdbf", size = 206523, upload-time = "2026-06-29T13:04:35.068Z" }, + { url = "https://files.pythonhosted.org/packages/a5/31/c757d5f30a8980fd945ce7b98be10be9e4ff59c7c42f5fd86804c2e87db8/jiter-0.16.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b8a3f9a6008048fe9def7bf465180564a6e458047d2ce499149cfbe73c3ae9db", size = 200366, upload-time = "2026-06-29T13:04:36.61Z" }, + { url = "https://files.pythonhosted.org/packages/7c/a2/d88de6d313d734a544a7901353ad5db67cb38dcfcd91713b7979dafc345d/jiter-0.16.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0fa25b09b13075c46f5bc174f2690525a925a4fc2f7c82969a2bbabff22386ce", size = 190516, upload-time = "2026-06-29T13:04:38.004Z" }, + { url = "https://files.pythonhosted.org/packages/06/d3/8e278946d43eeca2585b4dd0834a887cd71136329b837f3a16ed86a8b4b0/jiter-0.16.0-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:850ccb1d7eedb4200f4014b1c0e8a577de114fc3cd88faad646dcc9bc4bb12ad", size = 304518, upload-time = "2026-06-29T13:05:00.172Z" }, + { url = "https://files.pythonhosted.org/packages/72/43/28d4ef495028bf0506a413d4db3f4eb3e7288a382e0f065f306a17bbeb5e/jiter-0.16.0-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:e34e97bda77eb63242a410243c071e28ac7e0d8c0948c5ee658498690a4b2f2f", size = 310207, upload-time = "2026-06-29T13:05:02.123Z" }, + { url = "https://files.pythonhosted.org/packages/e0/ca/c366b1012da1d640de975d9683acd44e4d150d9068845d0ca2610435253f/jiter-0.16.0-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b7dc85ea77d4abbae8bad0d3538678aedee75bceec4e2f6c8dfb1c74772e5aa5", size = 342771, upload-time = "2026-06-29T13:05:03.55Z" }, + { url = "https://files.pythonhosted.org/packages/16/52/50cc4056fc1ae02e7154704e7ecc89df0afb8300222cfe8a52d3f67e4730/jiter-0.16.0-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:17ca7fae79f6d99cd9a042b75f917eaada7b895cfc7dd2ee3a16089dcaec7a85", size = 346468, upload-time = "2026-06-29T13:05:05.452Z" }, + { url = "https://files.pythonhosted.org/packages/98/ab/664fd8c4be028b2bedd3d2ff08769c4ede23d0dbc87a77c62384a0515b5d/jiter-0.16.0-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:f17d61a28b4b3e0e3e2ba98490c70501403b4d196f78732439160e7fd3678127", size = 303106, upload-time = "2026-06-29T13:05:07.118Z" }, + { url = "https://files.pythonhosted.org/packages/1a/07/421f1d5b65493a76e16027b848aba6a7d28073ae75944fa4289cc914d39f/jiter-0.16.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:96e38eea538c8ddf853a35727c7be0741c76c13f04148ac5c116222f50ece3b3", size = 304658, upload-time = "2026-06-29T13:05:08.708Z" }, + { url = "https://files.pythonhosted.org/packages/0a/db/bba1155f01a01c3c37a89425d571da751bbedf5c54247b831a04cb971798/jiter-0.16.0-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d284fb8d94d5855d60c44fefcab4bf966f1da6fada73992b01f6f0c9bc0c6702", size = 339719, upload-time = "2026-06-29T13:05:10.41Z" }, + { url = "https://files.pythonhosted.org/packages/78/f7/18a1afcd64f35314b68c1f23afcd9994d0bc13e65cc77517afff4e83986d/jiter-0.16.0-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64d613743df53199b1aa256a7d328340da6d7078aac7705a7db9d7a791e9cfd2", size = 343885, upload-time = "2026-06-29T13:05:12.087Z" }, +] + +[[package]] +name = "jobbench-harness" +version = "0.1.0" +source = { virtual = "." } +dependencies = [ + { name = "huggingface-hub" }, + { name = "mammoth" }, + { name = "openai" }, + { name = "openpyxl" }, + { name = "pandas" }, + { name = "pdfplumber" }, + { name = "python-pptx" }, + { name = "pyyaml" }, + { name = "xlrd" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pytest" }, + { name = "pytest-asyncio" }, +] + +[package.metadata] +requires-dist = [ + { name = "huggingface-hub", specifier = ">=0.30" }, + { name = "mammoth", specifier = ">=1.8" }, + { name = "openai", specifier = ">=1.0" }, + { name = "openpyxl", specifier = ">=3.1" }, + { name = "pandas", specifier = ">=2.0" }, + { name = "pdfplumber", specifier = ">=0.11" }, + { name = "python-pptx", specifier = ">=1.0" }, + { name = "pyyaml", specifier = ">=6.0" }, + { name = "xlrd", specifier = ">=2.0" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "pytest", specifier = ">=8.0" }, + { name = "pytest-asyncio", specifier = ">=0.23" }, +] + +[[package]] +name = "lxml" +version = "6.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/3b/aab6728cae887456f409b4d75e8a01856e4f04bd510de38052a47768b680/lxml-6.1.1.tar.gz", hash = "sha256:ba96ae44888e0185281e937633a743ea90d5a196c6000f82565ebb0580012d40", size = 4197430, upload-time = "2026-05-18T19:19:06.424Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/b0/83f481780d1548750b8ce2ec824073deef2f452d9cd1a6faff8507e3d16d/lxml-6.1.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:53b7d2b7a10b1c35c0a5e21e9224accf60c1bbfba523990732e521b2b73adef2", size = 8526461, upload-time = "2026-05-18T19:17:25.862Z" }, + { url = "https://files.pythonhosted.org/packages/b9/d5/30fa0f808002c7329397bfbb24e306789c0b29f04aa5842c07b174b4216f/lxml-6.1.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:ff3f333630ab480244a1bff72043e511a91eb22e7595dead8653ee5612dd8f3d", size = 4595375, upload-time = "2026-05-18T19:17:34.555Z" }, + { url = "https://files.pythonhosted.org/packages/4f/d2/edb71cf0e561581a7c5eb2626244320eb04e9f8ce6d563184fd668b45073/lxml-6.1.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:a4bbea04c97f6d78a48e3fbc1cb9116d2780b1b39e03a23f6eb9b603fd61f510", size = 4923654, upload-time = "2026-05-18T19:17:42.917Z" }, + { url = "https://files.pythonhosted.org/packages/4c/77/1bc7eeb0de4577d783fb625aa092cc9357883bba35845a3666bf1259f3dc/lxml-6.1.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:db1d75f6617a49c1c01bc7023713e0ff59ab32c9579ae62a7674c0e34f3b0b0a", size = 5067921, upload-time = "2026-05-18T19:17:49.175Z" }, + { url = "https://files.pythonhosted.org/packages/1b/3c/c0690d74bd2bc17bc03b5b0d093569ead597dd0bfa088bf99eef8c24e19c/lxml-6.1.1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a12689be69a28ddaa0ab99a5a1137da2afd5f8f16df7b5680b66f616d3eda1d", size = 5002456, upload-time = "2026-05-18T19:17:59.715Z" }, + { url = "https://files.pythonhosted.org/packages/66/8d/d1b3271af0c0f1e27e8472a849e4d2c65bc7766884b9ad2da9e76e145c88/lxml-6.1.1-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:18b73c339ae29b90fd2d06e58ebd555a751bde9cd6bbd36cc0281b9a2c94e9d8", size = 5202776, upload-time = "2026-05-18T19:18:08.924Z" }, + { url = "https://files.pythonhosted.org/packages/7a/45/689824ffb237fd10125ad273f32b28ff04dc6203c2822c85ff65a93df65e/lxml-6.1.1-cp311-cp311-manylinux_2_28_i686.whl", hash = "sha256:752d3bbfe874715ccd0aec7f88d7fc623c0f1fd7aa7b3238a084e017bad2a009", size = 5329945, upload-time = "2026-05-18T19:18:13.673Z" }, + { url = "https://files.pythonhosted.org/packages/5d/c0/ef73af53767e958fd87d437c170f272e2f6e6c0f854939f133a895f1e711/lxml-6.1.1-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:6b1761fbf9ec984e2e9d9c589ef5f5fd684b7c19f92aadd567a26c5224958db6", size = 4659237, upload-time = "2026-05-18T19:18:18.657Z" }, + { url = "https://files.pythonhosted.org/packages/a0/5e/e1158e40397585e91cb0472374a1f63d0926a1ddeaa92f13d1a1ffe306d5/lxml-6.1.1-cp311-cp311-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d680fbcb768404c601ecb43519ecd8461f6954cb11c06a78962f666832ccfca8", size = 5265904, upload-time = "2026-05-18T19:18:24.883Z" }, + { url = "https://files.pythonhosted.org/packages/a0/16/8687e5d1400ed1c0bc41dace232ebb7553952b618ea1f2e5fb6e2cfbbe23/lxml-6.1.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:162af1091cd785f2f27e62d3547ae9bc58ec5c86dd314d67021fd02463708d83", size = 5045225, upload-time = "2026-05-18T19:17:20.073Z" }, + { url = "https://files.pythonhosted.org/packages/ca/18/d877bd1ae2e5ffdfd4836565aba350db31feb2f2656d6ce70316ed66a05e/lxml-6.1.1-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:e9308ff8241c532df3f3e570f9a5aeed6c853f888512ba4b75638d7c11c95ef6", size = 4712721, upload-time = "2026-05-18T19:17:40.512Z" }, + { url = "https://files.pythonhosted.org/packages/44/4d/1f44fd1d770b10dacbf6b5c6e520f4d6e0708744930f719dc04e67cab981/lxml-6.1.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:5f6994074ebae6ffb04447268e37dc16edc304f9859cf91acb86e0af6c1b395c", size = 5252549, upload-time = "2026-05-18T19:17:51.236Z" }, + { url = "https://files.pythonhosted.org/packages/64/5d/1d66b84f850089254c230ef6ea6b267a5a54e2e179a5d960036a05d501d7/lxml-6.1.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:80c2dfadb855da477cf73373ad29a333535dedb9b12bad02c9814c8e2b43bf08", size = 5226877, upload-time = "2026-05-18T19:18:00.875Z" }, + { url = "https://files.pythonhosted.org/packages/ad/00/84c4b5302d42a2d0184f38d538c8a197f33b52a50bd4f7bcfe990bce3036/lxml-6.1.1-cp311-cp311-win32.whl", hash = "sha256:30a89d3ac8faec007453fb541f3f46807eeec88edd5826f6e3fe001752a2c621", size = 3594072, upload-time = "2026-05-18T19:17:12.714Z" }, + { url = "https://files.pythonhosted.org/packages/61/9d/2e2f7d876349f45e0f3e29f72da311668853d59b58d473a2dea4f0160135/lxml-6.1.1-cp311-cp311-win_amd64.whl", hash = "sha256:abbefa31eee84842140f67acef1c828e28bba8bbf0c3bc6e5492a9af88152c28", size = 4025469, upload-time = "2026-05-18T19:17:50.566Z" }, + { url = "https://files.pythonhosted.org/packages/b0/d5/570e6390e4110331e6208b2ba83d1482cc9146808ee118b22824a34c1070/lxml-6.1.1-cp311-cp311-win_arm64.whl", hash = "sha256:dcb292aa7fe485ceff7af4f92e46c5af397daec5dff64871a528f0fc47a3cc5b", size = 3667640, upload-time = "2026-05-19T19:22:48.293Z" }, + { url = "https://files.pythonhosted.org/packages/6a/6e/c4add832b6fc1e887125b96f880d7b9b70aae5248718e046b1704bcac4b9/lxml-6.1.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:104c09bda8d2a562824c0e319d0768ce26a779b7601e0931d33b09b53c392ef7", size = 8570821, upload-time = "2026-05-18T19:17:42.068Z" }, + { url = "https://files.pythonhosted.org/packages/22/00/ff3009c88e65de8011630acf8ab5a09cb2becd2aaf47fba2f3449f6224e9/lxml-6.1.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:25c6997a9a534e016695a0ba06b2f07945de682731ff01065b6d5a4474179da1", size = 4624252, upload-time = "2026-05-18T19:17:47.897Z" }, + { url = "https://files.pythonhosted.org/packages/42/95/bb63f0fd62e554fe078e1fb3c8fe9083c14ddc7ad7fa178d10e57e071ac7/lxml-6.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c921ba5c51e4e9f63b8b00267d06566e1f63407408a0496da2d1d0bfc819c7fc", size = 4930746, upload-time = "2026-05-18T19:18:29.637Z" }, + { url = "https://files.pythonhosted.org/packages/eb/99/0013e8d9b5960f4f041cf0b73e2f80c23eb5205b1f7bfb20203243651359/lxml-6.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:54a7f95e4de5fb94e2f9f4b9055c6ba33bf3d628fd77a1d647c5923caa2cdcdc", size = 5093723, upload-time = "2026-05-18T19:18:34.168Z" }, + { url = "https://files.pythonhosted.org/packages/29/91/317b332636bfc7bddcff828d41b3307f50043f4b237e40849c333d80fa1a/lxml-6.1.1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96f2ec43df44b1f76249ee0a615334f9b5b060e1c8bd90e706dad2d14d02f383", size = 5005557, upload-time = "2026-05-18T19:18:39.798Z" }, + { url = "https://files.pythonhosted.org/packages/42/2f/cc9bf06afe70f9c9093ae60855d9759da9db601ec4080f7473319666ffd7/lxml-6.1.1-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:70ef8a7e102a1508f8121aae5b0867abd663f72c14f0a9c937e6554cb4587b7b", size = 5631036, upload-time = "2026-05-18T19:18:44.858Z" }, + { url = "https://files.pythonhosted.org/packages/08/f6/af32e23e563971ffb0fb86be52bc5be5c2c118858ffc119bf6a9039b173d/lxml-6.1.1-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ebe6af670449830d6d9b752c256a983291c766a1365ba5d5460048f9e33a7818", size = 5240367, upload-time = "2026-05-18T19:18:49.217Z" }, + { url = "https://files.pythonhosted.org/packages/78/83/8555d40948b09ce86f1bd0c68a7ac31d07b1929f92cc1b074006c97ef2d2/lxml-6.1.1-cp312-cp312-manylinux_2_28_i686.whl", hash = "sha256:27acc820660aaffa4f7c087f29120e12980f7779d56d8492d263170111284740", size = 5350171, upload-time = "2026-05-18T19:18:52.779Z" }, + { url = "https://files.pythonhosted.org/packages/63/75/5d92da93729b7bad783689e6496049fa40927b45bec7bf183c981de3ca70/lxml-6.1.1-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:1db753c9115ec7100d073b744d17e25e88a8f90f5c39b2f5dd878149af59671f", size = 4694874, upload-time = "2026-05-18T19:18:55.139Z" }, + { url = "https://files.pythonhosted.org/packages/c5/b5/3aad415a9a25b822e783f15deeb4dffccf5113030f1afa2222dd929313d9/lxml-6.1.1-cp312-cp312-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c4f469aebd783bb741c2ecb2a681008fd26bfe5c16a9a72ed5467f834e810df2", size = 5244492, upload-time = "2026-05-18T19:19:01.28Z" }, + { url = "https://files.pythonhosted.org/packages/f1/a1/5fcf7eb9904b80086aa47dcf0027de07b1bb990afad2e6823144c368ae04/lxml-6.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:766b010012d59470072c1816b5b6c69f1d243e5db36ea5968e94accf430a4635", size = 5048232, upload-time = "2026-05-18T19:18:12.67Z" }, + { url = "https://files.pythonhosted.org/packages/77/74/1f601b63c7a69fcdf10fa9b148c81da8442204194f6c55509cc485c786b9/lxml-6.1.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:b8d812c6011c08b8111a15e54dd990b8923692d80adf35488bee34026c35accf", size = 4777023, upload-time = "2026-05-18T19:18:15.928Z" }, + { url = "https://files.pythonhosted.org/packages/a2/b9/7a78f51aec95b1bf780d78e12705a9f6533284f8693dc5c0e6724fa53d3f/lxml-6.1.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:fe0306bd29505a9177aac19f1877174b0e7422c222a59f70b2cd41633448c3dc", size = 5645773, upload-time = "2026-05-18T19:18:23.223Z" }, + { url = "https://files.pythonhosted.org/packages/a5/6e/98a7b7ad54e4e74fa1f20fff776913980619d0ebe5558232d7da6580bdd8/lxml-6.1.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:5ba186ad207446c65d3bb3d3e0412b032b1d9f595e59861e2354798c5703d955", size = 5233088, upload-time = "2026-05-18T19:18:31.433Z" }, + { url = "https://files.pythonhosted.org/packages/65/d1/bc0ed2427bf609f2ee10da303a6a226f9c8bce94f945dc29a32ce55de6e4/lxml-6.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:aa366a1e55b8ebfe8ca8ddc3cfe75c8ebade181aeb0f661d0cb05986b647f72a", size = 5260995, upload-time = "2026-05-18T19:18:37.091Z" }, + { url = "https://files.pythonhosted.org/packages/69/8b/6772e1a4b513fc50a8d931f19edde0e13ae6918510a1e13ff67864f3e5ed/lxml-6.1.1-cp312-cp312-win32.whl", hash = "sha256:126c93f7f56f0eda92f6d8c619edc463a4f23d9252f1c9d0405a76f25fa9f11a", size = 3596382, upload-time = "2026-05-18T19:17:18.37Z" }, + { url = "https://files.pythonhosted.org/packages/1b/89/45198e9624762af2dfd2cb8782598477ceb29f6e59caab560388ae1f4ec1/lxml-6.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:26e6eda8d38c1fcab1090dd196ee87cbd13788e531937610e2589085de074e77", size = 3997255, upload-time = "2026-05-18T19:17:56.781Z" }, + { url = "https://files.pythonhosted.org/packages/90/a9/7a54b6834088d9ae528a7b780584ba6a39a9457b0ac330479f20ffbc9449/lxml-6.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:6540377fbd53fe1b629172288c464fb18db11ce1fa7dc15891da10aa9dcc3e7f", size = 3659610, upload-time = "2026-05-19T19:22:50.843Z" }, + { url = "https://files.pythonhosted.org/packages/a5/eb/7e6f37c5584ccbb2ff267f56fd0339016938c1c8684cfefab9b33ffc2f36/lxml-6.1.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:68a9198d0fc122d14bb76837de9aa80cf84caed990b5b237f532ed87d3706736", size = 8559780, upload-time = "2026-05-18T19:17:57.661Z" }, + { url = "https://files.pythonhosted.org/packages/a1/36/587c2521cf23a2cd6c9c22108aa7528f683a1f195ed7ccd23a4b1786ad36/lxml-6.1.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7d47866cb32fb503450b6edc9df355d10dc49836af2e89901bd6ac6b0896d9d9", size = 4618006, upload-time = "2026-05-18T19:18:04.452Z" }, + { url = "https://files.pythonhosted.org/packages/6e/ca/ab7bfe2bf4c972af5e7878262845ead3a24a929a9b04bc11c7c1ece6c82a/lxml-6.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:eb7c9811bfaa8b1ed5ed319f5d370dfbcaa59d52ea64be2a5a85e18195930354", size = 4924139, upload-time = "2026-05-18T19:19:04.873Z" }, + { url = "https://files.pythonhosted.org/packages/6b/55/a0c72851dfee5ecc689f949723a73dea457758912542cb955b108eaf0d8f/lxml-6.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:762ff394d5bd56da0cf034a23dcce4e13923f15321a2adfa2ac00201dc6d3fca", size = 5082329, upload-time = "2026-05-18T19:19:09.728Z" }, + { url = "https://files.pythonhosted.org/packages/f0/b6/0608f7d61a3b96cc67e5648a3d906e31a5082093e10e7be65b3886289938/lxml-6.1.1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a088f287f7d8275a33c07f2cac6c50b9319309a0200a39e7e75d80c707723099", size = 4993564, upload-time = "2026-05-18T19:19:13.608Z" }, + { url = "https://files.pythonhosted.org/packages/4c/66/ae227524b066d29d55bf0b453d93d2d793c40218657d643dcbbca13b8faf/lxml-6.1.1-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e902da4b04e6b52e5893900d4b8ab46068f75f3561f01bf1080957f9fd932ed6", size = 5613467, upload-time = "2026-05-18T19:19:16.228Z" }, + { url = "https://files.pythonhosted.org/packages/a6/76/dbe4a00b50385e40194231dcfe5a12c059de7cf90e89c83407d2b085b719/lxml-6.1.1-cp313-cp313-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1d4962d4c66bf830a7e59ed6cfc17d148149898a3aefa8ec6e59763e6e3ed085", size = 5228304, upload-time = "2026-05-18T19:19:19.354Z" }, + { url = "https://files.pythonhosted.org/packages/1c/01/00b1b8442ed2041793336868ba0b9ea4b13d7da7c085c6404c207a63bf79/lxml-6.1.1-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:581d4c8ae690a6609e64862dd6b7c2489635c2d13907fc2b20f2bc200ff1d21e", size = 5341607, upload-time = "2026-05-18T19:19:22.297Z" }, + { url = "https://files.pythonhosted.org/packages/63/36/1ad29931e9a4638bb707869f01d423a6c815f82152138d1a40dfcfde2b95/lxml-6.1.1-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:876e1ff5930ed8bf295ec5ef9a8155e9b6b1876bbf1deed8b3a8069311875a8f", size = 4700168, upload-time = "2026-05-18T19:19:25.133Z" }, + { url = "https://files.pythonhosted.org/packages/3c/d1/a9536cecf9be18a0dc72d32bead283a2332d1ffebd2dd3ac70ce444686e5/lxml-6.1.1-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9eb9b5a968f6e0f6d640092a567e14529ff8cea2e29d00da6f78a79fa49f013c", size = 5232487, upload-time = "2026-05-18T19:19:28.603Z" }, + { url = "https://files.pythonhosted.org/packages/0e/77/b4fb1e03bf5d130e879214d3100092e386418807fb74dd0adc4b0a48f351/lxml-6.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:aa49e06d94aba782c6a02eecb7e507969e7e7a41b267f1b359bb35585f295d5b", size = 5044231, upload-time = "2026-05-18T19:18:42.246Z" }, + { url = "https://files.pythonhosted.org/packages/26/4c/d00daeeb0a5530c4028a9232aa1b93db3ef4ed2158c116ea73c79a9765b3/lxml-6.1.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:70cdfd80589d59e43e18005dd7244e8895e93db8ab6a620b7e23df5445a4e3d2", size = 4769450, upload-time = "2026-05-18T19:18:48.013Z" }, + { url = "https://files.pythonhosted.org/packages/ed/6a/715a3a8d156ce42f29cf014706f5410c2ff3b02267774110fc23266409fe/lxml-6.1.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:aad9aa39483ed8ec44d6d2e59e5b98a0d80676ef0d92f44bfc374836111f62f5", size = 5635874, upload-time = "2026-05-18T19:18:51.914Z" }, + { url = "https://files.pythonhosted.org/packages/45/37/0544bc21dde2a88f3a17b504e6fc79c0e01d25a33c2f6079724e9e72b9c7/lxml-6.1.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:d49514be2f28d895c38cf9d2b72d7b9a07d00314519f456c0b50b53cfcf4c785", size = 5223987, upload-time = "2026-05-18T19:18:59.715Z" }, + { url = "https://files.pythonhosted.org/packages/4d/f8/f6a5e8185bcb28c2befae3d31f8e3df3b811cb0f47746517a81279fcafe1/lxml-6.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:47402e62c52ff5988c1e8c6c63177f5708bccf48e366dea4e3dcf1e645e04947", size = 5250276, upload-time = "2026-05-18T19:19:03.834Z" }, + { url = "https://files.pythonhosted.org/packages/c7/f2/1a2b9f1b7a49d45495369be7ef9ad05b262930f2eab3e3145706fca8083f/lxml-6.1.1-cp313-cp313-win32.whl", hash = "sha256:3483644525531e1d5762b0c44a8e18b6efba321b6dcf8a8952de10b037618bca", size = 3596903, upload-time = "2026-05-18T19:17:29.863Z" }, + { url = "https://files.pythonhosted.org/packages/e6/99/f4ffb024f238eec2131aaa09f3278fb6129cf892741bf68e1fc1afb8c100/lxml-6.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:a10bd2fd62e8ce916ececb342f348f190724a098c1faa056fdfb2a22ad5e8660", size = 3995869, upload-time = "2026-05-18T19:18:02.596Z" }, + { url = "https://files.pythonhosted.org/packages/d1/53/70eb8c5c6037f27448f1e3c54ebede9545a801ae63f0a7254afca4fe8e45/lxml-6.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:424aa57aca0897eb922aef34395bd1289b3b6f04e6bae20ea123c0c7e333cffc", size = 3658490, upload-time = "2026-05-19T19:22:53.846Z" }, + { url = "https://files.pythonhosted.org/packages/13/e2/2e325795566de01d0d7c3bb57d3c370616b2d07b01214e84eec5d3b10963/lxml-6.1.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:19b7ab10b210b0b3ad7985d9ac4eb66ab09a90b20fe6e2f7ba55d01a234345d0", size = 8577146, upload-time = "2026-05-18T19:18:17.765Z" }, + { url = "https://files.pythonhosted.org/packages/93/cf/5630b5e4be7d2e6bee8efe83865c925221103cf0221303b104ce134b01e2/lxml-6.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c08e5c694306507275f2290073350c4f32e383db15213b2c69e7ff39c1193840", size = 4623866, upload-time = "2026-05-18T19:18:30.669Z" }, + { url = "https://files.pythonhosted.org/packages/d2/51/3904907c063451cf8d4a5c9fe0cad95fa1f4ec57f4e3884fa0731bd7a305/lxml-6.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:74a9717fd0d82effef5c2854f0d917231d5324b5a3eb7275c43ac9fa32f97a14", size = 4950022, upload-time = "2026-05-18T19:19:31.958Z" }, + { url = "https://files.pythonhosted.org/packages/94/cd/9c7611a51c37a2830928405817cc5d56a97f64fab83cc3f628748b135749/lxml-6.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:efe0374196335f93b53269acd811b944f2e6bdc88e8894f214bd636455484909", size = 5086695, upload-time = "2026-05-18T19:19:34.764Z" }, + { url = "https://files.pythonhosted.org/packages/da/d6/24e3b5906abb0b674ff2ae195bc3ce59708df2bcd17cf17703b2d7dd643a/lxml-6.1.1-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ac931cdc9442c1763b8a8f6cd62c0c938737eafc5be75eff88df55fc73bc0d00", size = 5031642, upload-time = "2026-05-18T19:19:37.771Z" }, + { url = "https://files.pythonhosted.org/packages/2d/db/6ec54f99019838bff54785c51da07f189eb4676861c5f2730962b0d8d665/lxml-6.1.1-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:aee395f5d0927f947758b4ec119fd5fc8ec71f07a1c5c52077b30b04c0fa6955", size = 5647338, upload-time = "2026-05-18T19:19:40.553Z" }, + { url = "https://files.pythonhosted.org/packages/42/3d/ef4dcfffd22d27a61805d8ed9f7fb888495bc6aa88648fa07c1eaa5586b6/lxml-6.1.1-cp314-cp314-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9395002973c827b3ed67db77e6ec09f092919a587022174554096a269378fb13", size = 5239528, upload-time = "2026-05-18T19:19:43.657Z" }, + { url = "https://files.pythonhosted.org/packages/62/bb/37fb3f0dff146bdcfa78eec47879273820b2a0bf350ec236ce14bd0b1c26/lxml-6.1.1-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:73bc2086f141224ebddb7fc5c6a36ca58b31b94b561e1dfe8e073e3270fad1e7", size = 5350730, upload-time = "2026-05-18T19:19:46.307Z" }, + { url = "https://files.pythonhosted.org/packages/90/42/43253f168388df4fae1f38c01df36ddb9bee39e2048167b54cdcbae85ea3/lxml-6.1.1-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:3779def59032b81e44a5f70096ef6bf2082f8d901937dca354474ba09782e245", size = 4697530, upload-time = "2026-05-18T19:19:49.889Z" }, + { url = "https://files.pythonhosted.org/packages/eb/a8/c5a8504f81bbdfc8e7094c2c850cdb4ed6777fc4d5ddd9e5ab819f3b0d54/lxml-6.1.1-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:86c89b9d55ebf820ad7c90bc533410f0d098054f293351f10603c0c46ff598f5", size = 5250670, upload-time = "2026-05-18T19:19:53.199Z" }, + { url = "https://files.pythonhosted.org/packages/77/b7/c7e76ab18744d75e21f320ebf9ff9d1ceae2b54dd431ea5a64caf26c9672/lxml-6.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:19607c6bbff2a44cf3fe8250abccd20942d3462473e0a721d01d379ed017e462", size = 5084485, upload-time = "2026-05-18T19:19:08.422Z" }, + { url = "https://files.pythonhosted.org/packages/31/31/b35c53f8ef7b7c31cacd23d3638652fff7bcd1deb6eedb709ab43b685908/lxml-6.1.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:c6ed5141a5c7507cf3ee76bd363b0d6f801e3321adc35b5d825a23115faa5465", size = 4737635, upload-time = "2026-05-18T19:19:12.321Z" }, + { url = "https://files.pythonhosted.org/packages/d9/06/31f23c813a7fe8e0cb1b175e915b08c9bf4e86d225b210feadbdbe519667/lxml-6.1.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:62aeb7e85b5d60320b9d77eef2e773994e2c0ce10121b277e0a19804e1654a5a", size = 5670681, upload-time = "2026-05-18T19:19:15.001Z" }, + { url = "https://files.pythonhosted.org/packages/1a/bc/ce619bccc89b1fd9ad8a8e1330ee3f3beff9f2ff95b712d7bbcdd6e22fc3/lxml-6.1.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:b1b963fd8f5caa68e99dfae060d54de1fe9cba899b8718b44a00cdca53c3e590", size = 5238229, upload-time = "2026-05-18T19:19:18.131Z" }, + { url = "https://files.pythonhosted.org/packages/2f/5d/b329acbbedc0b619ebc2be6cf7ee9ed07e80892c88d4dfd612c33805789a/lxml-6.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:63876be28efefa04a1df615b46770e82042cce445cfdce55160522f57b231ccb", size = 5264191, upload-time = "2026-05-18T19:19:21.118Z" }, + { url = "https://files.pythonhosted.org/packages/d6/85/be36fb1425b30db3c3f9df75fe86343ebffb79e6320bd7f588e25bfeac39/lxml-6.1.1-cp314-cp314-win32.whl", hash = "sha256:7f7a92e8583f06b1fd49d01158143b8461cfcd135dcb10ec807270a3051bd603", size = 3657202, upload-time = "2026-05-18T19:17:39.509Z" }, + { url = "https://files.pythonhosted.org/packages/b8/ce/3cf9a827342269f54d405a6202397de63f07c69cbd6ce7d183a3f0cba1e9/lxml-6.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:b2d444f2e66624d68e9c6b211e28a76e22fff5fcabcfff4deac18b529b7d4137", size = 4064497, upload-time = "2026-05-18T19:18:14.662Z" }, + { url = "https://files.pythonhosted.org/packages/d9/3e/1a957bde8f0760039e627f94699f82caa782c9d838d86c3d28245ee67212/lxml-6.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:3fd9728a2735fda14f4e8235830c86b539e9661e849665bf926d3f867943b4bf", size = 3741991, upload-time = "2026-05-19T19:22:59.111Z" }, + { url = "https://files.pythonhosted.org/packages/78/b2/00ed55b3a2efa4658fb795c38d1090ec9b3e8a6c3683d4441fa517f09c3b/lxml-6.1.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:787b2496d0dbe8cd180984e8d29e3a6f76e7ea34db781cb3bd55e4ba1ef8b4ee", size = 8827545, upload-time = "2026-05-18T19:18:41.193Z" }, + { url = "https://files.pythonhosted.org/packages/c0/73/74573db19baa618d5f266f2407898b087ff6927115b00b71e5fc1b700847/lxml-6.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:2c8daa471358dc2d6fcf02165e80ec68f77871a286df95bc5cc3816153b0fd2c", size = 4735736, upload-time = "2026-05-18T19:18:46.761Z" }, + { url = "https://files.pythonhosted.org/packages/16/02/6f7061f4f95f51e545d48e87647c54791d204a4e881be4156e7a26ba5338/lxml-6.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:acd7d70b64c0aae0c7922cca83d288a16f5f6da523637697872253415269baef", size = 4970291, upload-time = "2026-05-18T19:19:56.215Z" }, + { url = "https://files.pythonhosted.org/packages/b0/02/55fc057d8283427dea7d6edb102e7a840239c77a64a983d92f62a304c0e9/lxml-6.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4f0dd2f01f9f8a89f565d000e03abcf0a13d692a346c8d22f628d49af098777a", size = 5102822, upload-time = "2026-05-18T19:19:59.223Z" }, + { url = "https://files.pythonhosted.org/packages/e4/48/8e1cf78d89d66850121d9255a2a24414c98f775da93b90cf976956c24b14/lxml-6.1.1-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b7e8a14c8634bf6f7a568634cb395305a6d964aeb5b7ee32248094bed3a7e2c", size = 5027923, upload-time = "2026-05-18T19:20:01.549Z" }, + { url = "https://files.pythonhosted.org/packages/ed/00/0632a0647612c8af24d26997b3b961397daa9d5b2581444805933629a4cb/lxml-6.1.1-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:86281fbdd6a8162756f8d603f37e3435bfa38043adb79c6dc6a2dfee065e7525", size = 5595843, upload-time = "2026-05-18T19:20:03.93Z" }, + { url = "https://files.pythonhosted.org/packages/bc/86/ab008a7dc360711b66858d61c80a5979a70a09f2aa2b05d9698df80b803d/lxml-6.1.1-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c5d7152ec39ca7c402d8fb9bad86140a15b9503bd0c54484e3f1bbe3dd37ceca", size = 5224515, upload-time = "2026-05-18T19:20:06.381Z" }, + { url = "https://files.pythonhosted.org/packages/75/c6/2702ff375e728e34f56d9a45339a9cf7e4427e917f542225242d63a05afa/lxml-6.1.1-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:88d8cb75b9d82858497a5393e3c63cfbf03035225e4b35a49ed7ccb151e4dc0e", size = 5312511, upload-time = "2026-05-18T19:20:09.308Z" }, + { url = "https://files.pythonhosted.org/packages/b7/57/a5807c98f87a86f10ef9ffab35516df7c0f0c4b6d5d33e9f608ab9c04a31/lxml-6.1.1-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f64ec5397ea6a41fc1b4af0380d79b44a755b5531dcaccd9940fb260dca93038", size = 4639206, upload-time = "2026-05-18T19:20:11.704Z" }, + { url = "https://files.pythonhosted.org/packages/1f/e1/8a0a2c35734812395f4da4eaf33748a7e5705bfb2a58b128da764339d5ec/lxml-6.1.1-cp314-cp314t-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d34bbf07dbc7ca5970671b1512e928991fb5e9d95365636c9b2d8b4f53af405e", size = 5232404, upload-time = "2026-05-18T19:20:14.064Z" }, + { url = "https://files.pythonhosted.org/packages/c2/e2/0e6a4dd5ad84d01d99aa7bae7cfefd4a760a0e0f8176818241de17d9b6c0/lxml-6.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:17e0e18d4ad8adbd0399291bc44845b69d9dd68439a3cdebdf35ff902ec05072", size = 5083769, upload-time = "2026-05-18T19:19:23.758Z" }, + { url = "https://files.pythonhosted.org/packages/a0/7e/161f33d463f6ffc1c7679104b65086dea120080d49dde4d238f015aaee2f/lxml-6.1.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:3ab541146f1f6968c462d6c2ac495148e8cdba2f8347700b2141b6ec5a75bf52", size = 4758936, upload-time = "2026-05-18T19:19:27.256Z" }, + { url = "https://files.pythonhosted.org/packages/f1/fb/2369825e3f6ca99305bf9f7b7085fda91c8b0922a89e54d900974aa3ef85/lxml-6.1.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2a0217714657e023ef4293500f65aa20fce6164c8fd6b08fa5bd4a859fb14b9b", size = 5620296, upload-time = "2026-05-18T19:19:29.993Z" }, + { url = "https://files.pythonhosted.org/packages/30/90/d61e383146f74c5ab683947ea14dc7b82778838ab9b95ea73a23b60d0191/lxml-6.1.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:05a82eb6e1530a64f26225b55cbd178113bd0b5af1c2b625f25e5296742c26d2", size = 5228598, upload-time = "2026-05-18T19:19:33.523Z" }, + { url = "https://files.pythonhosted.org/packages/76/2d/2dafd8149e94b05bb070690efd5bb2680720681e03ff03fc57d2b70a1105/lxml-6.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9e36f163528fc50cbef305f02a5fd66d404edf7049cdaff211dbc2cba5a7013e", size = 5247845, upload-time = "2026-05-18T19:19:36.649Z" }, + { url = "https://files.pythonhosted.org/packages/ce/68/b30e913340c380ddac9580c6e6230991fc37240ec4f64704833e4f3e2769/lxml-6.1.1-cp314-cp314t-win32.whl", hash = "sha256:649dda677cf3bd6ac9ae14007ba0c824ded8ce5808b53fc7431d9140399118c1", size = 3897345, upload-time = "2026-05-18T19:17:33.562Z" }, + { url = "https://files.pythonhosted.org/packages/3c/4e/9eb2af5335545f9fbcd7af57bcf87c6025d31eaa31b14ec184a6c8675328/lxml-6.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:793033d6c5cdf33a573f910d9bea14ef8f5771820411d118da8e1182edb53d5e", size = 4393350, upload-time = "2026-05-18T19:18:10.076Z" }, + { url = "https://files.pythonhosted.org/packages/7f/2c/0f1e93c636720e8a3eb59af2bfda99d98b55891e1c53bc30c2e0e865f01b/lxml-6.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:58bb955caba94e467d2a96da17660d2d704e0675894cba21ab8a775b8621fd1c", size = 3817223, upload-time = "2026-05-19T19:22:56.823Z" }, + { url = "https://files.pythonhosted.org/packages/b5/32/86a3f0f724a3a402d4627937a7fc27b160e45e7012b4adf47f6e1e844511/lxml-6.1.1-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:31033dc34636ea6b7d5cc11b1ddbda78a14de858ba9d3e1ed4b69a3085bc521e", size = 3930127, upload-time = "2026-05-18T19:19:02.27Z" }, + { url = "https://files.pythonhosted.org/packages/40/44/d832e82af08723761556d004b1d04d281c09f9a8cecd7d3148548c9941a3/lxml-6.1.1-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:3893c14c4b6ac5b2d54ba8cf03e99fe5104e592de491f19bd6b82756c09f8004", size = 4210769, upload-time = "2026-05-18T19:20:41.427Z" }, + { url = "https://files.pythonhosted.org/packages/6d/39/0dc5949f759ed7d951e0bb8c2f2d9d7aca1908d22352fa84a8afd2ea54af/lxml-6.1.1-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c07da4cebf6889f03ebac8d238f62318e29f495de0aa18a51ea14e61ae907e2e", size = 4318163, upload-time = "2026-05-18T19:20:44.702Z" }, + { url = "https://files.pythonhosted.org/packages/e6/fb/8ab3845fe046ba4cbf74536bcf6801a774b7caf4350de1c5d37f1f0a9e90/lxml-6.1.1-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f6f0ce10945fab9c4c06ce14e22af9059d1a87493a9af4501a5b0b9187e21cf2", size = 4250945, upload-time = "2026-05-18T19:20:47.385Z" }, + { url = "https://files.pythonhosted.org/packages/68/1b/7553ab136894374ffae8851ec06f98f511cd8e66246e41b6be059d0a7289/lxml-6.1.1-pp311-pypy311_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f8844cd288697c6425c9beba919302241e3278871dc6519515e72b04e987abcf", size = 4401664, upload-time = "2026-05-18T19:20:50.489Z" }, + { url = "https://files.pythonhosted.org/packages/db/a4/441aee36c6f6b249823d20fd91f9be9ab89d7c5a8ae542a4a4ca6d342d56/lxml-6.1.1-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:ed21202aec73cda4d55d1ce57b389aadb90ffb044e6cd1080b8347efe1b1ec84", size = 3508989, upload-time = "2026-05-18T19:18:38.158Z" }, +] + +[[package]] +name = "mammoth" +version = "1.12.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cobble" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/46/92/aca6c4208b3a8e56a788b224e6c9b7bb70fc68e6641124fcf784008338e1/mammoth-1.12.1.tar.gz", hash = "sha256:40521e02568583b671d9976b0fbeed50be734b40f73ca8e1939ab7146bf69bbe", size = 53797, upload-time = "2026-08-09T14:11:20.096Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/be/b4/e7907b278386ffec0cf631db93836e02de9a6386add4653a52d7ac6b43ae/mammoth-1.12.1-py2.py3-none-any.whl", hash = "sha256:2af047e3e796faa25740112310ddf11f8de2a24c96dc57de3c87dfd7cb6543b3", size = 55091, upload-time = "2026-08-09T14:11:18.854Z" }, +] + +[[package]] +name = "numpy" +version = "2.4.6" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.12' and sys_platform == 'win32'", + "python_full_version < '3.12' and sys_platform == 'emscripten'", + "python_full_version < '3.12' and sys_platform != 'emscripten' and sys_platform != 'win32'", +] +sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/49/ec46835a70be8fa6446c495126ac84fdb28cb2558e1620ffb87a10c8b64c/numpy-2.4.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4", size = 16969194, upload-time = "2026-05-18T23:33:13.503Z" }, + { url = "https://files.pythonhosted.org/packages/0e/0d/f5957185c0ee2f3e12f78715aa9e3b353fd83633316c8532b38faa37e3f6/numpy-2.4.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d", size = 14964111, upload-time = "2026-05-18T23:33:17.795Z" }, + { url = "https://files.pythonhosted.org/packages/ad/40/40a40ee0ddf7ceb782c49af278894b686e586d65d8c1889c8b5da01a3d7d/numpy-2.4.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8", size = 5469159, upload-time = "2026-05-18T23:33:20.654Z" }, + { url = "https://files.pythonhosted.org/packages/63/13/f9a8046535cb21deae82f8d03de9617e08882d274fad2539630761888228/numpy-2.4.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538", size = 6798936, upload-time = "2026-05-18T23:33:22.987Z" }, + { url = "https://files.pythonhosted.org/packages/33/a8/6fa8c1a345a8c85dbb21932c447bee07c30a2c2a3f31e369c0a84b300147/numpy-2.4.6-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47", size = 15966692, upload-time = "2026-05-18T23:33:26.62Z" }, + { url = "https://files.pythonhosted.org/packages/02/03/74fe2a4cb3817d94d86402f2506554130a2f01414e299b5a843e5a8a957f/numpy-2.4.6-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93", size = 16918164, upload-time = "2026-05-18T23:33:29.955Z" }, + { url = "https://files.pythonhosted.org/packages/c5/80/3615be3313f7e7696609bc194b9f0101da809df79e859bdb84e0cd043f46/numpy-2.4.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8", size = 17322877, upload-time = "2026-05-18T23:33:34.724Z" }, + { url = "https://files.pythonhosted.org/packages/ca/ac/a691e0fe2675e370d0e08ff905adc49a1c8830e8cae03efe4477e92cd55d/numpy-2.4.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6", size = 18651487, upload-time = "2026-05-18T23:33:38.217Z" }, + { url = "https://files.pythonhosted.org/packages/15/a7/9bc1cd626d7bf6869bfedf27b91b6ab5dd607758bf8e959d6fa80c6a59cb/numpy-2.4.6-cp311-cp311-win32.whl", hash = "sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8", size = 6233945, upload-time = "2026-05-18T23:33:41.331Z" }, + { url = "https://files.pythonhosted.org/packages/c5/31/7fc6239c12bce7e931463251cca4426c465e1876ba3cc785402ef4dd8f4e/numpy-2.4.6-cp311-cp311-win_amd64.whl", hash = "sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147", size = 12608406, upload-time = "2026-05-18T23:33:44.131Z" }, + { url = "https://files.pythonhosted.org/packages/27/83/140f85a466595a16382996a1bf06b2b54bcd597488921b0c9daaeeda72af/numpy-2.4.6-cp311-cp311-win_arm64.whl", hash = "sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577", size = 10479528, upload-time = "2026-05-18T23:33:50.725Z" }, + { url = "https://files.pythonhosted.org/packages/95/2a/3d7b5ac8aac24feaf9ad7ed58f45b0bbc06d37e4338ae84c9f2298b570f9/numpy-2.4.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1", size = 16689119, upload-time = "2026-05-18T23:33:54.065Z" }, + { url = "https://files.pythonhosted.org/packages/ea/12/92c4c131527599e8288d6918e888d88726f84d805d784b771f32408aeaef/numpy-2.4.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb", size = 14699246, upload-time = "2026-05-18T23:33:57.621Z" }, + { url = "https://files.pythonhosted.org/packages/ad/fe/c0a6b7b2ca128a8fb228575147073b660656734b8ebe4d76c8fd748dcc79/numpy-2.4.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41", size = 5204410, upload-time = "2026-05-18T23:34:00.302Z" }, + { url = "https://files.pythonhosted.org/packages/f3/d4/9770d14ba719432bb90a421bfd443872ed0f70f7264b64bec12ea363d5fd/numpy-2.4.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698", size = 6551240, upload-time = "2026-05-18T23:34:02.852Z" }, + { url = "https://files.pythonhosted.org/packages/c9/c6/50a46a6205feba2343f1d6d17438107c5dc491ed1c736e6ea68689fd906b/numpy-2.4.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f", size = 15671012, upload-time = "2026-05-18T23:34:05.485Z" }, + { url = "https://files.pythonhosted.org/packages/99/60/14115e6364fa676c5397c2ad3004e527e9aa487abf5d0706ec81bbd08529/numpy-2.4.6-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853", size = 16645538, upload-time = "2026-05-18T23:34:09.265Z" }, + { url = "https://files.pythonhosted.org/packages/ae/c5/693cbe59e57db94d2231fa519ca3978dc9e19da5a8f088588f5c6e947ff2/numpy-2.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a", size = 17020706, upload-time = "2026-05-18T23:34:13.053Z" }, + { url = "https://files.pythonhosted.org/packages/ef/fc/85b7c4eff9b4966ade25c2273cf7e7012e92366c032058653934b37de044/numpy-2.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2", size = 18368541, upload-time = "2026-05-18T23:34:17.024Z" }, + { url = "https://files.pythonhosted.org/packages/f6/81/e1b27545deedce7f4a0b348618c6b62d74e36a4dc9ccd42f3eb2f85eee32/numpy-2.4.6-cp312-cp312-win32.whl", hash = "sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45", size = 5962825, upload-time = "2026-05-18T23:34:20.3Z" }, + { url = "https://files.pythonhosted.org/packages/ab/ca/feab00bd44aa5fe1ad2c18f08b4d3bb92e26484b0b1d1443897809ed528c/numpy-2.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751", size = 12321687, upload-time = "2026-05-18T23:34:23.095Z" }, + { url = "https://files.pythonhosted.org/packages/63/cf/5a6d34850a39d1093558564f77ee8e8e0bee5061151b8f05a55711001ec7/numpy-2.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8", size = 10221482, upload-time = "2026-05-18T23:34:25.876Z" }, + { url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" }, + { url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" }, + { url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" }, + { url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" }, + { url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" }, + { url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" }, + { url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" }, + { url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" }, + { url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" }, + { url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" }, + { url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" }, + { url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" }, + { url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" }, + { url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" }, + { url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" }, + { url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" }, + { url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" }, + { url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" }, + { url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" }, + { url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" }, + { url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" }, + { url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" }, + { url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" }, + { url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" }, + { url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" }, + { url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" }, + { url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" }, + { url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" }, + { url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" }, + { url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" }, + { url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" }, + { url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" }, + { url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" }, + { url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" }, + { url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" }, + { url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" }, + { url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" }, + { url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" }, + { url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" }, + { url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" }, + { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" }, + { url = "https://files.pythonhosted.org/packages/de/12/b422cc84439adc0d00de605bf4a308890ae5c26f2c71fbd73e5d08fbb0dd/numpy-2.4.6-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662", size = 16847511, upload-time = "2026-05-18T23:36:50.673Z" }, + { url = "https://files.pythonhosted.org/packages/44/53/f481bef68011740f8849418d82db07230e825013f31f4eef5ba5b805316a/numpy-2.4.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7", size = 14889064, upload-time = "2026-05-18T23:36:53.879Z" }, + { url = "https://files.pythonhosted.org/packages/7f/57/42ed575c10ced8af951d426bc4e1f8aff16fd851db33f067036215a7f860/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f", size = 5394157, upload-time = "2026-05-18T23:36:57.194Z" }, + { url = "https://files.pythonhosted.org/packages/6a/ef/f66cc724fcc36c1e364c67f51ae9146090b8b584f27d58b97fdae3edd737/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c", size = 6708728, upload-time = "2026-05-18T23:36:59.575Z" }, + { url = "https://files.pythonhosted.org/packages/1a/9c/c531f2293b91265d8b48e9b329f54fdd7ffae73cb4134ea10cca4237e9cc/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0", size = 15798374, upload-time = "2026-05-18T23:37:02.674Z" }, + { url = "https://files.pythonhosted.org/packages/1a/b0/413077f6b1153ed3cba361401c6783bbad6114804a000cc22eb71c13e190/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02", size = 16747286, upload-time = "2026-05-18T23:37:06.327Z" }, + { url = "https://files.pythonhosted.org/packages/15/ce/e5ec180bc41812edcd8daeb8639d205622c0e8c02259d8ab25a0201b3c2a/numpy-2.4.6-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73", size = 12504263, upload-time = "2026-05-18T23:37:09.715Z" }, +] + +[[package]] +name = "numpy" +version = "2.5.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14' and sys_platform == 'win32'", + "python_full_version >= '3.14' and sys_platform == 'emscripten'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'", + "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", +] +sdist = { url = "https://files.pythonhosted.org/packages/9a/80/db0b4559e57ec36362bedbb05530a87fafbcb6067708c946967a41d449e7/numpy-2.5.2.tar.gz", hash = "sha256:d482d171c406ae88c5b19cad3b6a1c4c5209f886ab74bc44c2c865c23f52d860", size = 20773161, upload-time = "2026-08-09T13:48:27.962Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/69/72/dccb0aaf40972777283303919f613964227266d0c13adebb79ac124f1c3e/numpy-2.5.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:14e373cfc6387177e8409dac3c7159be8eb05cd77096cd7c950268b86f62831c", size = 16891693, upload-time = "2026-08-09T13:44:51.702Z" }, + { url = "https://files.pythonhosted.org/packages/60/2e/b5aee50a1f74ac815cf8331812cb8251e29024025de462e0c047641c614c/numpy-2.5.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4bbd96c833ecc8cc069ce518078fc8c60cb9cbfb0fea5b7a803ad65035596d03", size = 11903109, upload-time = "2026-08-09T13:44:55.501Z" }, + { url = "https://files.pythonhosted.org/packages/f3/f4/29e78102a80601cf034d4e9767022cffeca2c3b4c926e1754572ca95593d/numpy-2.5.2-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:6e8172ddfcf5cf74b811d372b570b83c60bd2de87a6fbfbebdadb4a9bd9c6cbb", size = 5350202, upload-time = "2026-08-09T13:44:58.401Z" }, + { url = "https://files.pythonhosted.org/packages/11/4b/dcd3b7eadaf4035d2c7a4289d232523a6964f602598ef7674e4bd7291f93/numpy-2.5.2-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:65f188481f1669e26f62b701e8205d19e460fa4a9b52a1414ba382330e4a3414", size = 6687736, upload-time = "2026-08-09T13:45:00.813Z" }, + { url = "https://files.pythonhosted.org/packages/e5/21/4947e0e9d6c9fc2e2ff15b8949049ee44f63adb9cacc729ab8793f97e712/numpy-2.5.2-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8ee9c4eeb8454b3660a8b53493563c3e121c2fc94fbd72b848ef814ed7b676a9", size = 15612696, upload-time = "2026-08-09T13:45:04.151Z" }, + { url = "https://files.pythonhosted.org/packages/3a/5f/62d28cf019460c7f1394105b4d49d9911a9c444cb77ab0bd95a204c5a6de/numpy-2.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3cdec01fa790a186d430433fdd4d4ffb70eed6f0eeb4bf05c8dbe2dce0a9bcb8", size = 16722264, upload-time = "2026-08-09T13:45:07.714Z" }, + { url = "https://files.pythonhosted.org/packages/14/25/3f0be4c1b9fdf5dd5e708a6806978564d7c46a055c000496309ff2a2f8af/numpy-2.5.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:7999d4ddb0c4025018373fd787510d46e04c769467af22869707b3c1cfd459ab", size = 16974396, upload-time = "2026-08-09T13:45:11.316Z" }, + { url = "https://files.pythonhosted.org/packages/22/72/6262cbdeeb45da9d971e40715f579d791603ba8ec0b5e2db1ac55454421d/numpy-2.5.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c1f017dc0875c9209d219f97feceb7d54c2661bb243deb4114478e1295808af7", size = 18476044, upload-time = "2026-08-09T13:45:14.869Z" }, + { url = "https://files.pythonhosted.org/packages/36/33/29208b8b075bde62d26a81d14b358c42b0f69b6cabd98d4ff97f37f22b05/numpy-2.5.2-cp312-cp312-win32.whl", hash = "sha256:d6a48072864e3324e194a8fbb3c657bcc5b5c869dbc64c9537b1d5c862572c0a", size = 6072817, upload-time = "2026-08-09T13:45:17.867Z" }, + { url = "https://files.pythonhosted.org/packages/7f/b9/87fea2769fe1c47c1b5b01d8310772c9d1a85d485de7cf386ef7a3332b02/numpy-2.5.2-cp312-cp312-win_amd64.whl", hash = "sha256:28ac63476ec7651484215ee7fa15a1f78b57c14621f01e392afe17b9a1390ce4", size = 12464674, upload-time = "2026-08-09T13:45:20.734Z" }, + { url = "https://files.pythonhosted.org/packages/14/52/032b97e00461ab0809bbe4c588b035620e5a14b8cdee47ecddefc7b17d33/numpy-2.5.2-cp312-cp312-win_arm64.whl", hash = "sha256:27650bb0e7140fa3d37b9923b4803645e0b125d190f326eecfd3f4dad8e8ade1", size = 10397131, upload-time = "2026-08-09T13:45:23.73Z" }, + { url = "https://files.pythonhosted.org/packages/f5/d2/6b24738a0ef4557d189b150046cd07823c50e4273e8aebd651222e24306f/numpy-2.5.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8e4cb9a754c8a0c62eaa88273a5fba3391f4a610d1dee893c0755da31c083f15", size = 16886595, upload-time = "2026-08-09T13:45:27.323Z" }, + { url = "https://files.pythonhosted.org/packages/65/60/f2d208d366f263f39c6e69ed309290717aab41078b6d04c9be2a84fa2a07/numpy-2.5.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:52c808f96484f5571a5cc863775ce50247c17dfb3b0361f8ed6b4b0456f80080", size = 11896845, upload-time = "2026-08-09T13:45:31.638Z" }, + { url = "https://files.pythonhosted.org/packages/3c/79/81e0bf24f4d020a2b1d5cd297a9f60c3f24eeb116f9bba5870443f7b6a4a/numpy-2.5.2-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:29d81e97f668489cba8ebfd796b9bdd453525d35dd9e162e2daec94bf3fc7740", size = 5343880, upload-time = "2026-08-09T13:45:34.373Z" }, + { url = "https://files.pythonhosted.org/packages/ba/cc/e3141cf06d1a8a2c7e107543fe1269c1d1af760d4d683c0794a4ee1127c2/numpy-2.5.2-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:afb3f0632d6b2e3ba04dbce8d1e48d321b369138b73830b5ca371a0e8d479d56", size = 6682264, upload-time = "2026-08-09T13:45:36.7Z" }, + { url = "https://files.pythonhosted.org/packages/29/f1/2a64a307d92c5d98f5255a4014eb43bb6103ee477087b61ecae44a3aa9b9/numpy-2.5.2-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0aadf13b60048d501e05fa699efaf7734e2494f3498a4c2a5521d822640324f3", size = 15609566, upload-time = "2026-08-09T13:45:39.518Z" }, + { url = "https://files.pythonhosted.org/packages/7b/44/59a1eb68e773c4098d107ef34a0dbdeca501d72ffcfbff9a7707343921ce/numpy-2.5.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:29b86ff8a6cc556b47ec6b64b194815cc80e6bf5eedcc6cddfd65318cb0b4eee", size = 16709995, upload-time = "2026-08-09T13:45:43.661Z" }, + { url = "https://files.pythonhosted.org/packages/8a/4c/3e54d4ddbc359a1295f8b633e8106bcd4d7d4a206e82df051bdfb3058755/numpy-2.5.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6950c4b7dd562453090548ba7f5da7e59f57f85663f15d5dcc60e249192f7e59", size = 16972511, upload-time = "2026-08-09T13:45:47.094Z" }, + { url = "https://files.pythonhosted.org/packages/f2/9f/02e371638ebf19b66d46231e4be52999e87f32d1961b113bc45656608b22/numpy-2.5.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b9727f472d2f3888053b8a75ab0cb94745a9de224bb5846dbadc0092101bc71d", size = 18465609, upload-time = "2026-08-09T13:45:50.808Z" }, + { url = "https://files.pythonhosted.org/packages/eb/ae/ad6645abc7a3510fe48e8ea1ab4598166f500057ef4ebf38bfad4f1577de/numpy-2.5.2-cp313-cp313-win32.whl", hash = "sha256:4f9744f9fbdcea0bc552e8f19e1f141f811a3f9bc2be2cc6e86d982cab23e3f4", size = 6070204, upload-time = "2026-08-09T13:45:54.111Z" }, + { url = "https://files.pythonhosted.org/packages/15/20/f3489f86d81ea460b2bcdceaed094142ca6579f6be0ec527b781d39afe68/numpy-2.5.2-cp313-cp313-win_amd64.whl", hash = "sha256:85aaccb24182c25df891ad0ec333585967e115269d5f1b17f2c9ae005bc96657", size = 12460532, upload-time = "2026-08-09T13:45:57.167Z" }, + { url = "https://files.pythonhosted.org/packages/d5/21/35b31dde1b283b79de828b80f876afd8c94e28fe1e9c375f89e261cc4c0d/numpy-2.5.2-cp313-cp313-win_arm64.whl", hash = "sha256:bd68ece1553d2023c09a4226d9e41c586ad2d20594d1a456186c33513d2cb3f2", size = 10396725, upload-time = "2026-08-09T13:46:00.478Z" }, + { url = "https://files.pythonhosted.org/packages/ac/f8/c3b222bf075b50afd8e949a07a15c4b312a4a84bd8102a332bcd953cbbb4/numpy-2.5.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d787cf769c3baeb5f6235e778edb52c08dfa923789b5958f28e6450f96107cb1", size = 16885180, upload-time = "2026-08-09T13:46:03.939Z" }, + { url = "https://files.pythonhosted.org/packages/17/e1/2c1d4b1987795a92b5bbf7c24fe249ab96aa2573ab0d7604802c189d7b86/numpy-2.5.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:24b9dc2e3d84aa58523798805194e23e736f3f6ce2d1a5b92583ae734e6dbda8", size = 11907878, upload-time = "2026-08-09T13:46:07.045Z" }, + { url = "https://files.pythonhosted.org/packages/b9/ee/d08226fc858044355983a6e5b94f08ff6f3969e0a2b160a4a89f0ddb3445/numpy-2.5.2-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:9e9413326d726c2545bfa65d2c0876871e8d8386e77f992c1d426e180bbd4323", size = 5354922, upload-time = "2026-08-09T13:46:10.04Z" }, + { url = "https://files.pythonhosted.org/packages/94/f0/6d3d933056440ebbc5e6bad92065fc6c26a48a84a36b1208580e94eea76c/numpy-2.5.2-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:60e902ac295855348a5ca2ea4c89108989a9f5fddfad3dfc0a8f36b10358567e", size = 6679168, upload-time = "2026-08-09T13:46:12.275Z" }, + { url = "https://files.pythonhosted.org/packages/c4/3b/ecd49dd90033cceb2704d88ca905d4d7d89b0e8c739608754ffd325fa820/numpy-2.5.2-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:50e500dc868e9313530ce12ba470fe50ff3afe3d62993ed6eff652dacd555b65", size = 15624501, upload-time = "2026-08-09T13:46:15.322Z" }, + { url = "https://files.pythonhosted.org/packages/c7/99/461bd36dbdfac6c1c53efa370bd55a83227542d0d118f1677dbf1a3dacd5/numpy-2.5.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:318b9a4c845dbea06708a29c84ee429cc3065048db34cdb799047643492050ee", size = 16713701, upload-time = "2026-08-09T13:46:18.949Z" }, + { url = "https://files.pythonhosted.org/packages/f9/9c/2b251df9e8a5d647b62b0cbc1b90a91850c1cf4859ecb532fd0b4eacff6c/numpy-2.5.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:34c319e2963be042673fb46570501b2f06c41924e17e3563d58646b4380dfb68", size = 16986065, upload-time = "2026-08-09T13:46:23.006Z" }, + { url = "https://files.pythonhosted.org/packages/8f/25/20de43f53ff1390534a124475055a19f01fe10c920a0fd11b8e18d6d6052/numpy-2.5.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f06571a052127dc1b4e8b83029b4d1b20daa2b64a31cdd181fc6bc774e9000eb", size = 18470031, upload-time = "2026-08-09T13:46:27.102Z" }, + { url = "https://files.pythonhosted.org/packages/56/5e/0c577ca308d6da5eb79b546ba10bbe5b60148192194e2da060913b1de4f1/numpy-2.5.2-cp314-cp314-win32.whl", hash = "sha256:2cc779226e476d1e1f08c74068c419e60f41a9e0e069c92f6671d31d5c985e98", size = 6121028, upload-time = "2026-08-09T13:46:30.046Z" }, + { url = "https://files.pythonhosted.org/packages/15/5c/7bcbd5b11f94199073320410cddcbb80cee62415bfeb540874b265c2d922/numpy-2.5.2-cp314-cp314-win_amd64.whl", hash = "sha256:7587f53dfbd5edc0f7b87c6217b4c6d2d1f2ef9c3da70bc1315e7db5f8d7ec9d", size = 12597627, upload-time = "2026-08-09T13:46:32.886Z" }, + { url = "https://files.pythonhosted.org/packages/87/bc/4d0b06fba0da90ccc75af62823cb9dcedb6c9ea0cffa058cb2c9ee773a77/numpy-2.5.2-cp314-cp314-win_arm64.whl", hash = "sha256:3e4c367352d3747784248a227fbec218e193b56f7e6692e3b64fc805478ecfdf", size = 10680414, upload-time = "2026-08-09T13:46:36.036Z" }, + { url = "https://files.pythonhosted.org/packages/cd/17/f429aac9dc08833a0d0f188eba38c532a751b1a1f2ca6018a37b455cb321/numpy-2.5.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:b879fb674276e331513fb136b78dbc6bd3c848309e0d841cfd63be3896c4cfc1", size = 12026967, upload-time = "2026-08-09T13:46:39.084Z" }, + { url = "https://files.pythonhosted.org/packages/ca/9f/d0849de96a2a4ceaa16662f18ee13eaa9c0aa418269fdc8c4857c56b11da/numpy-2.5.2-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:fd0d703772bba096843785bd38371e31bb4a0c1151497ad5739d182114a73f7f", size = 5473874, upload-time = "2026-08-09T13:46:42.075Z" }, + { url = "https://files.pythonhosted.org/packages/89/3c/8df216d4a4a5422a3de045301cf7df8ea47286d76f5cb7160b0128ac26b7/numpy-2.5.2-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:3a2f061cebd9e3d23bdcfaaded5e2293a4c6a5b60fa42df85d410a725ce621bf", size = 6789276, upload-time = "2026-08-09T13:46:44.387Z" }, + { url = "https://files.pythonhosted.org/packages/e6/3a/20d7e9891c4ddfadd6ff8d95bf4b29f353d8e1770553de2099880551dfb9/numpy-2.5.2-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6df895598c0edcb41030126c89e0f353b07d93238116143b7405e937359736c4", size = 15659154, upload-time = "2026-08-09T13:46:47.538Z" }, + { url = "https://files.pythonhosted.org/packages/aa/d6/f3aa3d2688bf501b858835c6bd087ae9b51a56ae6fca8e2b0990abd177af/numpy-2.5.2-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1ab3d4a901f844ea836c3e80bf463c6a27d7f3c14e8e292fcf28d348b25b9bce", size = 16748909, upload-time = "2026-08-09T13:46:51.442Z" }, + { url = "https://files.pythonhosted.org/packages/7d/8f/1c5cae8d2baf86ab802ae97a00be55bc7e21ebc11b12bbc33376c5f05342/numpy-2.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:cebc2d6dbb605a7703d59751dea4bd6b0ab127a5a4338a6f432df1936fef8b26", size = 17027685, upload-time = "2026-08-09T13:46:55.095Z" }, + { url = "https://files.pythonhosted.org/packages/5c/27/71d3467404aedc1c24ce79610f91b52b0b0f466c43a701aa56fc75c145ab/numpy-2.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:eaca7ff36f0f52e2111ec71f169d8fd3e889e7ddc0d2592e0d703fd8d3ce8fac", size = 18501181, upload-time = "2026-08-09T13:46:59.09Z" }, + { url = "https://files.pythonhosted.org/packages/14/2f/42921d27c40aea7e077f4a423ae509fd9220b028cd787bafefd8ab2b3a5f/numpy-2.5.2-cp314-cp314t-win32.whl", hash = "sha256:ddf47472af2e4280d79bac82304f5e80150211f1b9e614b760061d5fdfbb6eba", size = 6271085, upload-time = "2026-08-09T13:47:01.903Z" }, + { url = "https://files.pythonhosted.org/packages/75/e6/bad5f5d56de9b1971bac959963dda276d35c40f1854475005434bbe08692/numpy-2.5.2-cp314-cp314t-win_amd64.whl", hash = "sha256:44ef9675d908e65f9953063837c3277730f3f4437615a4cdab67b366cabaf884", size = 12787971, upload-time = "2026-08-09T13:47:04.963Z" }, + { url = "https://files.pythonhosted.org/packages/df/05/f608795cb34391acd67e38d94a3c36abd8d8576293a3a80727d7595c372c/numpy-2.5.2-cp314-cp314t-win_arm64.whl", hash = "sha256:eaa088384c46f519dacb93b7ec483a6d6b19a4a2085ae4f25ab9b1c43d387d1e", size = 10750306, upload-time = "2026-08-09T13:47:07.976Z" }, + { url = "https://files.pythonhosted.org/packages/33/c6/28de0191c5f82b7d42a0a51390ba98587048aa93a39fafb05bdbe6e8d00c/numpy-2.5.2-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:078f9b027b478c9379b9677babbf0f8b8f1ecfada27636d7b9a93990c638739f", size = 16885274, upload-time = "2026-08-09T13:47:11.439Z" }, + { url = "https://files.pythonhosted.org/packages/dd/d1/973ca116000d244897e468ea1aff30b589e5022e3c8744b71706fe33bd57/numpy-2.5.2-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:50a68f4bacd8a2b33d8da3d2269d0d78500f86ea582e4786dc10f5ef2c2c6842", size = 11907846, upload-time = "2026-08-09T13:47:15.128Z" }, + { url = "https://files.pythonhosted.org/packages/78/d9/8c4b3937ef204cb2fd88d389ccd0f265a2ffb11f35a01d2064cf46714bd6/numpy-2.5.2-cp315-cp315-macosx_14_0_arm64.whl", hash = "sha256:e79aba74ffaf5f78a050d777c184cddf8fdffabab38acf5f3ef1fecbc17895d6", size = 5354892, upload-time = "2026-08-09T13:47:18.07Z" }, + { url = "https://files.pythonhosted.org/packages/74/9b/b6ee65ea2999fdb7023935e108e6fb776ee4082aa15f159acfa857e578c8/numpy-2.5.2-cp315-cp315-macosx_14_0_x86_64.whl", hash = "sha256:9a0731745a72a184490a582fb4af2533512bd071ace67785b5fdffc0ae58dce8", size = 6679309, upload-time = "2026-08-09T13:47:20.456Z" }, + { url = "https://files.pythonhosted.org/packages/43/f3/acb18d8b137a393c8e7803a8c994c9e64bde3930692a69d826993113a159/numpy-2.5.2-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4ec954036759bcee3aa484f8603bd9c14f3e776293b85578b8734c2d72777c69", size = 15625850, upload-time = "2026-08-09T13:47:24.365Z" }, + { url = "https://files.pythonhosted.org/packages/a9/bf/a8e9bb0db815a0e265b5744ebedd3af0bd5faad8604e5b50a1cd012f3c91/numpy-2.5.2-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dc649493697006bc90614a5f0bbc8cb3cb1866715c474e473694968d7e6b99ab", size = 16713664, upload-time = "2026-08-09T13:47:27.965Z" }, + { url = "https://files.pythonhosted.org/packages/0c/c3/6e913736b3dd6582344af32418b5fb9dab34282e8a8174ae1d54ceb0fc13/numpy-2.5.2-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:cf7de32f486e4ac9e2d93b810f9e9ac72a728dd46a32a0bb403222f27f653514", size = 16986749, upload-time = "2026-08-09T13:47:31.541Z" }, + { url = "https://files.pythonhosted.org/packages/80/09/7d3b23eff5c7428ef6c01e6f7052bb60d504c4d33e317b36b8959c24ad97/numpy-2.5.2-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2ffa7bacab3e2ee1b19ed31766bb60bb380b68c23f051e199c5cc598afd68710", size = 18470495, upload-time = "2026-08-09T13:47:35.364Z" }, + { url = "https://files.pythonhosted.org/packages/a5/a4/68a321d825374f6eb677ffe8ef8c6b9a328304e6fd2e39d9530822776607/numpy-2.5.2-cp315-cp315-win32.whl", hash = "sha256:6b588cc8f902d6bff201c19fd00c43ab8545671e3554d014e12e14139e5e8617", size = 6120696, upload-time = "2026-08-09T13:47:38.561Z" }, + { url = "https://files.pythonhosted.org/packages/c8/23/deafbb1700f79fae9cd1e91220f133d124cc267de1b584da3fbf6db2f6cd/numpy-2.5.2-cp315-cp315-win_amd64.whl", hash = "sha256:07d4e89f3a9ab0a9ba24264ccdb642b3dd951b2281e8883a5481a4aa79cc31a7", size = 12597324, upload-time = "2026-08-09T13:47:41.401Z" }, + { url = "https://files.pythonhosted.org/packages/33/cd/3272ba105e3bbbdaeb11357eda31e7a6825ffe159e8171665660299a948f/numpy-2.5.2-cp315-cp315-win_arm64.whl", hash = "sha256:a610dc7e3c52edd39c2bc2375ff9c3fd59cb3ad00e4472d36f83bc1457145788", size = 10680466, upload-time = "2026-08-09T13:47:44.873Z" }, + { url = "https://files.pythonhosted.org/packages/0e/0e/58370637b1bb70a5c9ce2b43f4b521ccb224e36ccb76a6596b17ae4b447c/numpy-2.5.2-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:40f4d451aed46a8046a1aae41c4e55fb3612273df9c502480135e1501576a34b", size = 16993947, upload-time = "2026-08-09T13:47:48.97Z" }, + { url = "https://files.pythonhosted.org/packages/10/93/2abcb807712b289d6d60fe4cf30532f98974a8396d885650f3ba5a13026e/numpy-2.5.2-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:c081cbe16ba1ab53078e5ff29013621e33c509eedab055775d956427712c236e", size = 12025331, upload-time = "2026-08-09T13:47:52.646Z" }, + { url = "https://files.pythonhosted.org/packages/8b/3a/2898e003a5fbaf87e76c039b4ee1f5eb390471b4ffe74887c1f34c4e791e/numpy-2.5.2-cp315-cp315t-macosx_14_0_arm64.whl", hash = "sha256:0090ccdd57ec2703e9b49d0bf554767370581c1dd0a6b2bb2b2d9def317d042a", size = 5472336, upload-time = "2026-08-09T13:47:55.403Z" }, + { url = "https://files.pythonhosted.org/packages/61/a5/23f69d07c544597b29758b31b55c27dc9d541012a2c1496189fef702aec2/numpy-2.5.2-cp315-cp315t-macosx_14_0_x86_64.whl", hash = "sha256:6a9bb119fb8dd21ba30b3f0e555b7e2b081bd9883af21ec9c1c633d161cda3a8", size = 6788387, upload-time = "2026-08-09T13:47:58.192Z" }, + { url = "https://files.pythonhosted.org/packages/15/ea/c0dbdbcf22f43782510a3e492dd3da73c6112b69cac8929d16d127536fc4/numpy-2.5.2-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a839318485284a6fb31be4f8f2c91c8f2cb22f4543c4a8903f12b0671ffe07cc", size = 15667096, upload-time = "2026-08-09T13:48:01.562Z" }, + { url = "https://files.pythonhosted.org/packages/fc/5e/29c73c31748cdb0f7566642125ba17fd5b56780cddf891b085dab27e4466/numpy-2.5.2-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba0a474801b8dc67b66bf465548abc90e82b44d2611b5770f33008dcabffe8ec", size = 16751730, upload-time = "2026-08-09T13:48:05.706Z" }, + { url = "https://files.pythonhosted.org/packages/47/95/02501e8454796bb58dadf7a99d3181e0b464bf264e1003039572f9779fac/numpy-2.5.2-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:0a4035ae1129ff8777f08bfbd44f1e5d8e9c049ce0c2dd78fc0d92c13e7251c0", size = 17038686, upload-time = "2026-08-09T13:48:09.627Z" }, + { url = "https://files.pythonhosted.org/packages/0e/b5/53a681d91b5c82687067d8ea5035e02d917b5509d6f334cb06484a954714/numpy-2.5.2-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:77843ca236b777e67f8d6b3660ea116e499612703a0ecd7093f316201eb9d8e2", size = 18507727, upload-time = "2026-08-09T13:48:13.744Z" }, + { url = "https://files.pythonhosted.org/packages/42/06/6e11443f7b64ee376c860506091103bf68f92d2cab9e8d96d4501babf07c/numpy-2.5.2-cp315-cp315t-win32.whl", hash = "sha256:7354826bc6f8f69402e9b7fe28d15fcd34feebd74f856f111585c5b0c9fb0251", size = 6269775, upload-time = "2026-08-09T13:48:17.543Z" }, + { url = "https://files.pythonhosted.org/packages/f1/18/195d6b86cd72dbbc501edfa778005fa6b87afd34c153e46028cd3a0938f4/numpy-2.5.2-cp315-cp315t-win_amd64.whl", hash = "sha256:e5651f3f87add730ee6608d915009e19c911fba0cb000c7e3ea994b7d768eb12", size = 12782559, upload-time = "2026-08-09T13:48:21.023Z" }, + { url = "https://files.pythonhosted.org/packages/b4/07/458c344f0f0c178f4481dad5cca790626ffe4c34eabf9467069d06ee4999/numpy-2.5.2-cp315-cp315t-win_arm64.whl", hash = "sha256:5f8e00be2ec6f45f4e8a41a527f68d44a7d96fee92a650e4d8b1326f77f61e6e", size = 10748103, upload-time = "2026-08-09T13:48:24.21Z" }, +] + +[[package]] +name = "openai" +version = "3.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio" }, + { name = "distro" }, + { name = "httpx2" }, + { name = "jiter" }, + { name = "pydantic" }, + { name = "sniffio" }, + { name = "tqdm" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/54/8c/2f500e8be09d1ae98c530467962535198b02cd4550cd418bbbaedc8b2910/openai-3.0.0.tar.gz", hash = "sha256:ffd00ef1678d70957e1f1ed98d5bfcf1d661f41ea4482f22e7d0144a66435a49", size = 1123740, upload-time = "2026-08-12T01:55:50.849Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7b/0d/9850e7eddb5e66da4439ed503e78e09ad1fd0195e6df51e4236c75763581/openai-3.0.0-py3-none-any.whl", hash = "sha256:8d32ac3a6647a66910d6cb8a64f0fa5a6c823604b6e82db83d9d055c6709bd51", size = 1665775, upload-time = "2026-08-12T01:55:48.678Z" }, +] + +[[package]] +name = "openpyxl" +version = "3.1.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "et-xmlfile" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/3d/f9/88d94a75de065ea32619465d2f77b29a0469500e99012523b91cc4141cd1/openpyxl-3.1.5.tar.gz", hash = "sha256:cf0e3cf56142039133628b5acffe8ef0c12bc902d2aadd3e0fe5878dc08d1050", size = 186464, upload-time = "2024-06-28T14:03:44.161Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c0/da/977ded879c29cbd04de313843e76868e6e13408a94ed6b987245dc7c8506/openpyxl-3.1.5-py2.py3-none-any.whl", hash = "sha256:5282c12b107bffeef825f4617dc029afaf41d0ea60823bbb665ef3079dc79de2", size = 250910, upload-time = "2024-06-28T14:03:41.161Z" }, +] + +[[package]] +name = "packaging" +version = "26.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" }, +] + +[[package]] +name = "pandas" +version = "3.0.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, + { name = "numpy", version = "2.5.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "python-dateutil" }, + { name = "tzdata", marker = "sys_platform == 'emscripten' or sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/be/4f/5f3422a2afec5ffc46308b79e53291365a93748b498ac2e58bead0197916/pandas-3.0.5.tar.gz", hash = "sha256:dca3734d6ab7c906e6730f0788b0a1dbb9f2467731f9711f77995c8e9d62d712", size = 4658219, upload-time = "2026-07-22T22:19:28.819Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/48/ef/f1fd7431d635bf20015489bf0bd69c17fff1018de773540f651455a3916b/pandas-3.0.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:2946e77e4a53cd248cbde631a12f0e51c8324ce354c3eba4d20147c1ad6f4282", size = 10397178, upload-time = "2026-07-22T22:17:48.274Z" }, + { url = "https://files.pythonhosted.org/packages/31/b4/0eafac990a431561187694126de01f9b12559549b4d86360c0c4bd870fde/pandas-3.0.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:71ecc8fb7ed1a7aa4392316b5309a6347e8e7f832f38fd897846b3a1457a9298", size = 9990736, upload-time = "2026-07-22T22:17:52.388Z" }, + { url = "https://files.pythonhosted.org/packages/de/21/359880af3ea9b7cb23bea5b51e8e70ef3866c03be09da9a2787e18e330a8/pandas-3.0.5-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b173f5951ff6b8b0ec7675e20dff3c97b7e7a57dfcce387c2d7c5afe87cb7899", size = 10814438, upload-time = "2026-07-22T22:17:54.708Z" }, + { url = "https://files.pythonhosted.org/packages/d1/50/d6cc4d7e508bbccf5d6027314a8312bc7ac73d0ec7f195f53838daafab40/pandas-3.0.5-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2c0cf1dd9b55a22d105fc46c1b489af3bd42264fcba7c66297bf47a9a1d9c78a", size = 11323634, upload-time = "2026-07-22T22:17:56.858Z" }, + { url = "https://files.pythonhosted.org/packages/70/2b/d5f0a8c90dd0ae04e64ba53b871afb796ec026b615086d382ddc2ade729b/pandas-3.0.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0fac0010c75e4efb6b99e249c183a8993ce0dc95c240f9b120a5e67c727b7928", size = 11850860, upload-time = "2026-07-22T22:17:59.1Z" }, + { url = "https://files.pythonhosted.org/packages/5c/30/183aec2e19adf778a98d29b5729a0a68f4cc4ebf9b9c3b70d0297355bcb1/pandas-3.0.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:08d24fe11a17dc33bd6e937dc9c665f9cba08fbdc9f657f405713515febe300d", size = 12411100, upload-time = "2026-07-22T22:18:01.485Z" }, + { url = "https://files.pythonhosted.org/packages/fa/9a/31f4983f191af51ab2a8f2d0c7b33dff3a84da26533f982fff02c2f9e28b/pandas-3.0.5-cp311-cp311-win_amd64.whl", hash = "sha256:b1261758dfb6cf12c3cff8300e21cefad30e7ec709abb4c24ac7318e6a52462a", size = 9968804, upload-time = "2026-07-22T22:18:03.903Z" }, + { url = "https://files.pythonhosted.org/packages/49/97/7886c89a39045c69ad82cbceaf3343810480c8ef49a216319ce8183860a6/pandas-3.0.5-cp311-cp311-win_arm64.whl", hash = "sha256:679f4e85b30ddb1515458ab1e788d3e260eae369b1f78da7a3aa4cac8ebf4a2a", size = 9205447, upload-time = "2026-07-22T22:18:06.134Z" }, + { url = "https://files.pythonhosted.org/packages/1c/54/1dc810ea558d1320b597aa140a514f2fdf1d2ea09c38cf556f13ea712ec9/pandas-3.0.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fa290c16964d4963fbfbc358928239cf3bd755b20e988ce944877def2f44471d", size = 10411717, upload-time = "2026-07-22T22:18:08.307Z" }, + { url = "https://files.pythonhosted.org/packages/68/56/fbe81c09195924d8b7b8d4461a20458fe80a6a5ed6b24f0314da684277e1/pandas-3.0.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c2e26bb46934b8a2ca0c3de1d3d606fc5f6746584791b2db264d58cf370e08dc", size = 9957095, upload-time = "2026-07-22T22:18:10.6Z" }, + { url = "https://files.pythonhosted.org/packages/e0/51/fac252f4a913ed5eabf3c11b880a9e8d5a6c10f0b2129d0462212d238b4d/pandas-3.0.5-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:73fa87b08a7ef706f8aafda39ddaccf2a99047bea62d8c88a0361bcafb2237bc", size = 10485458, upload-time = "2026-07-22T22:18:12.834Z" }, + { url = "https://files.pythonhosted.org/packages/12/98/e976540c1addf70442be7842a18cf70884a964abbf69442504f4d2939989/pandas-3.0.5-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d373ce03ffd84010ed9839fa73672a9c8256990532e158440c0085db7d914b34", size = 10998091, upload-time = "2026-07-22T22:18:15.209Z" }, + { url = "https://files.pythonhosted.org/packages/a4/8c/1f29b5be8d3fc47dd7567eb167fabba2085879b31e0287ce7cba6d3d2ff4/pandas-3.0.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2a29c53d85ea98c5e792c59ef82ee9fbe6ca902c0d0adb6b23f45ef894cd7bf6", size = 11499501, upload-time = "2026-07-22T22:18:17.689Z" }, + { url = "https://files.pythonhosted.org/packages/9d/e2/bd9c98ad2df7b38bde002adde4cdf353519da51881634323b126c55997f9/pandas-3.0.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a5ad3b02ed6bc7d7ae9b70804b2c6aa31827489d150f8e623ce82491b82085d7", size = 12060559, upload-time = "2026-07-22T22:18:20.147Z" }, + { url = "https://files.pythonhosted.org/packages/f3/9a/ffbd852d58bd74a617fe2f8ee6a58a96982271ce41cf981eab22190b4a4b/pandas-3.0.5-cp312-cp312-pyemscripten_2024_0_wasm32.whl", hash = "sha256:b2acb4650527eec6822c3dadb2b771277b65e7dae7a267d4bccf65fd1bb3fbce", size = 7197652, upload-time = "2026-07-22T22:18:22.502Z" }, + { url = "https://files.pythonhosted.org/packages/70/b5/d2d3e9ae73362ba4229651b0ee1455cf78073a1ce585f6ff693782ce263e/pandas-3.0.5-cp312-cp312-win_amd64.whl", hash = "sha256:80a611068e8a3ac23f7398c6c14eb46dc974e5cc9997f653e2dcfd1da74edd41", size = 9831691, upload-time = "2026-07-22T22:18:24.534Z" }, + { url = "https://files.pythonhosted.org/packages/52/51/dea1e89d6a6796b9c43f85a09b484ee03edb8a4c4842e73e200a8c11301c/pandas-3.0.5-cp312-cp312-win_arm64.whl", hash = "sha256:25ff585b972a18ef1fe9ffa3ac6544d9950508aa76832e5147640b6022821e49", size = 9105796, upload-time = "2026-07-22T22:18:27.064Z" }, + { url = "https://files.pythonhosted.org/packages/bf/09/7b95c4a0025227d6f118c4039b423412ac6a982db02864166185d812fbc7/pandas-3.0.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:c1c05a767fe8e5b4fe9e1c29806829c582052eaedb9120a3da83ba3f69e24a5b", size = 10385742, upload-time = "2026-07-22T22:18:29.346Z" }, + { url = "https://files.pythonhosted.org/packages/8d/0c/dc78fd8c4da477b4b5e8ad37295af352190d21ef63a9ee1bc071753074cc/pandas-3.0.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:b86765f268b56f7e665b93bce9d5df69dee7f99e595cf8fb839483ab315942a3", size = 9932067, upload-time = "2026-07-22T22:18:31.833Z" }, + { url = "https://files.pythonhosted.org/packages/3e/71/3592c055cf44df9808550f9368ceda80ff2b224d355ef73fe251dcda1802/pandas-3.0.5-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c597ecf5616b5c420372c1d4d4c00dbbfba7398bea857dcc984347e1ea48417b", size = 10466756, upload-time = "2026-07-22T22:18:34.195Z" }, + { url = "https://files.pythonhosted.org/packages/e3/70/4363150359f95b4cb4bcbb34ca23572bb5495749a621a8f3d5a1ddfd293c/pandas-3.0.5-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4b11c36e218331d0387cbe3a0a5f75162357a1d92d57b2b08a336ff94b19b2be", size = 10938525, upload-time = "2026-07-22T22:18:36.81Z" }, + { url = "https://files.pythonhosted.org/packages/f7/d0/317e7a0c67c0e69fa905a0161409397a7dc2d46ff611f6ca4803352c042b/pandas-3.0.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:cf52e1f61d229496da17dc7ab54acdee627357e7008fd4fecba3d0ba2937fa58", size = 11489303, upload-time = "2026-07-22T22:18:39.287Z" }, + { url = "https://files.pythonhosted.org/packages/f1/8d/36dade89b49e4f9d5cbdbe863772581f98c0c6d78fc39ad4c557f6f2e17e/pandas-3.0.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:db172144bb56422bd157812f3b021eacc255451470b31e2c633c349490a1cfee", size = 11989004, upload-time = "2026-07-22T22:18:42.208Z" }, + { url = "https://files.pythonhosted.org/packages/9c/ba/18c4ec8a746e177da05a9e7a7963781d8ea195780724f854601b6ebd6b78/pandas-3.0.5-cp313-cp313-win_amd64.whl", hash = "sha256:0d298e951f23016ce4699951d044ae6418dbc91bf68cefca0f77666fcbb4e5c6", size = 9826896, upload-time = "2026-07-22T22:18:44.539Z" }, + { url = "https://files.pythonhosted.org/packages/de/ec/28a57266b753799a87b8bc79e7887ac6fd981b8c6d2978a0b7e7b6bd708c/pandas-3.0.5-cp313-cp313-win_arm64.whl", hash = "sha256:66266d3442a5e8b3c90274c2b8b230bee42dd1c286bc822cc2f9f2c7e12b883e", size = 9094790, upload-time = "2026-07-22T22:18:47.468Z" }, + { url = "https://files.pythonhosted.org/packages/51/2f/cf6aae281264f4463f0875bcbb15fd2bb6d291cc535187dad1732475e4a9/pandas-3.0.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2f264fc46911cc8131a7322a16199bbf8e353d27c10bb211f5bd0c814324dc36", size = 10390034, upload-time = "2026-07-22T22:18:49.818Z" }, + { url = "https://files.pythonhosted.org/packages/06/ec/5189518c7a7659c4bdcc6b1eb32c46c6f3c86b0661ffd84143d1112c7732/pandas-3.0.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:53730687fcd161883b24e10411c06d6a4c0f2275d2faf3bb2bc25deb4ba8007c", size = 9980065, upload-time = "2026-07-22T22:18:52.249Z" }, + { url = "https://files.pythonhosted.org/packages/ea/f1/598503ce8d7e3c35601e0747ba288c7864baae66380725bc12f13f884dfe/pandas-3.0.5-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:960d3ebcf249f75206899fcd2c6de53f736b7265759ced0d3e559df0b8b709b0", size = 10545532, upload-time = "2026-07-22T22:18:54.813Z" }, + { url = "https://files.pythonhosted.org/packages/fa/de/ceae2adf7034e07e9910299fe412e1819c4f0dd520700a888bcb03625448/pandas-3.0.5-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9e94c2c5ca43bd3ca32bf64d32308887b65e5f9bfd8023ea52755107a999f93b", size = 10963120, upload-time = "2026-07-22T22:18:57.42Z" }, + { url = "https://files.pythonhosted.org/packages/66/25/86e0f4451874eb79e688deeebe3c451fec4557f8952005818d800ee8ac7e/pandas-3.0.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e819dd5f62966b481a8cb649d3299ebd886a1ea91ed5a99bf7ce77c98d18ab94", size = 11563178, upload-time = "2026-07-22T22:18:59.729Z" }, + { url = "https://files.pythonhosted.org/packages/f3/45/8643daa3b4147e433adfcccefdd0380d3aad79d86b15d8999730fe1944d5/pandas-3.0.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:3c5ed2e7c06e91d340dfd091d7934f9bc82e4a36b95f647f090b9d1c9ac649da", size = 12028708, upload-time = "2026-07-22T22:19:02.164Z" }, + { url = "https://files.pythonhosted.org/packages/96/58/ad979ae617615576e8aafd569c9d4b62f1191d896e38f51d66ba06f3b89a/pandas-3.0.5-cp314-cp314-win_amd64.whl", hash = "sha256:cd8f7c6dc98527058ee6264219343f5392240a6f1bfa654fc5d79023020d0c92", size = 9951806, upload-time = "2026-07-22T22:19:04.596Z" }, + { url = "https://files.pythonhosted.org/packages/69/32/7ac03886b304049a9d2625ee88f59af760d8a93bd30ed9239bce7b9869a8/pandas-3.0.5-cp314-cp314-win_arm64.whl", hash = "sha256:5183427f5a8156d480f30333777bc978be93650a49a7c01db26adffe95b31e85", size = 9238297, upload-time = "2026-07-22T22:19:06.836Z" }, + { url = "https://files.pythonhosted.org/packages/be/ed/1d1f2ee5547d5167face2376d11c8b2a4c7bfff5a416ee7a9046891fab1e/pandas-3.0.5-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:303da736987d481074ca720ada325f8bd80c64ebc2d45ed79b29df3aaa4a26ca", size = 10849690, upload-time = "2026-07-22T22:19:09.391Z" }, + { url = "https://files.pythonhosted.org/packages/57/55/17e17152e98fbb0c4b1e562bc65387a2f20a80db0f4a86bf8d3a0e4248d4/pandas-3.0.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3b2801bbb049d0136f6c213eae02b5fca969384fc2064dd728d8620552aa49da", size = 10509945, upload-time = "2026-07-22T22:19:11.773Z" }, + { url = "https://files.pythonhosted.org/packages/88/90/817d44dbf83facf9556f33576d9af0a241981e7bb5c00606c0bcb5df8dda/pandas-3.0.5-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cce3a9d11d2b1f82c69a27ec1f4948a170e2c403c4bbfa8cca62e3fdebe2ef3a", size = 10392197, upload-time = "2026-07-22T22:19:14.024Z" }, + { url = "https://files.pythonhosted.org/packages/f1/da/889f00c0a6f5aa1545add70abbf01502dff87ab577adb855bd631c54d2f2/pandas-3.0.5-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ef01af4d8dc6cd2c8d6c7736f149574ef93fe043811eeb5e445f2647154b5040", size = 10862726, upload-time = "2026-07-22T22:19:16.351Z" }, + { url = "https://files.pythonhosted.org/packages/bc/98/f1e934fb3c98fce859c6147c6785816c7b5b9ab7821115c5d8c4de9842b9/pandas-3.0.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e2759e890db96dfcffdbd9b86c3c2cb6afaf58def482820317e06163ec1066cd", size = 11414864, upload-time = "2026-07-22T22:19:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/fe/be/d448af7d657d82e1888dd8551f79c6d6fb161080b5b9752d84d910ec2319/pandas-3.0.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b58b1b39d46a5862e3fb18f50d1a201398619d16a0f9f73f57eea5583cf0e63c", size = 11925105, upload-time = "2026-07-22T22:19:21.515Z" }, + { url = "https://files.pythonhosted.org/packages/29/c1/ccb4238212c8c4f496c584f3044d94e0c030ed8e1d68999db46c91c2242f/pandas-3.0.5-cp314-cp314t-win_amd64.whl", hash = "sha256:1c10461f6eeb35d8f05b6184c65c8b9991663b66c46b1d559b682cb34ae7c6ea", size = 10387612, upload-time = "2026-07-22T22:19:24.257Z" }, + { url = "https://files.pythonhosted.org/packages/d2/cf/6a51b2c38980e04c279fd2fa908a1b0982064e860444acfca4ec2e2c8359/pandas-3.0.5-cp314-cp314t-win_arm64.whl", hash = "sha256:3c5015fd1730fbf883647e88068176c839c102cea883ba1769a6f4593bfc1f8c", size = 9509776, upload-time = "2026-07-22T22:19:26.694Z" }, +] + +[[package]] +name = "pdfminer-six" +version = "20260107" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "charset-normalizer" }, + { name = "cryptography" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/34/a4/5cec1112009f0439a5ca6afa8ace321f0ab2f48da3255b7a1c8953014670/pdfminer_six-20260107.tar.gz", hash = "sha256:96bfd431e3577a55a0efd25676968ca4ce8fd5b53f14565f85716ff363889602", size = 8512094, upload-time = "2026-01-07T13:29:12.937Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/20/8b/28c4eaec9d6b036a52cb44720408f26b1a143ca9bce76cc19e8f5de00ab4/pdfminer_six-20260107-py3-none-any.whl", hash = "sha256:366585ba97e80dffa8f00cebe303d2f381884d8637af4ce422f1df3ef38111a9", size = 6592252, upload-time = "2026-01-07T13:29:10.742Z" }, +] + +[[package]] +name = "pdfplumber" +version = "0.11.10" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pdfminer-six" }, + { name = "pillow" }, + { name = "pypdfium2" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/05/56/6f450312ba05a27d7713b73857c1a25100dbda04fbc1331b13fb227a607d/pdfplumber-0.11.10.tar.gz", hash = "sha256:b95b2d28c66efb0a794a83b88c6c6aea5987532a445d20a1cbcfa657022e6e57", size = 102892, upload-time = "2026-06-15T03:31:31.035Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a2/9a/07d658e1e7fad860f1c541ab941348125dbdab773be3a0afaf32361866c7/pdfplumber-0.11.10-py3-none-any.whl", hash = "sha256:7741ea81bf165b474b153e6789d10d18e06b6ddcf3ec84289c3ef2fed6802580", size = 60047, upload-time = "2026-06-15T03:31:29.702Z" }, +] + +[[package]] +name = "pillow" +version = "12.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1c/3d/bb7fca845737cf9d7dbde16ed1843984665ff2e0a518f5db43e77ec540b9/pillow-12.3.0.tar.gz", hash = "sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce", size = 47025035, upload-time = "2026-07-01T11:56:38.965Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fb/c8/0a78b0e02d7ac54bc03e5321c9220da52f0c2ea83b21f7c40e7f3169c502/pillow-12.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756", size = 5392415, upload-time = "2026-07-01T11:53:47.162Z" }, + { url = "https://files.pythonhosted.org/packages/b2/5b/a02d30018abd97ced9f5a6c63d28597694a00d066516b9c1c6de45859fc9/pillow-12.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6", size = 4785266, upload-time = "2026-07-01T11:53:49.079Z" }, + { url = "https://files.pythonhosted.org/packages/c8/98/766667a4be768150a202836acd9fad19c06824ca86c4286d3cf6b274964e/pillow-12.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd", size = 6263814, upload-time = "2026-07-01T11:53:51.32Z" }, + { url = "https://files.pythonhosted.org/packages/3b/2d/ede717bc1144f63886c21fd349bb95860b0d1a21149ff16f2bb362b612b6/pillow-12.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd", size = 6934408, upload-time = "2026-07-01T11:53:53.487Z" }, + { url = "https://files.pythonhosted.org/packages/a3/48/9c58b685e69d49c31af6c8eb9012055fab7e665785165c84796e2c73ce72/pillow-12.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c", size = 6337160, upload-time = "2026-07-01T11:53:55.457Z" }, + { url = "https://files.pythonhosted.org/packages/ff/fa/dc2a5c0ba6df93f67c31d34b808b7ce440b40cdbf96f0b81cde1d1e6fa93/pillow-12.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5", size = 7045172, upload-time = "2026-07-01T11:53:57.736Z" }, + { url = "https://files.pythonhosted.org/packages/86/a5/444817a4d4c4c2417df00513086ca196f388d8f9ef40c2e4ccd1ad1af54b/pillow-12.3.0-cp311-cp311-win32.whl", hash = "sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b", size = 6472232, upload-time = "2026-07-01T11:53:59.767Z" }, + { url = "https://files.pythonhosted.org/packages/63/c6/4bad1b18d132a50b27e1365e1ab163616f7a5bb56d330f66f9d1d9d4f9d4/pillow-12.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a", size = 7233653, upload-time = "2026-07-01T11:54:02.066Z" }, + { url = "https://files.pythonhosted.org/packages/fd/16/00f91ab7760dc842f5aad55217e80fc4a7067a0604535249bc8a2d6d9870/pillow-12.3.0-cp311-cp311-win_arm64.whl", hash = "sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26", size = 2568195, upload-time = "2026-07-01T11:54:04.622Z" }, + { url = "https://files.pythonhosted.org/packages/37/bf/fb3ebff8ddcb76aac5a01389251bbbb9519922a9b520d8247c1ca864a25d/pillow-12.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965", size = 5345969, upload-time = "2026-07-01T11:54:06.397Z" }, + { url = "https://files.pythonhosted.org/packages/d8/66/9a386a92561f402389a4fc70c18838bf6d35eb5eb5c6850b4b2dc64f5048/pillow-12.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7", size = 4780323, upload-time = "2026-07-01T11:54:09.351Z" }, + { url = "https://files.pythonhosted.org/packages/25/27/ac8f99618ffd3dde21db0f4d4b1d2ab00c0880595bfd17df103f7f39fd0c/pillow-12.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9", size = 6266838, upload-time = "2026-07-01T11:54:11.71Z" }, + { url = "https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91", size = 6940830, upload-time = "2026-07-01T11:54:13.732Z" }, + { url = "https://files.pythonhosted.org/packages/eb/51/8b08617af3ad95e33ce6d7dd2c99ed6c8298f7fb131636303956be022e25/pillow-12.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c", size = 6344383, upload-time = "2026-07-01T11:54:15.756Z" }, + { url = "https://files.pythonhosted.org/packages/1d/72/cf78ac9780bb93c28328f408973845a309d4d145041665f734572ced1b52/pillow-12.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df", size = 7052934, upload-time = "2026-07-01T11:54:17.721Z" }, + { url = "https://files.pythonhosted.org/packages/20/20/25e0f4dc178a6bc0696793720055519a0de89e7661dae886992decbd2f81/pillow-12.3.0-cp312-cp312-win32.whl", hash = "sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f", size = 6472684, upload-time = "2026-07-01T11:54:19.839Z" }, + { url = "https://files.pythonhosted.org/packages/45/89/da2f7971a317f83d807fdd4065c0af40208e59e692cc43d315a71a0e96d1/pillow-12.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09", size = 7227137, upload-time = "2026-07-01T11:54:22.025Z" }, + { url = "https://files.pythonhosted.org/packages/de/47/4845a0a6c0dbf1db8456bd9fc791f13c5ced7ced20606d08a0aacfd25b49/pillow-12.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510", size = 2568267, upload-time = "2026-07-01T11:54:24.051Z" }, + { url = "https://files.pythonhosted.org/packages/9d/ac/31fb64e1e7efb5a4b50cd3d92049ba89ac6e4d8d3bb6a74e15048ca3353e/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89", size = 4161684, upload-time = "2026-07-01T11:54:25.934Z" }, + { url = "https://files.pythonhosted.org/packages/87/b4/9805e23d2b4d77842b468513841fda254ee42f0289d25088340e4ff46e2d/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace", size = 4255487, upload-time = "2026-07-01T11:54:27.935Z" }, + { url = "https://files.pythonhosted.org/packages/df/39/ecf519435a200c693fe053a6ee4d835b41cf963a4dfc2551c4e637cb2a71/pillow-12.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec", size = 3696433, upload-time = "2026-07-01T11:54:29.813Z" }, + { url = "https://files.pythonhosted.org/packages/42/92/2fc3ffad878ae8dd5469ec1bc8eb83b71f48e13efdf68f02709003982a32/pillow-12.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66", size = 5345889, upload-time = "2026-07-01T11:54:31.97Z" }, + { url = "https://files.pythonhosted.org/packages/10/76/8803c13605b763d33d156c4678fc77f8443389c0c51c8aef707bb02015f4/pillow-12.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35", size = 4780109, upload-time = "2026-07-01T11:54:34.026Z" }, + { url = "https://files.pythonhosted.org/packages/1f/01/e18aff37cb0b4aac47ac90f016d347a49aca667ef97f190b06ac2aabc928/pillow-12.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65", size = 6263736, upload-time = "2026-07-01T11:54:36.131Z" }, + { url = "https://files.pythonhosted.org/packages/f7/62/de5bdd77d935331f4f802edc11e4d82950f642caad6cb2f949837b8560e2/pillow-12.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3", size = 6937129, upload-time = "2026-07-01T11:54:38.216Z" }, + { url = "https://files.pythonhosted.org/packages/70/4d/105627a13300c5e0df1d174230b32fd1273062c96f7745fd552b945d1e1d/pillow-12.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a", size = 6339562, upload-time = "2026-07-01T11:54:40.354Z" }, + { url = "https://files.pythonhosted.org/packages/6b/1d/f13de01a553988ab895ba1c722e06cf3144d4f57656fd5b81b6d881f1179/pillow-12.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e", size = 7049439, upload-time = "2026-07-01T11:54:42.489Z" }, + { url = "https://files.pythonhosted.org/packages/c9/f9/066794cca041b969964f779ee5fa66a9498bbf34248ac39c5d7954e4198f/pillow-12.3.0-cp313-cp313-win32.whl", hash = "sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f", size = 6473287, upload-time = "2026-07-01T11:54:44.9Z" }, + { url = "https://files.pythonhosted.org/packages/a6/9b/7a58e61d62be561da3a356fe2384d4059a6345fc130e23ef1c36a5b81d24/pillow-12.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8", size = 7239691, upload-time = "2026-07-01T11:54:47.141Z" }, + { url = "https://files.pythonhosted.org/packages/aa/b0/c4ed4f0ef8f8fa5ee8351537db6650bb8189f7e118842978dd6589065692/pillow-12.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b", size = 2568185, upload-time = "2026-07-01T11:54:49.137Z" }, + { url = "https://files.pythonhosted.org/packages/dc/01/001f65b68192f0228cc1dbbc8d2530ab5d58b61037ba0587f946fea607cd/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330", size = 4161736, upload-time = "2026-07-01T11:54:51.156Z" }, + { url = "https://files.pythonhosted.org/packages/1a/d2/0219746d0fd16fc8a84498e79452375be3797d3ce4044596ce565164b84f/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217", size = 4255435, upload-time = "2026-07-01T11:54:53.414Z" }, + { url = "https://files.pythonhosted.org/packages/c8/02/8d0bc62ef0302318c46ff2a512822d2610e81c7aa46c9b3abe6cbaca5ad0/pillow-12.3.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930", size = 3696262, upload-time = "2026-07-01T11:54:55.739Z" }, + { url = "https://files.pythonhosted.org/packages/85/e2/73c77d218410b14f5f2d565e8a998d5317b7b9c75368d29985139f7a46f0/pillow-12.3.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8", size = 5350344, upload-time = "2026-07-01T11:54:57.657Z" }, + { url = "https://files.pythonhosted.org/packages/c7/da/32c752228ae345f489e3a42499d817b6c3996da7e8a3bc7a04fc806b243b/pillow-12.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0", size = 4780131, upload-time = "2026-07-01T11:54:59.713Z" }, + { url = "https://files.pythonhosted.org/packages/b1/9d/8b2c807dbef61a5197c047afe99823787eb66f63daf9fb2432f91d6f0462/pillow-12.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321", size = 6263757, upload-time = "2026-07-01T11:55:01.778Z" }, + { url = "https://files.pythonhosted.org/packages/5c/44/c85361f65dbe00eea8576ee467c768d25129989efb76e94f205e9ca9bb46/pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b", size = 6936962, upload-time = "2026-07-01T11:55:03.93Z" }, + { url = "https://files.pythonhosted.org/packages/18/7e/e483414b35800b86b6f08dbbc7803fb5cd52c4d6f897f47d53ea2c7e6f65/pillow-12.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198", size = 6339171, upload-time = "2026-07-01T11:55:05.989Z" }, + { url = "https://files.pythonhosted.org/packages/f0/f4/68c491844841ede6bed70189546b3ee9731cf9f2cbad396faff5e1ccba45/pillow-12.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130", size = 7048116, upload-time = "2026-07-01T11:55:08.131Z" }, + { url = "https://files.pythonhosted.org/packages/a3/34/77f3f793fed8efc7d243f21b33c5a3f0d1c97ee70346d3db855587e155ff/pillow-12.3.0-cp314-cp314-win32.whl", hash = "sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a", size = 6467209, upload-time = "2026-07-01T11:55:10.408Z" }, + { url = "https://files.pythonhosted.org/packages/f1/e0/492879f69d94f91f60fc8cd05ba03650e9520afebb2fb7aa12777d7c7f38/pillow-12.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d", size = 7237707, upload-time = "2026-07-01T11:55:12.745Z" }, + { url = "https://files.pythonhosted.org/packages/c9/ac/6b11f2875f1c2ac040d84e1bbf9cf22a88038f901ca1037898b280b38365/pillow-12.3.0-cp314-cp314-win_arm64.whl", hash = "sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838", size = 2565995, upload-time = "2026-07-01T11:55:14.736Z" }, + { url = "https://files.pythonhosted.org/packages/52/69/c2208e56af9bfc1913afb24020297a691eb1d4ef688474c8a04913f65e04/pillow-12.3.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e", size = 5352503, upload-time = "2026-07-01T11:55:17.076Z" }, + { url = "https://files.pythonhosted.org/packages/07/70/e5686d753e898a45d778ff1718dba8516ead6ab6b95d85fc8c4b70650cf2/pillow-12.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17", size = 4782956, upload-time = "2026-07-01T11:55:19.448Z" }, + { url = "https://files.pythonhosted.org/packages/d5/37/25c6692f06927ee973ff18c8d9ee98ad0b4d84ee67a09610c2dd1447958e/pillow-12.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385", size = 6322855, upload-time = "2026-07-01T11:55:21.613Z" }, + { url = "https://files.pythonhosted.org/packages/cc/91/420637fcb8f1bc11029e403b4538e6694744428d8246118e45719f944556/pillow-12.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c", size = 6989642, upload-time = "2026-07-01T11:55:24.006Z" }, + { url = "https://files.pythonhosted.org/packages/10/08/b94d7811281ccf0d143a1cf768d1c49e1e54af63e7b708ab2ee3eb87face/pillow-12.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d", size = 6391281, upload-time = "2026-07-01T11:55:26.252Z" }, + { url = "https://files.pythonhosted.org/packages/d2/87/24233f785f55474dc02ce3e739c5528a77e3a862e9333d1dd7a25cc31f70/pillow-12.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931", size = 7096716, upload-time = "2026-07-01T11:55:28.318Z" }, + { url = "https://files.pythonhosted.org/packages/23/26/fcb2f6e37175b04f53570b59937867e2b80ee1685e744023153028fc14f9/pillow-12.3.0-cp314-cp314t-win32.whl", hash = "sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7", size = 6474125, upload-time = "2026-07-01T11:55:30.956Z" }, + { url = "https://files.pythonhosted.org/packages/90/de/3634abee5f1c9e13c56787b7d5517b0ba8d6de51700b95578cf338349c9f/pillow-12.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c", size = 7242939, upload-time = "2026-07-01T11:55:34.044Z" }, + { url = "https://files.pythonhosted.org/packages/ce/2a/fd13f8eb24de5714a6eb444a3d67e2842c6c576e159a43793adf23051351/pillow-12.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45", size = 2567506, upload-time = "2026-07-01T11:55:35.988Z" }, + { url = "https://files.pythonhosted.org/packages/5d/dc/8fdce34ec725a33c81c6ba122b904d6b9024e50ea9ac7bede62fab54506c/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139", size = 4162063, upload-time = "2026-07-01T11:55:37.941Z" }, + { url = "https://files.pythonhosted.org/packages/76/66/2044b9a63d3b84ff048228dfcb7cd9bf0df983e8470971bf7d4c57b693de/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402", size = 4255549, upload-time = "2026-07-01T11:55:40.022Z" }, + { url = "https://files.pythonhosted.org/packages/52/7e/1f67e6f4ece6b582ee4b539decbcc9f848dc245a93ed8cd7338bafef72f1/pillow-12.3.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c", size = 3696331, upload-time = "2026-07-01T11:55:41.98Z" }, + { url = "https://files.pythonhosted.org/packages/12/40/d306fc2c8e4d45d7f175c77edca7063be7b86fe7fe6e68f4353bf71d808c/pillow-12.3.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f", size = 5350370, upload-time = "2026-07-01T11:55:44.028Z" }, + { url = "https://files.pythonhosted.org/packages/dd/44/668fb1437e8ce420f62d6106eb66e44a5971602a4d794615bdf79315d82d/pillow-12.3.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701", size = 4780147, upload-time = "2026-07-01T11:55:46.073Z" }, + { url = "https://files.pythonhosted.org/packages/0c/08/93fa2e70e30a2d81547e481b6ee2bb9522117221fb1e0ce4b5df70967677/pillow-12.3.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace", size = 6273659, upload-time = "2026-07-01T11:55:48.264Z" }, + { url = "https://files.pythonhosted.org/packages/f8/6d/043e96ff814fc31a33077e4cba86082167db520c93632afdf2042febbb0c/pillow-12.3.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4", size = 6947439, upload-time = "2026-07-01T11:55:50.503Z" }, + { url = "https://files.pythonhosted.org/packages/af/92/ba71d2ee2ac0edf3fa33bd9d5ee9ee080da70b1766f3ca3934f9938ddac9/pillow-12.3.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39", size = 6353577, upload-time = "2026-07-01T11:55:52.697Z" }, + { url = "https://files.pythonhosted.org/packages/0f/ce/e63064e2122923ff687c8ad792d0d736a7b3920a56a46982e81a7fdd25d6/pillow-12.3.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71", size = 7060394, upload-time = "2026-07-01T11:55:55.149Z" }, + { url = "https://files.pythonhosted.org/packages/54/76/a09cc3ccc8d773a7283d34c38bec1708f9e3cc932093cbc4c5e71ac4060b/pillow-12.3.0-cp315-cp315-win32.whl", hash = "sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827", size = 6467375, upload-time = "2026-07-01T11:55:57.769Z" }, + { url = "https://files.pythonhosted.org/packages/3e/03/1846c49ba3b1d5550392a4bbd06d6fb4578e1cd91a803198b5c90f5f7d53/pillow-12.3.0-cp315-cp315-win_amd64.whl", hash = "sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5", size = 7237048, upload-time = "2026-07-01T11:55:59.975Z" }, + { url = "https://files.pythonhosted.org/packages/fb/bb/89f35dcc79610423f9f195504d7def7f0d1416a711541b42867e25fe3412/pillow-12.3.0-cp315-cp315-win_arm64.whl", hash = "sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658", size = 2566006, upload-time = "2026-07-01T11:56:02.143Z" }, + { url = "https://files.pythonhosted.org/packages/30/88/707027ba09942dfa2c28759b5c222d769290a41c6d20ea60ec250801941f/pillow-12.3.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf", size = 5352509, upload-time = "2026-07-01T11:56:04.2Z" }, + { url = "https://files.pythonhosted.org/packages/b0/6d/00352fa25332c2569cd387851f568cc5a4b75a9adbfb37ac4fbce4c02eec/pillow-12.3.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64", size = 4783167, upload-time = "2026-07-01T11:56:06.631Z" }, + { url = "https://files.pythonhosted.org/packages/13/4f/9e049dfa21af7c22427275720e2490267ba8138120add5c4c574deb69782/pillow-12.3.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e", size = 6329237, upload-time = "2026-07-01T11:56:08.868Z" }, + { url = "https://files.pythonhosted.org/packages/36/16/cf6eeaae8d0fce8dd390a33437cf68c5d5bd73834a2bc6e2f14efda0ab45/pillow-12.3.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777", size = 6997047, upload-time = "2026-07-01T11:56:11.379Z" }, + { url = "https://files.pythonhosted.org/packages/1e/69/dbf769bdd55f48bf5733cac28edc6364ffaa072ec9ba336266e4fe66be55/pillow-12.3.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1", size = 6400440, upload-time = "2026-07-01T11:56:13.908Z" }, + { url = "https://files.pythonhosted.org/packages/a0/e1/ffc9cfc2eea0d178da8018e18e959301ad9d6bc9f3edb7181e748a474b97/pillow-12.3.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9", size = 7105895, upload-time = "2026-07-01T11:56:16.575Z" }, + { url = "https://files.pythonhosted.org/packages/18/f0/a5595c1e8c3ae44b9828cb2f0fa8155e5095ef04d6327b8f61cf44a3df85/pillow-12.3.0-cp315-cp315t-win32.whl", hash = "sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8", size = 6474384, upload-time = "2026-07-01T11:56:18.855Z" }, + { url = "https://files.pythonhosted.org/packages/e4/04/62bcd9f844984c5938d3b05264a61d797a29d3e0812341a8204af70bbdee/pillow-12.3.0-cp315-cp315t-win_amd64.whl", hash = "sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418", size = 7243537, upload-time = "2026-07-01T11:56:21.214Z" }, + { url = "https://files.pythonhosted.org/packages/3d/68/1f3066acedf37673694a7141381d8f811ae97f30d34413d236abe7d489f1/pillow-12.3.0-cp315-cp315t-win_arm64.whl", hash = "sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59", size = 2567491, upload-time = "2026-07-01T11:56:23.506Z" }, + { url = "https://files.pythonhosted.org/packages/75/18/2e8b40223153ccbc60df07f9e8928dc0c76202aa4e55ae9f53962b6510d6/pillow-12.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468", size = 5302510, upload-time = "2026-07-01T11:56:25.736Z" }, + { url = "https://files.pythonhosted.org/packages/46/3e/51fabf59d5ab801ceab709453d3ab6b180083496579549de4c45ced6528a/pillow-12.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94", size = 4736058, upload-time = "2026-07-01T11:56:28.041Z" }, + { url = "https://files.pythonhosted.org/packages/bf/20/22fe9384b7949e25fb1293bcfc84fb82590ff4ea6b37c95b24d26d793d86/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e", size = 5237776, upload-time = "2026-07-01T11:56:30.263Z" }, + { url = "https://files.pythonhosted.org/packages/08/14/f6ba68107680ffa74b39985f3f30884e41318fbc4250caa423c79b4788bb/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3", size = 5860358, upload-time = "2026-07-01T11:56:32.68Z" }, + { url = "https://files.pythonhosted.org/packages/36/54/0169bc772ec491108b62f644f8ecf1fe5d8ae5ebafde2ee2142210166903/pillow-12.3.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a", size = 7231786, upload-time = "2026-07-01T11:56:35.046Z" }, +] + +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + +[[package]] +name = "pycparser" +version = "3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" }, +] + +[[package]] +name = "pydantic" +version = "2.13.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-types" }, + { name = "pydantic-core" }, + { name = "typing-extensions" }, + { name = "typing-inspection" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/18/a5/b60d21ac674192f8ab0ba4e9fd860690f9b4a6e51ca5df118733b487d8d6/pydantic-2.13.4.tar.gz", hash = "sha256:c40756b57adaa8b1efeeced5c196f3f3b7c435f90e84ea7f443901bec8099ef6", size = 844775, upload-time = "2026-05-06T13:43:05.343Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fd/7b/122376b1fd3c62c1ed9dc80c931ace4844b3c55407b6fb2d199377c9736f/pydantic-2.13.4-py3-none-any.whl", hash = "sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba", size = 472262, upload-time = "2026-05-06T13:43:02.641Z" }, +] + +[[package]] +name = "pydantic-core" +version = "2.46.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9d/56/921726b776ace8d8f5db44c4ef961006580d91dc52b803c489fafd1aa249/pydantic_core-2.46.4.tar.gz", hash = "sha256:62f875393d7f270851f20523dd2e29f082bcc82292d66db2b64ea71f64b6e1c1", size = 471464, upload-time = "2026-05-06T13:37:06.98Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5c/fa/6d7708d2cfc1a832acb6aeb0cd16e801902df8a0f583bb3b4b527fde022e/pydantic_core-2.46.4-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:0e96592440881c74a213e5ad528e2b24d3d4f940de2766bed9010ab1d9e51594", size = 2111872, upload-time = "2026-05-06T13:40:27.596Z" }, + { url = "https://files.pythonhosted.org/packages/ae/6f/aa064a3e74b5745afbdf250594f38e7ead05e2d651bcb35994b9417a0d4d/pydantic_core-2.46.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:e0d65b8c354be7fb5f720c3caa8bc940bc2d20ce749c8e06135f07f8ed95dd7c", size = 1948255, upload-time = "2026-05-06T13:39:12.574Z" }, + { url = "https://files.pythonhosted.org/packages/43/3a/41114a9f7569b84b4d84e7a018c57c56347dac30c0d4a872946ec4e36c46/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7bfb192b3f4b9e8a89b6277b6ce787564f62cfd272055f6e685726b111dc7826", size = 1972827, upload-time = "2026-05-06T13:38:19.841Z" }, + { url = "https://files.pythonhosted.org/packages/ef/25/1ab42e8048fe551934d9884e8d64daa7e990ad386f310a15981aeb6a5b08/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:9037063db01f09b09e237c282b6792bd4da634b5402c4e7f0c61effed7701a04", size = 2041051, upload-time = "2026-05-06T13:38:10.447Z" }, + { url = "https://files.pythonhosted.org/packages/94/c2/1a934597ddf08da410385b3b7aae91956a5a76c635effef456074fad7e88/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:fc010ab034c8c7452522748bf937df58020d256ccae0874463d1f4d01758af8e", size = 2221314, upload-time = "2026-05-06T13:40:13.089Z" }, + { url = "https://files.pythonhosted.org/packages/02/6d/9e8ad178c9c4df27ad3c8f25d1fe2a7ab0d2ba0559fad4aee5d3d1f16771/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8c5dac79fa1614d1e06ca695109c6105923bd9c7d1d6c918d4e637b7e6b32fd3", size = 2285146, upload-time = "2026-05-06T13:38:59.224Z" }, + { url = "https://files.pythonhosted.org/packages/80/50/540cd3aeefc041beb111125c4bff779831a2111fc6b15a9138cda277d32c/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f9fa868638bf362d3d138ea55829cefb3d5f4b0d7f142234382a15e2485dbec4", size = 2089685, upload-time = "2026-05-06T13:38:17.762Z" }, + { url = "https://files.pythonhosted.org/packages/6b/a4/b440ad35f05f6a38f89fa0f149accb3f0e02be94ca5e15f3c449a61b4bc9/pydantic_core-2.46.4-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:17299feefe090f2caa5b8e37222bb5f663e4935a8bfa6931d4102e5df1a9f398", size = 2115420, upload-time = "2026-05-06T13:37:58.195Z" }, + { url = "https://files.pythonhosted.org/packages/99/61/de4f55db8dfd57bfdfa9a12ec90fe1b57c4f41062f7ca86f08586b3e0ac0/pydantic_core-2.46.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4c63ebc82684aa89d9a3bcbd13d515b3be44250dc68dd3bd81526c1cb31286c3", size = 2165122, upload-time = "2026-05-06T13:37:01.167Z" }, + { url = "https://files.pythonhosted.org/packages/f7/52/7c529d7bdb2d1068bd52f51fe32572c8301f9a4febf1948f10639f1436f5/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:aaa2a54443eff1950ba5ddc6b6ccda0d9c84a364276a62f969bdf2a390650848", size = 2182573, upload-time = "2026-05-06T13:38:45.04Z" }, + { url = "https://files.pythonhosted.org/packages/37/b3/7c40325848ba78247f2812dcf9c7274e38cd801820ca6dd9fe63bcfb0eb4/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:18e5ceec2ab67e6d5f1a9085e5a24c9c4e2ac4545730bfe668680bca05e555f3", size = 2317139, upload-time = "2026-05-06T13:37:15.539Z" }, + { url = "https://files.pythonhosted.org/packages/d9/37/f913f81a657c865b75da6c0dbed79876073c2a43b5bd9edbe8da785e4d49/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:a0f62d0a58f4e7da165457e995725421e0064f2255d8eccebc49f41bbc23b109", size = 2360433, upload-time = "2026-05-06T13:37:30.099Z" }, + { url = "https://files.pythonhosted.org/packages/c4/67/6acaa1be2567f9256b056d8477158cac7240813956ce86e49deae8e173b4/pydantic_core-2.46.4-cp311-cp311-win32.whl", hash = "sha256:041bde0a48fd37cf71cab1c9d56d3e8625a3793fef1f7dd232b3ff37e978ecda", size = 1985513, upload-time = "2026-05-06T13:38:15.669Z" }, + { url = "https://files.pythonhosted.org/packages/aa/e6/c505f83dfeda9a2e5c995cfd872949e4d05e12f7feb3dca72f633daefa94/pydantic_core-2.46.4-cp311-cp311-win_amd64.whl", hash = "sha256:6f2eeda33a839975441c86a4119e1383c50b47faf0cbb5176985565c6bb02c33", size = 2071114, upload-time = "2026-05-06T13:40:35.416Z" }, + { url = "https://files.pythonhosted.org/packages/0f/da/7a263a96d965d9d0df5e8de8a475f33495451117035b09acb110288c381f/pydantic_core-2.46.4-cp311-cp311-win_arm64.whl", hash = "sha256:14f4c5d6db102bd796a627bbb3a17b4cf4574b9ae861d8b7c9a9661c6dd3362d", size = 2044298, upload-time = "2026-05-06T13:38:29.754Z" }, + { url = "https://files.pythonhosted.org/packages/ce/8c/af022f0af448d7747c5154288d46b5f2bc5f17366eaa0e23e9aa04d59f3b/pydantic_core-2.46.4-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3245406455a5d98187ec35530fd772b1d799b26667980872c8d4614991e2c4a2", size = 2106158, upload-time = "2026-05-06T13:38:57.215Z" }, + { url = "https://files.pythonhosted.org/packages/19/95/6195171e385007300f0f5574592e467c568becce2d937a0b6804f218bc49/pydantic_core-2.46.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:962ccbab7b642487b1d8b7df90ef677e03134cf1fd8880bf698649b22a69371f", size = 1951724, upload-time = "2026-05-06T13:37:02.697Z" }, + { url = "https://files.pythonhosted.org/packages/8e/bc/f47d1ff9cbb1620e1b5b697eef06010035735f07820180e74178226b27b3/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8233f2947cf85404441fd7e0085f53b10c93e0ee78611099b5c7237e36aacbf7", size = 1975742, upload-time = "2026-05-06T13:37:09.448Z" }, + { url = "https://files.pythonhosted.org/packages/5b/11/9b9a5b0306345664a2da6410877af6e8082481b5884b3ddd78d47c6013ce/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3a233125ac121aa3ffba9a2b59edfc4a985a76092dc8279586ab4b71390875e7", size = 2052418, upload-time = "2026-05-06T13:37:38.234Z" }, + { url = "https://files.pythonhosted.org/packages/f1/b7/a65fec226f5d78fc39f4a13c4cc0c768c22b113438f60c14adc9d2865038/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5b712b53160b79a5850310b912a5ef8e57e56947c8ad690c227f5c9d7e561712", size = 2232274, upload-time = "2026-05-06T13:38:27.753Z" }, + { url = "https://files.pythonhosted.org/packages/68/f0/92039db98b907ef49269a8271f67db9cb78ae2fc68062ef7e4e77adb5f61/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9401557acd873c3a7f3eb9383edef8ac4968f9510e340f4808d427e75667e7b4", size = 2309940, upload-time = "2026-05-06T13:38:05.353Z" }, + { url = "https://files.pythonhosted.org/packages/5f/97/2aab507d3d00ca626e8e57c1eac6a79e4e5fbcc63eb99733ff55d1717f65/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:926c9541b14b12b1681dca8a0b75feb510b06c6341b70a8e500c2fdcff837cce", size = 2094516, upload-time = "2026-05-06T13:39:10.577Z" }, + { url = "https://files.pythonhosted.org/packages/22/37/a8aca44d40d737dde2bc05b3c6c07dff0de07ce6f82e9f3167aeaf4d5dea/pydantic_core-2.46.4-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:56cb4851bcaf3d117eddcef4fe66afd750a50274b0da8e22be256d10e5611987", size = 2136854, upload-time = "2026-05-06T13:40:22.59Z" }, + { url = "https://files.pythonhosted.org/packages/24/99/fcef1b79238c06a8cbec70819ac722ba76e02bc8ada9b0fd66eba40da01b/pydantic_core-2.46.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c68fcd102d71ea85c5b2dfac3f4f8476eff42a9e078fd5faefff6d145063536b", size = 2180306, upload-time = "2026-05-06T13:40:10.666Z" }, + { url = "https://files.pythonhosted.org/packages/ae/6c/fc44000918855b42779d007ae63b0532794739027b2f417321cddbc44f6a/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:b2f69dec1725e79a012d920df1707de5caf7ed5e08f3be4435e25803efc47458", size = 2190044, upload-time = "2026-05-06T13:40:43.231Z" }, + { url = "https://files.pythonhosted.org/packages/6b/65/d9cadc9f1920d7a127ad2edba16c1db7916e59719285cd6c94600b0080ba/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:8d0820e8192167f80d88d64038e609c31452eeca865b4e1d9950a27a4609b00b", size = 2329133, upload-time = "2026-05-06T13:39:57.365Z" }, + { url = "https://files.pythonhosted.org/packages/d0/cf/c873d91679f3a30bcf5e7ac280ce5573483e72295307685120d0d5ad3416/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:fbdb89b3e1c94a30cc5edfce477c6e6a5dc4d8f84665b455c27582f211a1c72c", size = 2374464, upload-time = "2026-05-06T13:38:06.976Z" }, + { url = "https://files.pythonhosted.org/packages/47/bd/6f2fc8188f31bf10590f1e98e7b306336161fac930a8c514cd7bd828c7dc/pydantic_core-2.46.4-cp312-cp312-win32.whl", hash = "sha256:9aa768456404a8bf48a4406685ac2bec8e72b62c69313734fa3b73cf33b3a894", size = 1974823, upload-time = "2026-05-06T13:40:47.985Z" }, + { url = "https://files.pythonhosted.org/packages/40/8c/985c1d41ea1107c2534abd9870e4ed5c8e7669b5c308297835c001e7a1c4/pydantic_core-2.46.4-cp312-cp312-win_amd64.whl", hash = "sha256:e9c26f834c65f5752f3f06cb08cb86a913ceb7274d0db6e267808a708b46bc89", size = 2072919, upload-time = "2026-05-06T13:39:21.153Z" }, + { url = "https://files.pythonhosted.org/packages/c4/ba/f463d006e0c47373ca7ec5e1a261c59dc01ef4d62b2657af925fb0deee3a/pydantic_core-2.46.4-cp312-cp312-win_arm64.whl", hash = "sha256:4fc73cb559bdb54b1134a706a2802a4cddd27a0633f5abb7e53056268751ac6a", size = 2027604, upload-time = "2026-05-06T13:39:03.753Z" }, + { url = "https://files.pythonhosted.org/packages/51/a2/5d30b469c5267a17b39dec53208222f76a8d351dfac4af661888c5aee77d/pydantic_core-2.46.4-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:5d5902252db0d3cedf8d4a1bc68f70eeb430f7e4c7104c8c476753519b423008", size = 2106306, upload-time = "2026-05-06T13:37:48.029Z" }, + { url = "https://files.pythonhosted.org/packages/c1/81/4fa520eaffa8bd7d1525e644cd6d39e7d60b1592bc5b516693c7340b50f1/pydantic_core-2.46.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c94f0688e7b8d0a67abf40e57a7eaaecd17cc9586706a31b76c031f63df052b4", size = 1951906, upload-time = "2026-05-06T13:37:17.012Z" }, + { url = "https://files.pythonhosted.org/packages/03/d5/fd02da45b659668b05923b17ba3a0100a0a3d5541e3bd8fcc4ecb711309e/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f027324c56cd5406ca49c124b0db10e56c69064fec039acc571c29020cc87c76", size = 1976802, upload-time = "2026-05-06T13:37:35.113Z" }, + { url = "https://files.pythonhosted.org/packages/21/f2/95727e1368be3d3ed485eaab7adbd7dda408f33f7a36e8b48e0144002b91/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e739fee756ba1010f8bcccb534252e85a35fe45ae92c295a06059ce58b74ccd3", size = 2052446, upload-time = "2026-05-06T13:37:12.313Z" }, + { url = "https://files.pythonhosted.org/packages/9c/86/5d99feea3f77c7234b8718075b23db11532773c1a0dbd9b9490215dc2eeb/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9d56801be94b86a9da183e5f3766e6310752b99ff647e38b09a9500d88e46e76", size = 2232757, upload-time = "2026-05-06T13:39:01.149Z" }, + { url = "https://files.pythonhosted.org/packages/d2/3a/508ac615935ef7588cf6d9e9b91309fdc2da751af865e02a9098de88258c/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2412e734dcb48da14d4e4006b82b46b74f2518b8a26ee7e58c6844a6cd6d03c4", size = 2309275, upload-time = "2026-05-06T13:37:41.406Z" }, + { url = "https://files.pythonhosted.org/packages/07/f8/41db9de19d7987d6b04715a02b3b40aea467000275d9d758ffaa31af7d50/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9551187363ffc0de2a00b2e47c25aeaeb1020b69b668762966df15fc5659dd5a", size = 2094467, upload-time = "2026-05-06T13:39:18.847Z" }, + { url = "https://files.pythonhosted.org/packages/2c/e2/f35033184cb11d0052daf4416e8e10a502ea2ac006fc4f459aee872727d1/pydantic_core-2.46.4-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:0186750b482eefa11d7f435892b09c5c606193ef3375bcf94aa00ae6bfb66262", size = 2134417, upload-time = "2026-05-06T13:40:17.944Z" }, + { url = "https://files.pythonhosted.org/packages/7e/7b/6ceeb1cc90e193862f444ebe373d8fdf613f0a82572dde03fb10734c6c71/pydantic_core-2.46.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:5855698a4856556d86e8e6cd8434bc3ac0314ee8e12089ae0e143f64c6256e4e", size = 2179782, upload-time = "2026-05-06T13:40:32.618Z" }, + { url = "https://files.pythonhosted.org/packages/5a/f2/c8d7773ede6af08036423a00ae0ceffce266c3c52a096c435d68c896083f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:cbaf13819775b7f769bf4a1f066cb6df7a28d4480081a589828ef190226881cd", size = 2188782, upload-time = "2026-05-06T13:36:51.018Z" }, + { url = "https://files.pythonhosted.org/packages/59/31/0c864784e31f09f05cdd87606f08923b9c9e7f6e51dd27f20f62f975ce9f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:633147d34cf4550417f12e2b1a0383973bdf5cdfde212cb09e9a581cf10820be", size = 2328334, upload-time = "2026-05-06T13:40:37.764Z" }, + { url = "https://files.pythonhosted.org/packages/c2/eb/4f6c8a41efa30baa755590f4141abf3a8c370fab610915733e74134a7270/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:82cf5301172168103724d49a1444d3378cb20cdee30b116a1bd6031236298a5d", size = 2372986, upload-time = "2026-05-06T13:39:34.152Z" }, + { url = "https://files.pythonhosted.org/packages/5b/24/b375a480d53113860c299764bfe9f349a3dc9108b3adc0d7f0d786492ebf/pydantic_core-2.46.4-cp313-cp313-win32.whl", hash = "sha256:9fa8ae11da9e2b3126c6426f147e0fba88d96d65921799bb30c6abd1cb2c97fb", size = 1973693, upload-time = "2026-05-06T13:37:55.072Z" }, + { url = "https://files.pythonhosted.org/packages/7e/e8/cff247591966f2d22ec8c003cd7587e27b7ba7b81ab2fb888e3ab75dc285/pydantic_core-2.46.4-cp313-cp313-win_amd64.whl", hash = "sha256:6b3ace8194b0e5204818c92802dcdca7fc6d88aabbb799d7c795540d9cd6d292", size = 2071819, upload-time = "2026-05-06T13:38:49.139Z" }, + { url = "https://files.pythonhosted.org/packages/c6/1a/f4aee670d5670e9e148e0c82c7db98d780be566c6e6a97ee8035528ca0b3/pydantic_core-2.46.4-cp313-cp313-win_arm64.whl", hash = "sha256:184c081504d17f1c1066e430e117142b2c77d9448a97f7b65c6ac9fd9aee238d", size = 2027411, upload-time = "2026-05-06T13:40:45.796Z" }, + { url = "https://files.pythonhosted.org/packages/8d/74/228a26ddad29c6672b805d9fd78e8d251cd04004fa7eed0e622096cd0250/pydantic_core-2.46.4-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:428e04521a40150c85216fc8b85e8d39fece235a9cf5e383761238c7fa9b96fb", size = 2102079, upload-time = "2026-05-06T13:38:41.019Z" }, + { url = "https://files.pythonhosted.org/packages/ad/1f/8970b150a4b4365623ae00fc88603491f763c627311ae8031e3111356d6e/pydantic_core-2.46.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:23ace664830ee0bfe014a0c7bc248b1f7f25ed7ad103852c317624a1083af462", size = 1952179, upload-time = "2026-05-06T13:36:59.812Z" }, + { url = "https://files.pythonhosted.org/packages/95/30/5211a831ae054928054b2f79731661087a2bc5c01e825c672b3a4a8f1b3e/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce5c1d2a8b27468f433ca974829c44060b8097eedc39933e3c206a90ee49c4a9", size = 1978926, upload-time = "2026-05-06T13:37:39.933Z" }, + { url = "https://files.pythonhosted.org/packages/57/e9/689668733b1eb67adeef047db3c2e8788fcf65a7fd9c9e2b46b7744fe245/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7283d57845ecf5a163403eb0702dfc220cc4fbdd18919cb5ccea4f95ee1cdab4", size = 2046785, upload-time = "2026-05-06T13:38:01.995Z" }, + { url = "https://files.pythonhosted.org/packages/60/d9/6715260422ff50a2109878fd24d948a6c3446bb2664f34ee78cd972b3acd/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8daafc69c93ee8a0204506a3b6b30f586ef54028f52aeeeb5c4cfc5184fd5914", size = 2228733, upload-time = "2026-05-06T13:40:50.371Z" }, + { url = "https://files.pythonhosted.org/packages/18/ae/fdb2f64316afca925640f8e70bb1a564b0ec2721c1389e25b8eb4bf9a299/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd2213145bcc2ba85884d0ac63d222fece9209678f77b9b4d76f054c561adb28", size = 2307534, upload-time = "2026-05-06T13:37:21.531Z" }, + { url = "https://files.pythonhosted.org/packages/89/1d/8eff589b45bb8190a9d12c49cfad0f176a5cbd1534908a6b5125e2886239/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7a5f930472650a82629163023e630d160863fce524c616f4e5186e5de9d9a49b", size = 2099732, upload-time = "2026-05-06T13:39:31.942Z" }, + { url = "https://files.pythonhosted.org/packages/06/d5/ee5a3366637fee41dee51a1fc91562dcf12ddbc68fda34e6b253da2324bb/pydantic_core-2.46.4-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:c1b3f518abeca3aa13c712fd202306e145abf59a18b094a6bafb2d2bbf59192c", size = 2129627, upload-time = "2026-05-06T13:37:25.033Z" }, + { url = "https://files.pythonhosted.org/packages/94/33/2414be571d2c6a6c4d08be21f9292b6d3fdb08949a97b6dfe985017821db/pydantic_core-2.46.4-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1a7dd0b3ee80d90150e3495a3a13ac34dbcbfd4f012996a6a1d8900e91b5c0fb", size = 2179141, upload-time = "2026-05-06T13:37:14.046Z" }, + { url = "https://files.pythonhosted.org/packages/7b/79/7daa95be995be0eecc4cf75064cb33f9bbbfe3fe0158caf2f0d4a996a5c7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:3fb702cd90b0446a3a1c5e470bfa0dd23c0233b676a9099ddcc964fa6ca13898", size = 2184325, upload-time = "2026-05-06T13:36:53.615Z" }, + { url = "https://files.pythonhosted.org/packages/9f/cb/d0a382f5c0de8a222dc61c65348e0ce831b1f68e0a018450d31c2cace3a5/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:b8458003118a712e66286df6a707db01c52c0f52f7db8e4a38f0da1d3b94fc4e", size = 2323990, upload-time = "2026-05-06T13:40:29.971Z" }, + { url = "https://files.pythonhosted.org/packages/05/db/d9ba624cc4a5aced1598e88c04fdbd8310c8a69b9d38b9a3d39ce3a61ed7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:372429a130e469c9cd698925ce5fc50940b7a1336b0d82038e63d5bbc4edc519", size = 2369978, upload-time = "2026-05-06T13:37:23.027Z" }, + { url = "https://files.pythonhosted.org/packages/f2/20/d15df15ba918c423461905802bfd2981c3af0bfa0e40d05e13edbfa48bc3/pydantic_core-2.46.4-cp314-cp314-win32.whl", hash = "sha256:85bb3611ff1802f3ee7fdd7dbff26b56f343fb432d57a4728fdd49b6ef35e2f4", size = 1966354, upload-time = "2026-05-06T13:38:03.499Z" }, + { url = "https://files.pythonhosted.org/packages/fc/b6/6b8de4c0a7d7ab3004c439c80c5c1e0a3e8d78bbae19379b01960383d9e5/pydantic_core-2.46.4-cp314-cp314-win_amd64.whl", hash = "sha256:811ff8e9c313ab425368bcbb36e5c4ebd7108c2bbf4e4089cfbb0b01eff63fac", size = 2072238, upload-time = "2026-05-06T13:39:40.807Z" }, + { url = "https://files.pythonhosted.org/packages/32/36/51eb763beec1f4cf59b1db243a7dcc39cbb41230f050a09b9d69faaf0a48/pydantic_core-2.46.4-cp314-cp314-win_arm64.whl", hash = "sha256:bfec22eab3c8cc2ceec0248aec886624116dc079afa027ecc8ad4a7e62010f8a", size = 2018251, upload-time = "2026-05-06T13:37:26.72Z" }, + { url = "https://files.pythonhosted.org/packages/e8/91/855af51d625b23aa987116a19e231d2aaef9c4a415273ddc189b79a45fee/pydantic_core-2.46.4-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:af8244b2bef6aaad6d92cda81372de7f8c8d36c9f0c3ea36e827c60e7d9467a0", size = 2099593, upload-time = "2026-05-06T13:39:47.682Z" }, + { url = "https://files.pythonhosted.org/packages/fb/1b/8784a54c65edb5f49f0a14d6977cf1b209bba85a4c77445b255c2de58ab3/pydantic_core-2.46.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5a4330cdbc57162e4b3aa303f588ba752257694c9c9be3e7ebb11b4aca659b5d", size = 1935226, upload-time = "2026-05-06T13:40:40.428Z" }, + { url = "https://files.pythonhosted.org/packages/e8/e7/1955d28d1afc56dd4b3ad7cc0cf39df1b9852964cf16e5d13912756d6d6b/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:29c61fc04a3d840155ff08e475a04809278972fe6aef51e2720554e96367e34b", size = 1974605, upload-time = "2026-05-06T13:37:32.029Z" }, + { url = "https://files.pythonhosted.org/packages/93/e2/3fedbf0ba7a22850e6e9fd78117f1c0f10f950182344d8a6c535d468fdd8/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c50f2528cf200c5eed56faf3f4e22fcd5f38c157a8b78576e6ba3168ec35f000", size = 2030777, upload-time = "2026-05-06T13:38:55.239Z" }, + { url = "https://files.pythonhosted.org/packages/f8/61/46be275fcaaba0b4f5b9669dd852267ce1ff616592dccf7a7845588df091/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0cbe8b01f948de4286c74cdd6c667aceb38f5c1e26f0693b3983d9d74887c65e", size = 2236641, upload-time = "2026-05-06T13:37:08.096Z" }, + { url = "https://files.pythonhosted.org/packages/60/db/12e93e46a8bac9988be3c016860f83293daea8c716c029c9ace279036f2f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:617d7e2ca7dcb8c5cf6bcb8c59b8832c94b36196bbf1cbd1bfb56ed341905edd", size = 2286404, upload-time = "2026-05-06T13:40:20.221Z" }, + { url = "https://files.pythonhosted.org/packages/e2/4a/4d8b19008f38d31c53b8219cfedc2e3d5de5fe99d90076b7e767de29274f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7027560ee92211647d0d34e3f7cd6f50da56399d26a9c8ad0da286d3869a53f3", size = 2109219, upload-time = "2026-05-06T13:38:12.153Z" }, + { url = "https://files.pythonhosted.org/packages/88/70/3cbc40978fefb7bb09c6708d40d4ad1a5d70fd7213c3d17f971de868ec1f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:f99626688942fb746e545232e7726926f3be91b5975f8b55327665fafda991c7", size = 2110594, upload-time = "2026-05-06T13:40:02.971Z" }, + { url = "https://files.pythonhosted.org/packages/9d/20/b8d36736216e29491125531685b2f9e61aa5b4b2599893f8268551da3338/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fc3e9034a63de20e15e8ade85358bc6efc614008cab72898b4b4952bea0509ff", size = 2159542, upload-time = "2026-05-06T13:39:27.506Z" }, + { url = "https://files.pythonhosted.org/packages/1d/a2/367df868eb584dacf6bf82a389272406d7178e301c4ac82545ab98bc2dd9/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:97e7cf2be5c77b7d1a9713a05605d49460d02c6078d38d8bef3cbe323c548424", size = 2168146, upload-time = "2026-05-06T13:38:31.93Z" }, + { url = "https://files.pythonhosted.org/packages/c1/b8/4460f77f7e201893f649a29ab355dddd3beee8a97bcb1a320db414f9a06e/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:3bf92c5d0e00fefaab325a4d27828fe6b6e2a21848686b5b60d2d9eeb09d76c6", size = 2306309, upload-time = "2026-05-06T13:37:44.717Z" }, + { url = "https://files.pythonhosted.org/packages/64/c4/be2639293acd87dc8ddbcec41a73cee9b2ebf996fe6d892a1a74e88ad3f7/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:3ecbc122d18468d06ca279dc26a8c2e2d5acb10943bb35e36ae92096dc3b5565", size = 2369736, upload-time = "2026-05-06T13:37:05.645Z" }, + { url = "https://files.pythonhosted.org/packages/30/a6/9f9f380dbb301f67023bf8f707aaa75daadf84f7152d95c410fd7e81d994/pydantic_core-2.46.4-cp314-cp314t-win32.whl", hash = "sha256:e846ae7835bf0703ae43f534ab79a867146dadd59dc9ca5c8b53d5c8f7c9ef02", size = 1955575, upload-time = "2026-05-06T13:38:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/40/1f/f1eb9eb350e795d1af8586289746f5c5677d16043040d63710e22abc43c9/pydantic_core-2.46.4-cp314-cp314t-win_amd64.whl", hash = "sha256:2108ba5c1c1eca18030634489dc544844144ee36357f2f9f780b93e7ddbb44b5", size = 2051624, upload-time = "2026-05-06T13:38:21.672Z" }, + { url = "https://files.pythonhosted.org/packages/f6/d2/42dd53d0a85c27606f316d3aa5d2869c4e8470a5ed6dec30e4a1abe19192/pydantic_core-2.46.4-cp314-cp314t-win_arm64.whl", hash = "sha256:4fcbe087dbc2068af7eda3aa87634eba216dbda64d1ae73c8684b621d33f6596", size = 2017325, upload-time = "2026-05-06T13:40:52.723Z" }, + { url = "https://files.pythonhosted.org/packages/ee/a4/73995fd4ebbb46ba0ee51e6fa049b8f02c40daebb762208feda8a6b7894d/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:14d4edf427bdcf950a8a02d7cb44a08614388dd6e1bdcbf4f67504fa7887da9c", size = 2111589, upload-time = "2026-05-06T13:37:10.817Z" }, + { url = "https://files.pythonhosted.org/packages/fb/7f/f37d3a5e8bfcc2e403f5c57a730f2d815693fb42119e8ea48b3789335af1/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:0ce40cd7b21210e99342afafbd4d0f76d784eb5b1d60f3bdc566be4983c6c73b", size = 1944552, upload-time = "2026-05-06T13:36:56.717Z" }, + { url = "https://files.pythonhosted.org/packages/15/3c/d7eb777b3ff43e8433a4efb39a17aa8fd98a4ee8561a24a67ef5db07b2d6/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:90884113d8b48f760e9587002789ddd741e76ab9f89518cd1e43b1f1a52ec44b", size = 1982984, upload-time = "2026-05-06T13:39:06.207Z" }, + { url = "https://files.pythonhosted.org/packages/63/87/70b9f40170a81afd55ca26c9b2acb25c20d64bcfbf888fafecb3ba077d4c/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:66ce7632c22d837c95301830e111ad0128a32b8207533b60896a96c4915192ea", size = 2138417, upload-time = "2026-05-06T13:39:45.476Z" }, + { url = "https://files.pythonhosted.org/packages/9d/1d/8987ad40f65ae1432753072f214fb5c74fe47ffbd0698bb9cbbb585664f8/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:1d8ba486450b14f3b1d63bc521d410ec7565e52f887b9fb671791886436a42f7", size = 2095527, upload-time = "2026-05-06T13:39:52.283Z" }, + { url = "https://files.pythonhosted.org/packages/64/d3/84c282a7eee1d3ac4c0377546ef5a1ea436ce26840d9ac3b7ed54a377507/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:3009f12e4e90b7f88b4f9adb1b0c4a3d58fe7820f3238c190047209d148026df", size = 1936024, upload-time = "2026-05-06T13:40:15.671Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ca/eac61596cdeb4d7e174d3dc0bd8a6238f14f75f97a24e7b7db4c7e7340a0/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ad785e92e6dc634c21555edc8bd6b64957ab844541bcb96a1366c202951ae526", size = 1990696, upload-time = "2026-05-06T13:38:34.717Z" }, + { url = "https://files.pythonhosted.org/packages/fa/c3/7c8b240552251faf6b3a957db200fcfbbcec36763c050428b601e0c9b83b/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0", size = 2147590, upload-time = "2026-05-06T13:39:29.883Z" }, + { url = "https://files.pythonhosted.org/packages/11/cb/428de0385b6c8d44b716feba566abfacfbd23ee3c4439faa789a1456242f/pydantic_core-2.46.4-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:0c563b08bca408dc7f65f700633d8442fffb2421fc47b8101377e9fd65051ff0", size = 2112782, upload-time = "2026-05-06T13:37:04.016Z" }, + { url = "https://files.pythonhosted.org/packages/0b/b5/6a17bdadd0fc1f170adfd05a20d37c832f52b117b4d9131da1f41bb097ce/pydantic_core-2.46.4-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:db06ffe51636ffe9ca531fe9023dd64bdd794be8754cb5df57c5498ae5b518a7", size = 1952146, upload-time = "2026-05-06T13:39:43.092Z" }, + { url = "https://files.pythonhosted.org/packages/2a/dc/03734d80e362cd43ef65428e9de77c730ce7f2f11c60d2b1e1b39f0fbf99/pydantic_core-2.46.4-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:133878133d271ade3d41d1bfb2a45ec38dbdbda40bc065921c6b04e4630127e2", size = 2134492, upload-time = "2026-05-06T13:36:58.124Z" }, + { url = "https://files.pythonhosted.org/packages/de/df/5e5ffc085ed07cc22d298134d3d911c63e91f6a0eb91fe646750a3209910/pydantic_core-2.46.4-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:9bc519fbf2b7578398853d815009ae5e4d4603d12f4e3f91da8c06852d3da3e9", size = 2156604, upload-time = "2026-05-06T13:37:49.88Z" }, + { url = "https://files.pythonhosted.org/packages/81/44/6e112a4253e56f5705467cbab7ab5e91ee7398ba3d56d358635958893d3e/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:c7a7bd4e39e8e4c12c39cd480356842b6a8a06e41b23a55a5e3e191718838ddf", size = 2183828, upload-time = "2026-05-06T13:37:43.053Z" }, + { url = "https://files.pythonhosted.org/packages/ac/ad/5565071e937d8e752842ac241463944c9eb14c87e2d269f2658a5bd05e98/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:d396ec2b979760aaf3218e76c24e65bd0aca24983298653b3a9d7a45f9e47b30", size = 2310000, upload-time = "2026-05-06T13:37:56.694Z" }, + { url = "https://files.pythonhosted.org/packages/4f/c3/66883a5cec183e7fba4d024b4cbbe61851a63750ef606b0afecc46d1f2bf/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:86e1a4418c6cd97d60c95c71164158eaf7324fae7b0923264016baa993eba6fc", size = 2361286, upload-time = "2026-05-06T13:40:05.667Z" }, + { url = "https://files.pythonhosted.org/packages/4b/2d/69abac8f838090bbecd5df894befb2c2619e7996a98ddb949db9f3b93225/pydantic_core-2.46.4-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:d51026d73fcfd93610abc7b27789c26b313920fcfb20e27462d74a7f8b06e983", size = 2193071, upload-time = "2026-05-06T13:38:08.682Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pypdfium2" +version = "5.13.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ec/78/a52cb80611339ec95f35c7a10d7bfe7a6f97f3b50a35a9f94283d062512e/pypdfium2-5.13.0.tar.gz", hash = "sha256:7ca2d8e31bd8d0d40c496416b7d8bea423388669ffd494929f50e8c3a82326b8", size = 273639, upload-time = "2026-08-13T10:58:15.837Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7c/9c/a49050af85055054299c7fab658ac63f8fddde575774aecbf8f71c7a9e5f/pypdfium2-5.13.0-py3-none-android_23_arm64_v8a.whl", hash = "sha256:882f4bbd4b17a335b43603169a14cde9341de12b238acd5c39e690cbca7c4293", size = 3417299, upload-time = "2026-08-13T10:57:40.522Z" }, + { url = "https://files.pythonhosted.org/packages/50/ad/f23027328843ee2bdd05afe16bb101f5906befd0c70de35fa8c53f60a5ff/pypdfium2-5.13.0-py3-none-android_23_armeabi_v7a.whl", hash = "sha256:d96929bde3bd64c771ab3558ca1ffd7704cc4d872ab92cd9f8f8b8a20f7f36b8", size = 2864708, upload-time = "2026-08-13T10:57:42.259Z" }, + { url = "https://files.pythonhosted.org/packages/08/99/1fe58428b69d2722dcbcfaa08ce71834a332c5b518fd58874bcef936b823/pypdfium2-5.13.0-py3-none-macosx_13_0_arm64.whl", hash = "sha256:da5c7b74eebf40b5c1fbe1de01aa1edc8827a79fb1efd999616bc20dcaf77ba4", size = 3507415, upload-time = "2026-08-13T10:57:43.978Z" }, + { url = "https://files.pythonhosted.org/packages/9f/41/06e26da88a4f5b4ed289325868717a186020661b7b221aa6df622711d31b/pypdfium2-5.13.0-py3-none-macosx_13_0_x86_64.whl", hash = "sha256:2abedfb5c70992b19c780ed58d7f7b929e8ce8ee52c9140158f44317c90ec6c7", size = 3670979, upload-time = "2026-08-13T10:57:45.607Z" }, + { url = "https://files.pythonhosted.org/packages/fe/31/f8210d53775f142be934336665b1d60e800c3f176f28c29b4908d945c518/pypdfium2-5.13.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9ee8c2bb2e68b396ab4a763215ac100dacb6b96d0da5bebeb239a021aecc3a7e", size = 3676486, upload-time = "2026-08-13T10:57:47.267Z" }, + { url = "https://files.pythonhosted.org/packages/94/50/d339fa09fbe592564b100bfc76833170a1104a764a458ac2abfffcb632f2/pypdfium2-5.13.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:07f58e91b8c45ca144a1ff3008faf3c73ef8a5e9fb32988831788363288228cd", size = 3400883, upload-time = "2026-08-13T10:57:49.189Z" }, + { url = "https://files.pythonhosted.org/packages/c3/e0/b10cf41b5e9f0212d014c40635659c6ab95bb4fcc6fc47f5d3c571f8d57f/pypdfium2-5.13.0-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:46b2f5be9e7ae941ee4216e3d20b66f9dc3d81944a3d57756272de5275204709", size = 3803912, upload-time = "2026-08-13T10:57:50.865Z" }, + { url = "https://files.pythonhosted.org/packages/a7/d8/25ba4ce9a9059ece82f4514df0658fde0aa9bbeafe135e76017c052bf56f/pypdfium2-5.13.0-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d96beb7f379e6c76d874ca93fcd182ac3168dd499056407070f9927fb1061b8e", size = 4218231, upload-time = "2026-08-13T10:57:52.525Z" }, + { url = "https://files.pythonhosted.org/packages/d3/7c/74a2fb48e5b0d2402d9ca64b39074c722d67e9a8a2c58449a843a8c2329a/pypdfium2-5.13.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:81df25c1ab4c13ff773102d3cbea1967511d079123b067fc077bd0c4d57d91d8", size = 3730077, upload-time = "2026-08-13T10:57:54.021Z" }, + { url = "https://files.pythonhosted.org/packages/59/12/8c922f00518c26dc47d3676cc09c1d3c95e991c1977e31067d23cc2215cb/pypdfium2-5.13.0-py3-none-manylinux_2_27_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d66a32d89fa5b4a2715810171239eb194df4aba604727483ab760512f3c6a851", size = 4031512, upload-time = "2026-08-13T10:57:55.736Z" }, + { url = "https://files.pythonhosted.org/packages/c6/48/a171d034c2dac01adcc57d3dad3c97ba11f19d916f421176002c9e02c904/pypdfium2-5.13.0-py3-none-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b90b0a5ac310bb34db8eb848e58fcab4e201e124e3cf3cb1ccb7b85293e034af", size = 3995485, upload-time = "2026-08-13T10:57:57.39Z" }, + { url = "https://files.pythonhosted.org/packages/36/2e/dcb24776d409bb9e5b7fb26a0c62a87b98ab0e30dfcca645eaf31e35123b/pypdfium2-5.13.0-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:ada81c36483cd61d07e32bc7814620ee96256b4f421b913f566861bf91800248", size = 5016636, upload-time = "2026-08-13T10:57:59.181Z" }, + { url = "https://files.pythonhosted.org/packages/93/24/1fab8470fc6de6f4481f009c90757b1a1ee0a61d8e864ed273f72ffca855/pypdfium2-5.13.0-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:3826e521e895648983cb9ee6b934d4bf51552600043984f84e9c2b3b14b696f3", size = 4555251, upload-time = "2026-08-13T10:58:00.753Z" }, + { url = "https://files.pythonhosted.org/packages/cd/ef/6e8dbea1eddcb55cf34172753ffccd39566333c803cc94d43c653f369f2f/pypdfium2-5.13.0-py3-none-musllinux_1_2_i686.whl", hash = "sha256:5c029d7163a91f264eafab51fb442a84a33efd9fd83d5a06c0136a7857a3cc8d", size = 5263483, upload-time = "2026-08-13T10:58:02.48Z" }, + { url = "https://files.pythonhosted.org/packages/53/fe/2ff673730189a621c01f9193c74b0f6aa70d8740889fdf11949e1c541869/pypdfium2-5.13.0-py3-none-musllinux_1_2_ppc64le.whl", hash = "sha256:be2dccbde0ce7efe334ecd8f348df4308db360756ede4f0821d82dfc9a58caa8", size = 5144135, upload-time = "2026-08-13T10:58:04.351Z" }, + { url = "https://files.pythonhosted.org/packages/19/0b/759b9037c007317fa5c990dd3f6eff2b99d3fbced251d1e2512be92f2e2e/pypdfium2-5.13.0-py3-none-musllinux_1_2_riscv64.whl", hash = "sha256:bcd81394fe101405e026eedb3e40bef84635c1e5d974dd6036420eb6937753c6", size = 4648156, upload-time = "2026-08-13T10:58:06.036Z" }, + { url = "https://files.pythonhosted.org/packages/db/3b/ffe29679c52efe8eb02d77aa6656e6d6201395423329af018ebd5923a3d0/pypdfium2-5.13.0-py3-none-musllinux_1_2_s390x.whl", hash = "sha256:2ed32ff685f8e05e637c990bedbf5fca66727bf27718d8bc33eeab21ce0630d1", size = 5089852, upload-time = "2026-08-13T10:58:07.791Z" }, + { url = "https://files.pythonhosted.org/packages/7b/b6/cebacc1601ddfdcd1e6a1dc321533d215ceccf9b825fa9b91b11c6dc39fb/pypdfium2-5.13.0-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:9c777edba28d1d5fd15435ed3a78ee2fdb93dd069be37cb53b559bc122793770", size = 5074153, upload-time = "2026-08-13T10:58:09.396Z" }, + { url = "https://files.pythonhosted.org/packages/54/40/cf14c4f534f817788966857afdedb90002198dca5ce4fe2c6ecb031955ae/pypdfium2-5.13.0-py3-none-win32.whl", hash = "sha256:d33ee7077db67478b75efe4b5ea9610fb96c5416a0bc4949227f0f59c34dfcd9", size = 3753164, upload-time = "2026-08-13T10:58:10.97Z" }, + { url = "https://files.pythonhosted.org/packages/5d/99/a37b6b902457569468ed5908c94e56cb6c4032541f02cf89f723d42a9148/pypdfium2-5.13.0-py3-none-win_amd64.whl", hash = "sha256:47dcca2a8d507b5fd24f94c3c9d48fb379430f097bc20f01beff6c963ffbcedb", size = 3885553, upload-time = "2026-08-13T10:58:12.709Z" }, + { url = "https://files.pythonhosted.org/packages/50/7f/d39f6e64375c2ffd50ea100e3c73af79085c880c2791eb7203bc61d8913f/pypdfium2-5.13.0-py3-none-win_arm64.whl", hash = "sha256:554a0b23376460af1410e3c915906895e2dac67a086b9e6ccde0643a795d3b0d", size = 3700026, upload-time = "2026-08-13T10:58:14.206Z" }, +] + +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + +[[package]] +name = "pytest-asyncio" +version = "1.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/43/7c/d36d04db312ecf4298932ef77e6e4a9e8ad017906e24e34f0b0c361a2473/pytest_asyncio-1.4.0.tar.gz", hash = "sha256:c6c0d2259945122819f171a32ecea2c349ead889ee28176caaf492143424be42", size = 58514, upload-time = "2026-05-26T09:56:04.083Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/03/e2/08a497ef684b88559c9cc5f4ad53a37e7b99e727094a86d6ea32536d5d3c/pytest_asyncio-1.4.0-py3-none-any.whl", hash = "sha256:933ca923a23075a87fb7070c0ec272a6848489824d887c85c812670932835aa1", size = 16930, upload-time = "2026-05-26T09:56:02.576Z" }, +] + +[[package]] +name = "python-dateutil" +version = "2.9.0.post0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "six" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432, upload-time = "2024-03-01T18:36:20.211Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892, upload-time = "2024-03-01T18:36:18.57Z" }, +] + +[[package]] +name = "python-pptx" +version = "1.0.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "lxml" }, + { name = "pillow" }, + { name = "typing-extensions" }, + { name = "xlsxwriter" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/52/a9/0c0db8d37b2b8a645666f7fd8accea4c6224e013c42b1d5c17c93590cd06/python_pptx-1.0.2.tar.gz", hash = "sha256:479a8af0eaf0f0d76b6f00b0887732874ad2e3188230315290cd1f9dd9cc7095", size = 10109297, upload-time = "2024-08-07T17:33:37.772Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d9/4f/00be2196329ebbff56ce564aa94efb0fbc828d00de250b1980de1a34ab49/python_pptx-1.0.2-py3-none-any.whl", hash = "sha256:160838e0b8565a8b1f67947675886e9fea18aa5e795db7ae531606d68e785cba", size = 472788, upload-time = "2024-08-07T17:33:28.192Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" }, + { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" }, + { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" }, + { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" }, + { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" }, + { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" }, + { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" }, + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, +] + +[[package]] +name = "six" +version = "1.17.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031, upload-time = "2024-12-04T17:35:28.174Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" }, +] + +[[package]] +name = "sniffio" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372, upload-time = "2024-02-25T23:20:04.057Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" }, +] + +[[package]] +name = "tqdm" +version = "4.70.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/3b/6c24bec5be5e743ffd99576daa5cc077722fc7d5bbc00bd133fa0c698dc6/tqdm-4.70.0.tar.gz", hash = "sha256:55b0b0dbd97462d06ebee91e4dac24ed4d4702be82b24f07e6c1d27e08cea220", size = 795438, upload-time = "2026-07-27T11:33:15.271Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f9/1c/01bfd571a64e7f270e6bab5e33777debe0edc56759233ce84f27dec92d14/tqdm-4.70.0-py3-none-any.whl", hash = "sha256:7f585706bfddbdebf89daac705b2dfcc16890130727d3197ca62c732b4310953", size = 80184, upload-time = "2026-07-27T11:33:13.167Z" }, +] + +[[package]] +name = "truststore" +version = "0.10.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" }, +] + +[[package]] +name = "typing-inspection" +version = "0.4.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a3/26/b09b8010994eccc3c09092e6b34058f36a460eea2d4c3e8b910c695975a0/typing_inspection-0.4.4.tar.gz", hash = "sha256:547274fa6b0a561ccf549cc9524b999a578e737d015d8709d021f9d0d13bea47", size = 76928, upload-time = "2026-08-12T12:37:25.997Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/67/81/4add07e5172b7ac40d8ed5ff580409a7801a4fe26d529bdd915401dabfbe/typing_inspection-0.4.4-py3-none-any.whl", hash = "sha256:65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147", size = 14750, upload-time = "2026-08-12T12:37:24.648Z" }, +] + +[[package]] +name = "tzdata" +version = "2026.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/92/ff/5a28bdfd8c3ebec42564ac7d0e54ca3db65044a9314a97f9564fa7a1e926/tzdata-2026.3.tar.gz", hash = "sha256:4a1518b8993086a7982523e071643f3c0e5f213e75b21318e78bcabfff9d1415", size = 198674, upload-time = "2026-07-10T08:50:37.887Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e5/6d/b53b99a9f2766d095985947a5782f1702cabb129a34f7a802d7197af832f/tzdata-2026.3-py2.py3-none-any.whl", hash = "sha256:dc096730c87af6cab1b171c9d532be840741ff5d459015e7f6947bd7d7e54931", size = 348168, upload-time = "2026-07-10T08:50:36.46Z" }, +] + +[[package]] +name = "xlrd" +version = "2.0.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/07/5a/377161c2d3538d1990d7af382c79f3b2372e880b65de21b01b1a2b78691e/xlrd-2.0.2.tar.gz", hash = "sha256:08b5e25de58f21ce71dc7db3b3b8106c1fa776f3024c54e45b45b374e89234c9", size = 100167, upload-time = "2025-06-14T08:46:39.039Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1a/62/c8d562e7766786ba6587d09c5a8ba9f718ed3fa8af7f4553e8f91c36f302/xlrd-2.0.2-py2.py3-none-any.whl", hash = "sha256:ea762c3d29f4cca48d82df517b6d89fbce4db3107f9d78713e48cd321d5c9aa9", size = 96555, upload-time = "2025-06-14T08:46:37.766Z" }, +] + +[[package]] +name = "xlsxwriter" +version = "3.2.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/46/2c/c06ef49dc36e7954e55b802a8b231770d286a9758b3d936bd1e04ce5ba88/xlsxwriter-3.2.9.tar.gz", hash = "sha256:254b1c37a368c444eac6e2f867405cc9e461b0ed97a3233b2ac1e574efb4140c", size = 215940, upload-time = "2025-09-16T00:16:21.63Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3a/0c/3662f4a66880196a590b202f0db82d919dd2f89e99a27fadef91c4a33d41/xlsxwriter-3.2.9-py3-none-any.whl", hash = "sha256:9a5db42bc5dff014806c58a20b9eae7322a134abb6fce3c92c181bfb275ec5b3", size = 175315, upload-time = "2025-09-16T00:16:20.108Z" }, +]