From 2bac66ceecc37520519c28931a1266585462772a Mon Sep 17 00:00:00 2001 From: DB Lee Date: Thu, 14 May 2026 10:51:37 -0700 Subject: [PATCH] docs(model-direct): align dataset JSONL example with what 'agentops init' creates Tutorial Section 3 showed JSONL rows with an 'id' field, but the seed file generated by 'agentops init' (src/agentops/templates/smoke.jsonl) has no 'id' field - just 'input' and 'expected'. Users reading the tutorial and inspecting their seed file would see a mismatch with no explanation. Aligned the doc example to the actual init output (drop the 'id' key and use the same spacing). Verified end-to-end against the aifappframework Foundry project: - agentops init produces the now-matching seed. - model-direct eval against gpt-5.1 (target) with gpt-4.1-443723 as judge: results.json + report.md, threshold PASSED. - AZURE_AI_FOUNDRY_PROJECT_ENDPOINT is still required for model-direct invocation as the tutorial documents. Refs #126 --- docs/tutorial-model-direct.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/tutorial-model-direct.md b/docs/tutorial-model-direct.md index a550ebeb..9399f862 100644 --- a/docs/tutorial-model-direct.md +++ b/docs/tutorial-model-direct.md @@ -41,9 +41,9 @@ the deployment, and skips agent infrastructure entirely. `.agentops/data/smoke.jsonl` (one JSON object per line): ```jsonl -{"id":"1","input":"Answer with exactly this sentence: Paris is the capital of France and one of Europe's major cultural centers.","expected":"Paris is the capital of France and one of Europe's major cultural centers."} -{"id":"2","input":"Answer with exactly this sentence: Mars is known as the Red Planet because iron-rich dust gives its surface a reddish color.","expected":"Mars is known as the Red Planet because iron-rich dust gives its surface a reddish color."} -{"id":"3","input":"Answer with exactly this sentence: Water has the chemical formula H2O because each molecule contains two hydrogen atoms and one oxygen atom.","expected":"Water has the chemical formula H2O because each molecule contains two hydrogen atoms and one oxygen atom."} +{"input": "Answer with exactly this sentence: Paris is the capital of France and one of Europe's major cultural centers.", "expected": "Paris is the capital of France and one of Europe's major cultural centers."} +{"input": "Answer with exactly this sentence: Mars is known as the Red Planet because iron-rich dust gives its surface a reddish color.", "expected": "Mars is known as the Red Planet because iron-rich dust gives its surface a reddish color."} +{"input": "Answer with exactly this sentence: Water has the chemical formula H2O because each molecule contains two hydrogen atoms and one oxygen atom.", "expected": "Water has the chemical formula H2O because each molecule contains two hydrogen atoms and one oxygen atom."} ``` The first model-direct smoke test intentionally uses short factual