A curated map of Jev / TypeSafe System One resources, real-world use cases, and reproducible benchmarks.
Independent community project. Not affiliated with or endorsed by TypeSafe.
English | 简体中文
Jev turns natural-language state into typed decisions: Noul for yes/no, Choice for selecting an option, and Score for ordered ratings. This repository combines an ecosystem guide with independent tests, raw artifacts, and the scripts needed to reproduce them.
- Start here
- Official resources
- SDKs and integrations
- Curated use cases
- Reproducible benchmarks
- Run the lab
- Research archive
- Contributing
| Goal | Best entry point |
|---|---|
| Understand Jev in five minutes | TypeSafe Quick Start |
| Choose an architecture pattern | Official patterns |
| See where people are using it | Curated use cases |
| Check claims against measurements | Jev Lab validation report |
| Reproduce the results | Run the lab |
- TypeSafe - product overview and access.
- Documentation - API concepts, guides, and reference.
- Models - aliases, versioned model IDs, and pricing.
- Patterns - fan-out, confidence-gated routing, composite scoring, and intent routing.
- Use-case map - official task and application taxonomy.
- TypeSafe GitHub organization - official SDKs, skills, and adapters. Beware similarly named, unrelated organizations.
| Project | What it provides | Source |
|---|---|---|
| Python SDK | Official TypeSafe API client for Python | typesafe-ai/typesafe-sdk-python |
| JavaScript SDK | Official TypeSafe API client for JavaScript and TypeScript | typesafe-ai/typesafe-sdk-js |
| Agent skills | Official skills for building with the System One API | typesafe-ai/skills |
| System One adapter | Run the TypeSafe client contract against other LLM APIs | typesafe-ai/system-one-adapter-python |
| Vercel AI Gateway | typesafe-ai/jev through AI SDK's evaluation API |
Vercel announcement |
| LangChain | Python and JS integration, model routing, and tool-risk middleware | Integration guide |
| OpenRouter | Hosted access to typesafe-ai/jev |
Model page |
| Cline browser harness | Jev-driven browser actions over structured DOM observations | cline/plugins/jev-browser |
These are representative projects and first-hand reports, not endorsements. Evidence labels distinguish inspectable implementations from author-reported results. See the full 28-case audit for broader coverage and caveats.
- Cline Jev Browser - browser action selection with a bounded loop and explicit review states. Open source.
- Jev Voice Browser - voice intent and browser target selection while speech is still streaming. Open source demo.
- jev-experiments - a collection of predictive UI, routing, and decision experiments. Open source experiments.
- TypeSafe Screening MCP - screens PubMed titles and abstracts against clinical criteria. Open source; first-hand run reported.
- ERP, knowledge-base, and mail search - parallel retrieval with Jev reranking and disambiguation. Author-reported production use.
- Edge k3s decision pipeline - separates deterministic code, typed decisions, and general LLM reasoning. Author-reported field test.
- AI content scoring - scores generated variants before publishing. Author-reported experiment.
- Sprite Fusion level generation - chooses the next terrain segment from bounded candidates. Published demo with timings.
- awesome-jev - a broader project and integration list.
- outjev.lol - community projects labeled as products, demos, or experiments.
- Jev Lab community survey - source-audited index with risks and selection criteria.
All numbers below come from this repository's pilot datasets. They are observations, not production guarantees.
| Question | Result | Evidence |
|---|---|---|
| Does 1 to 20-question fan-out add latency? | No growth observed; P50 stayed around 1.4 s | Report / raw artifact |
| What did 108 requests cost? | Approximately $0.0044 | Validation report |
| Were decisions repeatable? | 100% decision agreement across 3 x 36 support cases | Stability report |
| Did Jev beat a simple baseline? | No; keyword rules scored 97.2% vs 91.7% on an easy pilot set | Baseline |
| Is top-1 probability enough for a safety gate? | No; failed cases had confidence 0.04-0.16 while top-1 probability reached 0.58 | Confidence study |
| Did full-path latency match sub-100 ms claims? | No; this environment measured roughly 1.5 s end to end | Validation report |
Practical takeaways:
- Pin the versioned model ID after tuning thresholds; aliases can move.
- Gate on the API's
confidence, not only the largest answer probability. - Treat probabilities near 0.5 as uncertainty, not an automatic "no".
- Test CJK workloads separately; official documentation says English performs best.
- Keep raw inputs, outputs, errors, and retries so a result can be audited.
The test suite and comparison scripts use only the Python standard library. API benchmarks require a TypeSafe key.
cp .env.example .env
python3 -m unittest discover -s tests
# Support routing and a deterministic keyword baseline
python3 scripts/evaluate_support.py --run-name support-routing-v1
python3 scripts/evaluate_keyword_baseline.py
# Fan-out latency and confidence-gated escalation
python3 scripts/benchmark_fanout.py
python3 scripts/evaluate_confidence_escalation.pyRepository layout:
docs/ Source-audited reports and research notes
evals/ Versioned benchmark datasets
scripts/ Reproducible evaluators and comparison tools
artifacts/ Raw JSON outputs and generated reports
tests/ Offline unit tests
experiments/ Runnable integration experiments
| Topic | Reports |
|---|---|
| Benchmarks and calibration | Initial validation · Confidence gates |
| Agent control and reasoning supervision | Thinking supervision · Ecosystem cross-check |
| Official claims and integrations | Official/platform audit |
| Integration experiments | json-render + Jev |
| Community adoption | First-hand use cases · 28-case audit · Community survey |
| Round 6 survey | Evaluators, clone wave, and scale evidence |
| Round 7 survey | Competitor narrative checked, open-implementation selection map |
Independent replications, counterexamples, and new datasets are especially valuable. Read CONTRIBUTING.md, suggest a resource, or share a benchmark result without changing code.
This repository is research material. Do not treat a single-run probability as production accuracy or permission to execute an action. Prices, versions, and ecosystem status reflect the capture dates in each report.
No open-source license has been declared yet. Until one is added, standard copyright restrictions apply.