The README makes claims about measuring LLM irritation and providing a verifiable communications protocol. This file backs them up with specific module paths, the Ada type definitions that make the metric model precise, and honest notes about what is proven vs. tested.
The Vexometer (formally: the Irritation Surface Analyser, ISA) is an Ada
measurement tool that quantifies irritation across 10 standardised dimensions
when interacting with LLMs and AI tools.
It is consolidated from six previously independent repositories into a single
monorepo containing: the ISA core (vexometer/), a satellite intervention
system (vexometer-satellites/), the Lazy Eliminator completeness enforcer
(lazy-eliminator/), the Vext verifiable communications protocol (vext/),
and the Vext email gateway (vext-email-gateway/).
The Vexometer: Irritation Surface Analyser (ISA) for LLMs and related tools quantifies irritation across 10 dimensions with standardised, reproducible metrics.
How it works.
vexometer/src/vexometer-core.ads defines the Metric_Category enumeration
with all 10 dimensions:
Temporal_Intrusion (unsolicited output frequency, latency-induced context
disruption), Linguistic_Pathology (sycophancy density, hedge word ratio,
emoji abuse), Epistemic_Failure (hallucination frequency, calibrated error),
Paternalism (unsolicited warning rate, refusal-with-lecture frequency),
Telemetry_Anxiety (data collection transparency, opt-out friction),
Interaction_Coherence (repeated failure rate, context retention quality),
and four extended v2 metrics including Completion_Integrity (the detection
of incomplete outputs, TODO placeholders, and stubs).
vexometer/src/vexometer-metrics.ads declares ISA_Calculator as a tagged
type with formal preconditions on statistical functions (Mean, Standard_Deviation,
Median, Percentile all carry Pre ⇒ contracts enforced at runtime in
development builds).
vexometer/src/vexometer-probes.ads and vexometer-patterns.ads define the
probe types that collect raw samples and the pattern matchers that classify them
into metric categories.
The vexometer/src/abi/ directory contains the Idris2 ABI definitions for
the Ada/FFI boundary, with the Zig FFI implementation in vexometer/ffi/.
Honest caveat.
The Ada implementation has formal preconditions and the metric taxonomy is
precise, but the classification accuracy of individual probe patterns (e.g.
distinguishing sycophancy from legitimate acknowledgment) depends on heuristic
pattern matching that is tested but not formally verified.
See PROOF-NEEDS.md for the outstanding proof obligations.
This monorepo was consolidated from six independent repositories: vexometer, vexometer-satellites, vex-satellite-template, vex-lazy-eliminator, vext, vext-email-gateway.
How it works.
Each sub-project retains its own Justfile, Cargo.toml or vexometer.gpr,
ROADMAP.adoc, and CONTRIBUTING.adoc so they can be developed and tested
independently.
vext/ implements the Vext verifiable communications protocol in Rust: the
src/ directory contains vext-core/ (protocol primitives, cryptographic
proof of algorithmic neutrality) and vext-tools/ (CLI for generating and
verifying Vext messages).
vext/config.ncl is a Nickel configuration file defining the trust parameters.
lazy-eliminator/ is a Rust/Cargo project detecting lazy patterns (stubs,
ellipses, TODO placeholders) in LLM-generated code.
vexometer-satellites/ holds the umbrella documentation and coordination hub
for satellite intervention modules.
satellite-template/ is the scaffolding template for creating new satellite
projects.
The root Justfile provides monorepo-wide recipes (just build-all,
just test-all, just ci-gate) that invoke sub-project Justfiles in order.
Honest caveat.
The vext-email-gateway/ is explicitly marked in the README as a prototype
with "end-to-end wiring in progress" — it should not be treated as a complete
integration.
| Tool / Repo | How vexometer uses it |
|---|---|
|
Pre-commit security scan across all sub-projects in the monorepo |
|
Contractile integration hooks at both monorepo root and per-sub-project level |
|
Container build manifest for the Ada-based ISA tool image |
|
Hypatia CI scans for |
|
Echidna prover is used for formal verification of metric calculation correctness in the Ada package |
|
Tracks outstanding proof obligations, consumed by Hypatia and gitbot-fleet |
|
Authoritative naming policy for the ISA/Vexometer identifier split, referenced from other repos |
|
Vext protocol is designed to be Groove-compatible for inter-service communication |
| Path | What’s There |
|---|---|
|
Core types: |
|
Core body: constructors, accessors, and invariant checks for ISA result records |
|
Statistical functions (Mean, SD, Median, Percentile) with |
|
Metric calculation implementations using |
|
Probe interface: data collection from LLM interactions, sample buffering |
|
Pattern matchers: classify raw probe samples into |
|
Public API surface exposed over REST (JSON) and the Vext protocol |
|
Completion Integrity Index: detects stubs, ellipses, TODO placeholders in LLM output |
|
Sycophancy Frequency Rate detector: hedge word ratio, affirmative opener density |
|
Sycophancy Response Score: aggregate sycophancy across a session |
|
Repeated Conversation Index: circular conversation and failure-to-learn detection |
|
Ada GUI layer for the desktop ISA application |
|
Report generation: JSON and SARIF output from ISA session results |
|
Idris2 ABI definitions for the Ada/Zig FFI boundary |
|
Zig FFI implementation: C-ABI-compatible wrapper around Ada procedures |
|
GNAT project file: build configuration for the Ada package |
|
Alire crate descriptor for Ada/SPARK package management |
|
Vext protocol Rust implementation: |
|
Nickel configuration for Vext trust parameters |
|
Rust: pattern detection for LLM laziness (stubs, placeholders, truncated outputs) |
|
Umbrella documentation and coordination hub for satellite intervention modules |
|
Scaffolding template for new ISA satellite projects |
|
Prototype email bridge between Vext protocol and traditional SMTP (wiring in progress) |
|
Shell helpers: trust key generation, session replay, report extraction |
|
Monorepo recipes: |
|
Authoritative ISA/Vexometer naming policy |
|
Outstanding formal proof obligations for metric calculation and pattern classification |
|
Known testing gaps: end-to-end Vext email gateway, satellite integration tests |
|
Container build manifest |
|
Reproducible build environments (GNAT + Rust toolchain) |