logprobs

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

Typed choices and probabilities from models running in llama.cpp

An experimental approach for AI search visibility.

Calibrated yes/no, pick-one and scale judgments from your own LLMs, read straight off the logprobs. Runs on vLLM.

Interactive Streamlit app that visualises token-level log probabilities from Azure OpenAI

Token-level entropy trajectories from LLM logprobs. Models can measure their own uncertainty — grounding it in truth requires external verification.

An interactive client for exploring token-level log probabilities from OpenAI-compatible chat models.

Minimal demo of calibrated LLM-as-a-judge scores from the single-call token-logprob distribution (Wang et al. 2025; G-Eval).

A demo to show confidence probabilities in AI responses

MSc thesis: mechanistic interpretability of Chain-of-Thought reasoning in LLMs via SAE features and logprob signals

Stream LLM tokens with logprobs, compute per-token Shannon entropy, render a color-coded HTML uncertainty map

A drop-in confidence gate for LLM agents: turn token logprobs into an accept / verify / ask-a-human routing decision, with honest per-model calibration. Local & open-model first, plus an MCP server.

LLM token visualizer for decoder risk, logprobs, embeddings, branching, and consistency checks.

Measuring the computational cost of deception in LLMs through logprob analysis

Analyze LLM token log-probabilities: perplexity, per-token confidence, low-confidence span detection, and a terminal heatmap. Zero deps, CLI + library.

Confidence-gated invoice extraction: per-field scores from constrained decoding, measured for calibration on DocILE

Renders every token a model generated with the tokens it rejected, ghosted above it by probability

Single-pass CES entropy risk scoring for LLM outputs with calibration, provider logprobs, CLI, and agent/RAG guardrails.