activation-probing

Activation-probe security scanner for AI agent tooling. Reads a model's internal activations to detect poisoned MCP servers, skills, and packages before install.

Infrastructure for capturing LLM activations and SAE (Sparse Autoencoders) features, training probes for prompt maliciousness detection, and evaluating out-of-distribution generalization with Leave-One-Dataset-Out (LODO)

Temporal activation-verbalization probe for frozen vision-language model hidden states.

Architecture and training decisions determine how observable an LLM is. Transformer activations carry decision-quality signals that output confidence misses; training can preserve or erase them during convergence, even as predictive performance improves.

Evaluation framework of different methods for probing and steering LLMs activations to mitigate Chain-of-Thought Unfaithfulness. Research project by Giovanni M. Occhipinti (University of Bologna), Alessandro Abate e Nandi Schoots (University of Oxford).

Research code for claim-level correctness probes on Llama activations.

An empirical study probing the residual stream vectors of google/gemma-2-2b-it during contrastive corporate finance compliance constraints. Features a native PyTorch layer-tracking notebook and an interactive 4-column Streamlit web dashboard analyzing geometric activation divergence deltas.

Research: can LLM internal activations predict reasoning failure before it's visible? Linear probe on DeepSeek-R1 hidden states at 150 tokens achieves AUC 0.612 vs 0.445 baseline (p=0.001) on AIME math problems. Signal emerges when surface-level features carry zero information.