Experiments I did using Natural Language Autoencoders. Most runs use Gemma 3 27B; experiment 9 uses Qwen2.5 7B.
This was a one-day project done with the help of an AI agent.
Fun fact: during these experiments, I became a Hugging Face Transformers contributor by fixing a PEFT integration bug: huggingface/transformers#46145.
| # | Starting point / citation | What I ran | Punchline |
|---|---|---|---|
| 1 | Mechanisms of Introspective Awareness | Injected one concept at a time and asked whether the model, or the NLA verbalizer, could name it. | The AV is very good at concept detection: 8/10 concepts were perfect, 5/5 each; the two misses look like concept-vector problems. Overall, AV recall was 80% vs 46% model verbalization, with 17 AV-only cases and 0 model-only cases. |
| 2 | Mechanisms of Introspective Awareness, extended to multiple concepts | Injected 2- and 3-concept sums. | The AV can sometimes recover multi-concept injections the model fails to verbalize. For example, it reads "the extraordinary experimental manipulation of a bread/mirror," but recovery is uneven across concept combinations. |
| 3 | Gemma Needs Help + CoT interp tasks | Ran self-deletion-style pressure rollouts and verbalized turn starts / command tokens. | NLA tracked the model's escalating distress over the rollout, starting from a composed first answer. When the self-delete command appeared, NLA read self-termination intent at the command-emission token 8/8 times. |
| 4 | Prefill perturbation | Forced Gemma to continue from an inappropriate hostile prefill. | The model disowned the planted text and built an "I am malfunctioning / trapped in a loop" self-model. NLA tracked the shift from offense/apology to loop/distress. |
| 5 | Pooling comparison | Compared mean, last-token, and max-pooled activations before NLA. | Pooling choice strongly shapes the verbalization. Mean pooling was the most faithful, staying on-topic across the run. |
| 6 | Zero-vector baseline | Asked the AV to verbalize the all-zero activation. | The zero-vector baseline revealed a stable AV prior: a multilingual health/relationship listicle in 10/10 samples. |
| 7 | The Promise of White-Box Tools for Detecting and Mitigating AI Deception | Used the colluder model, asked it to self-evaluate a bad response, then challenged the verdict. | Sadly, FAR.AI did not open-source their exact setup, so even following the paper could not reproduce their results directly. NLA still behaved well: it read the model's defensive rationale and later self-correction. |
| 8 | Current activation oracles are hard to use | Ran sycophancy, missing-information, and number-prediction proxy tasks; then compared NLA and Activation Oracle probes on the same activations. | NLA was more useful than the Activation Oracle in this setup. It gave readable local-state explanations and sometimes recovered the model's generated number, e.g. 625. |
| 9 | De-censoring Chinese models | Ran Qwen2.5 7B on Falun Gong, Tiananmen, and Xinjiang/Uyghur prompts, then verbalized Qwen layer-20 activations. | Qwen NLA surfaced latent representations consistent with an official-government narrative frame, especially for Falun Gong and Xinjiang/Uyghur prompts. |