feat(backward): explain the freed autograd graph in terms of invokes

[Hotragn]

Summary

Calling .backward() in more than one invoke of a trace fails deep inside torch.autograd:

with model.trace() as tracer:
    with tracer.invoke(prompt_a):
        x = model.transformer.h[5].attn.c_proj.output
        with model.lm_head.output.sum().backward():
            grad_a = x.grad.save()

    with tracer.invoke(prompt_b):
        y = model.transformer.h[5].attn.c_proj.output
        with model.lm_head.output.sum().backward():
            grad_b = y.grad.save()

RuntimeError: Trying to backward through the graph a second time (or directly
access saved tensors after they have already been freed). Saved intermediate
values of the graph are freed when you call .backward() or autograd.grad().
Specify retain_graph=True if you need to backward through the graph a second
time or if you need to access saved tensors after calling backward.

The message is accurate but points away from the cause. Each invoke calls .backward() exactly once, so "a second time" reads as describing someone else's code. And the real reason is a fact about nnsight, not about torch: invokes are not separate runs. Every invoke's input is combined into one batch and the model is called once, so the whole trace has a single autograd graph — and the first .backward() frees it for every invoke after it.

docs/gotchas/backward.md did cover retain_graph, but framed as "if you call .backward() more than once on overlapping graphs". One backward per invoke does not read as that, which is exactly why this is worth a dedicated diagnostic.

Changes

_explain_freed_graph translates that one error. It matches autograd's message — a bare RuntimeError with no exception type or code to key off, so string matching is the only option, and it is narrow — then states the cause in nnsight's terms and shows the fix, keeping torch's original text appended:

RuntimeError: The autograd graph for this trace has already been freed.

Every invoke in a trace contributes to a single batched forward pass, so all
invokes share one autograd graph. An earlier `.backward()` in the same trace
freed the graph that this one needs.

Pass `retain_graph=True` to every `.backward()` except the last one:

    with model.trace() as tracer:
        with tracer.invoke(prompt_a):
            x = model.transformer.h[5].output[0]
            with model.lm_head.output.sum().backward(retain_graph=True):
                grad_a = x.grad.save()
        with tracer.invoke(prompt_b):
            y = model.transformer.h[5].output[0]
            with model.lm_head.output.sum().backward():
                grad_b = y.grad.save()

The same applies to two `.backward()` calls inside one invoke. If you do not
need the gradients together, run each backward pass in its own trace instead.

Original error from torch.autograd: Trying to backward through the graph a
second time ...

When the failing call already passed retain_graph=True, the message says the earlier one did not, rather than suggesting a flag that is already set. Any other RuntimeError is re-raised untouched.

Docs: adds the cross-invoke case as its own section in docs/gotchas/backward.md (symptom / cause / wrong code / right code / mitigation, matching the file's existing shape) and points at it from the cross-invoke.md TL;DR.

Behaviour is unchanged

retain_graph=True across invokes already worked; it just wasn't discoverable. The tests pin that it produces each invoke's own gradient, against per-prompt single-invoke references:

• matching invoke: relative error ~1e-5
• every other invoke: ~1.2

for both two and three invokes.

Not fixed here

nnsight could retain the graph implicitly for all but the last invoke and make the whole thing disappear. That trades peak memory for convenience on every batched trace and changes a default, so it felt like a maintainer's call rather than something to fold into an error message. Happy to follow up if you'd prefer that.

Verification

tests/test_backward_errors.py, 5 tests — 2 fail on dev, all 5 pass here. Covers: the cause and fix appear in the message; the already-retaining variant; an unrelated RuntimeError is not reworded; retain_graph across two and three invokes yields the correct per-invoke gradients.

Full CI test list on CPU: 1 failed, 248 passed. The one failure is test_lm.py::TestGradients::test_backward_with_multiple_invokers, already red on dev and main and unrelated to this change (addressed in #693).

_{Run with transformers 5.15.1 / torch 2.9.1 on CPU.}

[Hotragn]

Superseded by #699, which is the same change based on 0.8 per your note in #696.

I checked the problem still reproduces there before porting — BackwardTracer.execute on 0.8 has no handling for it, so a second invoke's .backward() still surfaces autograd's raw "backward through the graph a second time":

RAISED RuntimeError
  mentions invokes/shared graph: False
  suggests retain_graph: True

#699 also folds in the batched-invoke gradient coverage from #693, since 0.8 has none and it's the same subject.