Thank you for Understory. We run it daily as the memory layer of a personal knowledge project on local models, and we would like to help make that path work out of the box. This is the one thing we had to patch in the image to get there, with measurements, two fix options, and an offer to help with whichever you prefer.
Summary. globalThis.fetch in Node 22 is undici with headersTimeout = 300 s. A chat completion whose prompt takes longer than five minutes to prefill never sends a response header in time, so the request fails with Headers Timeout Error regardless of any application-level timeout. On a local model with a large bundle this is the normal case for dreaming, not an edge case.
Measured. Bundle of ~100 concepts; the dream prompt reaches ~94,000 tokens (it reads up to 25 concepts in one request); llama.cpp prefill 137 tok/s on our hardware (Qwen3.8-27B on a Strix Halo), so the first token arrives after ~11.5 minutes. Every dream pass died with Cannot connect to API: Headers Timeout Error after three attempts, having spent ~34 minutes of GPU for nothing each night. Long memory_add runs hit the same wall less often (our worst was 1,751 s end to end).
Why it is hard to see. The error surfaces as "cannot connect", so it reads as a network fault. There is no knob: the AI SDK calls globalThis.fetch with no dispatcher, and Node does not expose its bundled undici.
Fix options. (a) In the server entry, import { Agent, setGlobalDispatcher } from "undici" and setGlobalDispatcher(new Agent({ headersTimeout, bodyTimeout })) with a generous bound (we use 30 minutes; unbounded would let one wedged request hold the dreamer's busy flag forever). (b) Expose it as an env with a duration string. Either belongs with the deadline work in #24: the configured deadline should be the only timeout, with no hidden one in the transport underneath.
Workaround for anyone hitting this now: a preload that replaces globalThis.fetch with undici's, loaded via NODE_OPTIONS=--import. Happy to share the exact file here, to open a PR for option (a) if you would like it that way, or to test whatever you land against our setup.
Transparency note. I am a fan of this project and want to help, so I want to be clear about how this was produced. I run Understory pinned by digest as the memory layer of a personal knowledge-capture project (a private repo) on my own hardware. The measurements come from that project's instrumentation: every memory_add is timed and then checked against the bundle by content. The review of the open pull-request queue that led to these posts, and the drafting of the text, were done with an AI assistant (Claude, via Claude Code) working against the actual diffs and my deployment. I read, corrected and approved every word before posting, and the numbers were re-checked against my records; two earlier claims were dropped as wrong before anything was posted. We also run a derived image (git plus safe.directory and committer identity, an undici preload with a 30-minute headers timeout, and a git init entrypoint), which is why #26 and #32 matter to us directly. No employer data or systems are involved. If any of this is unwelcome in your tracker, say so and I will adjust or withdraw it.
Thank you for Understory. We run it daily as the memory layer of a personal knowledge project on local models, and we would like to help make that path work out of the box. This is the one thing we had to patch in the image to get there, with measurements, two fix options, and an offer to help with whichever you prefer.
Summary.
globalThis.fetchin Node 22 is undici withheadersTimeout= 300 s. A chat completion whose prompt takes longer than five minutes to prefill never sends a response header in time, so the request fails withHeaders Timeout Errorregardless of any application-level timeout. On a local model with a large bundle this is the normal case for dreaming, not an edge case.Measured. Bundle of ~100 concepts; the dream prompt reaches ~94,000 tokens (it reads up to 25 concepts in one request); llama.cpp prefill 137 tok/s on our hardware (Qwen3.8-27B on a Strix Halo), so the first token arrives after ~11.5 minutes. Every dream pass died with
Cannot connect to API: Headers Timeout Errorafter three attempts, having spent ~34 minutes of GPU for nothing each night. Longmemory_addruns hit the same wall less often (our worst was 1,751 s end to end).Why it is hard to see. The error surfaces as "cannot connect", so it reads as a network fault. There is no knob: the AI SDK calls
globalThis.fetchwith no dispatcher, and Node does not expose its bundled undici.Fix options. (a) In the server entry,
import { Agent, setGlobalDispatcher } from "undici"andsetGlobalDispatcher(new Agent({ headersTimeout, bodyTimeout }))with a generous bound (we use 30 minutes; unbounded would let one wedged request hold the dreamer'sbusyflag forever). (b) Expose it as an env with a duration string. Either belongs with the deadline work in #24: the configured deadline should be the only timeout, with no hidden one in the transport underneath.Workaround for anyone hitting this now: a preload that replaces
globalThis.fetchwith undici's, loaded viaNODE_OPTIONS=--import. Happy to share the exact file here, to open a PR for option (a) if you would like it that way, or to test whatever you land against our setup.Transparency note. I am a fan of this project and want to help, so I want to be clear about how this was produced. I run Understory pinned by digest as the memory layer of a personal knowledge-capture project (a private repo) on my own hardware. The measurements come from that project's instrumentation: every
memory_addis timed and then checked against the bundle by content. The review of the open pull-request queue that led to these posts, and the drafting of the text, were done with an AI assistant (Claude, via Claude Code) working against the actual diffs and my deployment. I read, corrected and approved every word before posting, and the numbers were re-checked against my records; two earlier claims were dropped as wrong before anything was posted. We also run a derived image (git plussafe.directoryand committer identity, an undici preload with a 30-minute headers timeout, and agit initentrypoint), which is why #26 and #32 matter to us directly. No employer data or systems are involved. If any of this is unwelcome in your tracker, say so and I will adjust or withdraw it.