Problem
When an mdsthin fetch loses its connection mid-pull (SSH tunnel drops, or a reused ControlMaster times out under ControlPersist), every remaining channel's read throws and comes back ok=False. _write_h5 records those in the channels_missing attr — indistinguishably from channels that genuinely have no data.
On the next run with the same window, _existing_channels returns channels_fetched | channels_missing, and fetch_shot filters those out. So the transiently-failed channels are treated as "already attempted" and silently skipped — the user gets a permanently-incomplete shot file until they know to pass force=True.
This is a pre-existing shape (the DIII-D PTDATA path has it too), but the new -O forward ControlMaster fast path for NSTX makes an abrupt mid-pull master death somewhat more likely (the master isn't owned by the fetch process).
Where
src/magnetics/data/fetch/toksearch.py — _fetch_mdsthin_tree / _fetch_mdsthin (channels marked ok=False on connection errors), _write_h5 (channels_missing), _existing_channels (merge/skip).
Suggested fix
Distinguish a connection/transport error from a genuine no-data result, and only cache the latter as "missing". Options:
- Tag
Channel.error with a class (e.g. transport vs no_data) and have _write_h5/_existing_channels exclude transport failures from the skip set (so they're retried next run without --force).
- Or, if any channel failed with a transport error, don't write the
channels_missing set at all for that run.
Context
Found in the PR #52 review (feature/nstxu-data-fetch). Deferred as a separate cross-cutting change.
🤖 Generated with Claude Code
Problem
When an mdsthin fetch loses its connection mid-pull (SSH tunnel drops, or a reused ControlMaster times out under
ControlPersist), every remaining channel's read throws and comes backok=False._write_h5records those in thechannels_missingattr — indistinguishably from channels that genuinely have no data.On the next run with the same window,
_existing_channelsreturnschannels_fetched | channels_missing, andfetch_shotfilters those out. So the transiently-failed channels are treated as "already attempted" and silently skipped — the user gets a permanently-incomplete shot file until they know to passforce=True.This is a pre-existing shape (the DIII-D PTDATA path has it too), but the new
-O forwardControlMaster fast path for NSTX makes an abrupt mid-pull master death somewhat more likely (the master isn't owned by the fetch process).Where
src/magnetics/data/fetch/toksearch.py—_fetch_mdsthin_tree/_fetch_mdsthin(channels markedok=Falseon connection errors),_write_h5(channels_missing),_existing_channels(merge/skip).Suggested fix
Distinguish a connection/transport error from a genuine no-data result, and only cache the latter as "missing". Options:
Channel.errorwith a class (e.g.transportvsno_data) and have_write_h5/_existing_channelsexcludetransportfailures from the skip set (so they're retried next run without--force).channels_missingset at all for that run.Context
Found in the PR #52 review (feature/nstxu-data-fetch). Deferred as a separate cross-cutting change.
🤖 Generated with Claude Code