Skip to content

Exclude specific sensors/channels from all analysis (bad-data drop) #56

Description

@matt-pharr

Summary

We need a way to exclude specific sensors/channels from all analysis. If a single
probe misbehaves on a given shot — dead channel, railed/clipped signal, wrong gain,
disconnected integrator — its samples currently flow straight into both core analyses
and silently corrupt the fits. There is no way to say "ignore MPID66M on shot X"
short of editing code.

A bad sensor is not a rare edge case for real DIII-D data; the operator needs a first-class
"drop this channel" control, and it must apply everywhere — not per-view.

Why this bites both analyses

A single bad channel is not isolated to the plot it appears on:

  • Rotating (MODESPEC): every toroidal-array node stacks all present channels via
    nodes._array_channels(shot, families)_stack. A railed channel biases the SVD /
    phase-vs-φ fit and the n-spectrum for the whole array. There is no exclusion step between
    h5source.channel_names(shot) and the stack.
  • Quasi-stationary (SLCONTOUR): the fit selects channels by channel_filter (a regex or
    a friendly subset name, _slcontour/io_data.valid_channels). valid_channels already drops
    all-NaN channels, but a channel that carries plausible-looking but wrong signal passes
    the filter and skews K / χ² / the mode amplitudes. There's no per-shot manual exclude.

So "exclude sensor" has to live below both paths, at channel selection, not in the GUI
per-view.

Proposed shape (for discussion — this is a contract change, not a hack)

The natural seam is a single exclusion set threaded through channel selection:

  1. Data/selection layer: an excluded: set[str] (channel names) consulted in
    nodes._array_channels (rotating) and passed into the QS channel_filter resolution
    (subtract excluded names from the resolved regex/list). One source of truth so both
    analyses honor the same exclusions.
  2. Seam / params: carry the exclusion list as a /api/node query param (e.g.
    exclude=MPID66M,MPID132M) so it rides the existing params dict
    (app.pynodes.build_node) — and add it to the kind-node contract on both sides
    (core/contracts.pygui/web/src/lib/contract.ts) rather than faking it client-side.
  3. Scope decision (needs a call):
    • Per-shot (bad channel on one shot) vs persistent (a probe known-dead for a date
      range) — persistent could live shot-indexed alongside geometry in
      data/device/diiid.json, like the availability segments.
    • Whether exclusions are session-only (GUI state) or saved.
  4. GUI: a channel toggle in the Sensors view and/or a small "excluded sensors" control —
    clicking a bad sensor dot to drop it, with the analyses re-fitting live. Keep the user
    informed (which sensors are active vs excluded, and the resulting channel count).

Open questions

  • Per-shot vs persistent/date-ranged exclusions (or both)?
  • Should an excluded channel still render in the Sensors map (greyed out) so the operator
    can see what was dropped, while being pulled from every fit?
  • Auto-flagging: do we want a heuristic that suggests bad channels (railed / flatlined /
    σ-outlier), separate from the manual exclude? (Could be a follow-up.)

Touch points

  • src/magnetics/service/nodes.py_array_channels, _stack (rotating channel selection)
  • src/magnetics/core/qs_bridge.py + src/magnetics/_slcontour/io_data.py::valid_channels
    (QS channel selection)
  • src/magnetics/service/app.py — query-param passthrough
  • src/magnetics/core/contracts.pygui/web/src/lib/contract.ts — contract
  • gui/web/.../SensorsTab — the exclude UI

Cross-team (Data Streamers own selection, Interfacers own the seam/GUI, both analysis teams
consume it) — worth a quick Slack/PR coordination before implementing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions