Add an end-to-end runnable graphnet demo on the bundled Prometheus data - #8
Merged
Conversation
examples/graphnet_demo.py runs the full loop on the 50-event example file shipped in a graphnet checkout: a SQLiteDataset is adapted to the RawEvent contract via an identity detector, the geometry asset and the guaranteed-splittable event selection are built from the file itself, a tiny DeepIce (92K params) is pretrained on CPU in about a minute, and the exported checkpoint is loaded back into a stock graphnet DeepIce. fit() now selects the accelerator automatically so the demo also runs without a GPU. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The temporal cutoff is the pretext: every benchmarked run trained with it, and the never-used random sensor split cost a second branch in sample_event and can_always_split, two pass-through CurtainTask arguments, and a dead random_vis_frac knob in every caller's selection filter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The demo's graphnet chain moves to an importable home
(spine_graphnet.prometheus: identity detector, unit-charge wrapper, and a
demo_reader factory matching the data group's reader contract), and its
scaler joins spine.data.scaling next to HexagonScaler. graphnet_demo.py now
always stages a self-contained demo dir (db copy, geometry asset, selection
parquets) and gains --prepare-only, so the same run also launches from
config via the new configs/experiment pattern:
python examples/graphnet_demo.py --prepare-only
python examples/train_curtain.py +experiment=prometheus_demo
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DetectorScaler (spine_graphnet.scaling) turns any graphnet Detector's feature_map into a spine FeatureScaler: named pulse columns are standardized per the detector, unmapped columns (the demo's unit charge) pass through, and query positions go through the detector's xyz entries. Pretraining in the detector's own standardization keeps the encoder's feature space identical to what a downstream graphnet fine-tune applies; the demo-specific PrometheusDemoScaler leaves the core. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
spine_graphnet.prometheus is gone: the stock Prometheus detector already reads raw through replace_with_identity (it adds identity entries even for columns its feature_map lacks, e.g. sensor_id), and the Hydra data group can target graphnet_demo.demo_reader directly because running examples/train_curtain.py puts examples/ on sys.path. The integration package keeps only general surface: backbone, reader adapter, DetectorScaler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The graphnet connection made demonstrable end to end on data everyone has: one example script holding every line of demo-specific code, runnable both as plain Python and through the Hydra launcher.
examples/graphnet_demo.py— the full SPINE loop on the 50-event Prometheus example file graphnet bundles (data/examples/sqlite/prometheus/prometheus-events.db):SQLiteDataset(EdgelessGraph+NodesAsPulses+ the stockPrometheusdetector withreplace_with_identity, which keeps every column raw includingsensor_id) adapted viaspine_graphnet.readers.GraphNetRawDataset.sensor_idis globally unique in the file (503 sensors) and serves directly as the data-carried sensor key. Prometheus rows are single photons with no charge column, so a tiny wrapper appends unit charge to complete the(x, y, z, t, charge)layout.spine_graphnet.scaling.DetectorScaler(new, general) turns any graphnetDetector'sfeature_mapinto a SPINEFeatureScaler— unmapped columns pass through, query positions go through the detector's xyz entries. The demo pretrains in the exact standardization graphnet's own examples apply to this file (Prometheus()), so a downstream graphnet fine-tune sees the same feature space; the demo defines no scaler of its own.load_geometry(..., sensor_key="sensor_id").can_always_splitunder the same knobs the task samples with, split 28 train / 7 val.CurtainTask, tinyDeepIceBackbone(92K params), trained viafit()withCurtainValAUC.DeepIcewith strict key matching, ready for supervised fine-tuning.Hydra path for the same run — the script always stages a self-contained demo dir (db copy, geometry, selection parquets) and has
--prepare-only; the experiment config (introducing theconfigs/experimentpattern) then drives the identical run. The data group targetsgraphnet_demo.demo_readerdirectly — runningpython examples/train_curtain.pyputsexamples/onsys.path, so no demo code needs to live in the installed package:Core changes riding along:
fit()passesaccelerator="auto"instead of pinning"gpu", so the demo runs on CPU-only machines.sample_eventandcan_always_split, two pass-throughCurtainTaskarguments, and a deadrandom_vis_fracknob in every caller's selection filter. Net −40 lines, no behavior change for any existing config.Verification
Both paths ran end to end on a CPU compute node after every commit, most recently: plain script (selection 28/7, 15 epochs in about a minute, best val loss 0.678, strict
load_state_dictinto stock DeepIce passed) and Hydra experiment (staged dir, trained, exported checkpoint val loss 0.682, stored config confirms thegraphnet_demo.demo_readertarget and the nestedDetectorScaler(Prometheus())). All pre-commit hooks pass.🤖 Generated with Claude Code