Skip to content

Add C-RADIO as a teacher#33

Open
abtonmoy wants to merge 1 commit into
rai-opensource:mainfrom
abtonmoy:add-cradio-teacher
Open

Add C-RADIO as a teacher#33
abtonmoy wants to merge 1 commit into
rai-opensource:mainfrom
abtonmoy:add-cradio-teacher

Conversation

@abtonmoy

@abtonmoy abtonmoy commented Jul 5, 2026

Copy link
Copy Markdown

Summary

Adds C-RADIO (nvidia/C-RADIOv3-H) as a distillation teacher.

C-RADIO is an agglomerative (multi-teacher-distillation) VFM whose weights are under the NVIDIA Open Model License, which permits commercial use — so it works as a default teacher. It fits the existing setup directly: translator heads are sized from MODEL_FEATURE_SIZES, so it slots in with no architecture changes.

Changes

  • foundation_models/vision_models/cradio.pyget_cradio_model / get_cradio_feature, mirroring the existing teacher modules. Input conditioning is handled internally by C-RADIO (pixels passed in [0, 1]); a forward pass returns (summary, spatial_features), reshaped to (B, C, H, W).
  • foundation_models/__init__.py — export the two functions.
  • preprocessing/feature_extraction_core/models.py — dispatch branches in get_model and get_feature_outputs.
  • foundation_models/common.py — add nvidia/C-RADIOv3-H to MODELS and MODEL_FEATURE_SIZES ((1280, 14, 14)).
  • configs/training/target_models/cradio.yaml — C-RADIO as a single teacher.
  • configs/training/target_models/cddsvr.yaml — the existing cddsv full set plus C-RADIO.

Testing

Verified locally on a single GPU:

  • Loads via AutoModel.from_pretrained("nvidia/C-RADIOv3-H", trust_remote_code=True); spatial features are (1280, 14, 14) at 224 input, matching the registered size.
  • A DeiT-tiny student distills C-RADIO both on its own and jointly alongside the existing teachers (DINOv2 / CLIP / ViT), with the per-teacher losses decreasing and cosine similarity to the teacher features rising.

Notes

  • Feature-normalization stats (mean/var) for the new teacher are generated from the dataset with the existing scripts/preprocessing/calc_feature_mean.py, as documented for the other teachers. Happy to commit the .npy files if you'd prefer them in-tree.
  • The checkpoint (nvidia/C-RADIOv3-H) and config naming are easy to change if you'd prefer a different variant or convention.
  • C-RADIO's trust_remote_code path pulls in timm, einops, and open_clip_torch; I can add these to the dependencies if you'd like.

Adds a cradio.py extractor mirroring the existing teachers, wired into get_model / get_feature_outputs / common.py, plus cradio and cddsvr target-model configs. Translator heads are sized from MODEL_FEATURE_SIZES, so no architecture changes are needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant