RAMS: Resource-Adaptive Model Switching for Edge Perception

Kushal Khemani, Evan Leri, Amit Hod, George Xu

RAMS is a runtime controller that selects among warm-loaded YOLOv8 NANO, SMALL, and MEDIUM detectors according to measured resource pressure and recent detections. It is intended for reproducible runtime-perception experiments on embedded and desktop edge devices.

RAMS reports runtime behavior and detector outcomes. Its VRU-retention policies are reactive prioritization policies, not safety guarantees. They cannot repair a VRU that the active detector missed before the policy received a detection.

The accompanying paper is available on arXiv.

How it works

The controller keeps three detector tiers warm and makes one selection per frame. The resource monitor samples CPU, memory, thermal, and, where supported, NVIDIA GPU telemetry. A configurable pressure calculation informs the policy; the selected model then returns detections in source-image coordinates.

resource telemetry + previous detections
              |
              v
       pressure calculation
              |
              v
       switching policy -----> warm detector tier
              |                         |
              +----- next-frame state <-+

The repository retains the code names threshold, predictive, adaptive, safety, and safety2. In the paper, these are respectively threshold, EWMA-smoothed, variance-adaptive EWMA, VRU-retention, and two-level VRU-retention. Fixed NANO, SMALL, and MEDIUM policies are also available for fair controller-path comparisons.

Runtime features

• CPU, memory, thermal, battery, and optional NVIDIA GPU telemetry.
• Configurable pressure with CPU, memory, thermal, GPU-utilization, and GPU-memory terms when those signals are supported.
• One source-image coordinate contract across ONNX, TensorRT, and Ultralytics inference paths.
• Normalized bounding-box area for two-level VRU retention, avoiding a resolution-dependent proximity threshold.
• Raw timing components for policy selection, preprocessing, inference, postprocessing, and controller end-to-end latency.
• Ordered replay manifests, block-bootstrap latency intervals, Wilson recall intervals, and documented TDP-profile energy estimates.

Repository layout

rams/                         The runtime library used by every controller and replay run.
  controller.py               Coordinates monitoring, policy selection, inference, and records.
  monitor.py                  Samples CPU, memory, thermal, clock, and optional GPU telemetry.
  models.py                   Defines backend-independent detection and coordinate contracts.
  policy.py                   Implements fixed-tier, threshold, EWMA, and VRU-retention policies.
  energy.py                   Applies the documented TDP-profile energy-estimation model.
  config.py                   Loads and validates controller and device configuration.

benchmark/                    The common frame-aware evaluation harness.
  run.py                      Replays ordered frames, times all controller stages, and writes records.
  load_injector.py            Provides the documented steady and burst host-load protocols.
                              Every fixed-tier and adaptive comparison uses this same path.

experiments/                  Named analyses used to build paper tables and figures.
  exp1...exp14                Policy comparison, load sweep, hysteresis, Pareto, accuracy,
                              retention, and KITTI Raw temporal carry-over experiments.
  complete_runall*.py         Legacy-compatible entry points for complete Windows and Jetson runs.

scripts/                      Operator-facing commands for a reproducible device run.
  calibrate.py                Measures idle and injected-load behavior before runtime phases.
  run_paper_suite.py          Runs the phased paper suite, including manifests and raw records.
  prepare_kitti_validation.py Creates or verifies the fixed KITTI replay subset and manifest.
  aggregate.py                Converts raw records into summaries, confidence intervals, and figures.
  build_platform_packages.py  Builds the distributable Windows, macOS, and Jetson ZIP packages.
  verify_*.py                 Checks process-load behavior and Jetson TensorRT prerequisites.

configs/                      Versioned runtime defaults and device-specific TDP energy profiles.
  default.yaml                Tier, confidence, pressure, hysteresis, and policy defaults.
  energy_profile*.json        Transparent power-cap and component assumptions for energy estimates.

docs/                         Reproducibility references and platform operator runbooks.
  RAMS_*_Runbook.md           Windows, macOS, and Jetson package guides copied into each ZIP.
  COMPLETE_RUNALL_*.md        Focused reference for the phased Windows full-evaluation sequence.

packages/                     Ready-to-send ZIP archives built from repository source and `docs/` guides.
  RAMS_*_validation.zip       Beginner-facing instructions from download through evidence collection.

results/                      Curated evidence organized by device and execution route.
  <device>/                   Calibration, manifests, records, figures, and provenance for one route.
  old/                        Historical evidence retained for traceability, not pooled with new runs.
  README.md                   Evidence index, device boundaries, and interpretation rules.

tests/                        Regression and integration coverage for runtime and protocol contracts.
  test_*.py                   Policy, monitor, model, controller, benchmark, and load-protocol checks.

For a new device evaluation, begin with the matching archive in packages/. For source development, read rams/, then benchmark/, and use scripts/run_paper_suite.py rather than invoking individual experiments unless you are deliberately reproducing a single analysis.

Quick start

If you received one of the platform archives, do not use this generic source checkout quick start. Go directly to Platform packages and follow the README inside the archive. The package README is the authoritative setup and execution guide for that computer.

The base installation supports a simulated smoke check. It verifies the controller and harness only. It is not paper evidence.

python -m venv .venv
# macOS/Linux
source .venv/bin/activate
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
python -m benchmark.run --n 5 --policy threshold --profile heavy --simulate

For real inference, install the inference dependencies and download the three model tiers:

pip install -r requirements-inference.txt
python -c "from ultralytics import YOLO; [YOLO(name) for name in ('yolov8n.pt', 'yolov8s.pt', 'yolov8m.pt')]"
python -c "from ultralytics import YOLO; YOLO('yolov8n.pt').export(format='onnx', imgsz=320, opset=12); YOLO('yolov8s.pt').export(format='onnx', imgsz=416, opset=12); YOLO('yolov8m.pt').export(format='onnx', imgsz=640, opset=12)"

Expected input sizes are NANO 320, SMALL 416, and MEDIUM 640. The repository does not include checkpoints, ONNX exports, datasets, or TensorRT engines.

Full evaluation setup

This section describes the shared asset contract for repository users. A person running an archive should use its package README instead: it contains the exact operating-system commands, checks, and stop conditions.

A paper-facing run requires real KITTI replay frames, COCO validation images and labels, the three models, and a completed device energy-profile JSON. The expected data root is:

<data-root>/kitti/images/val/
<data-root>/kitti/labels/val/
<data-root>/coco/images/val2017/
<data-root>/coco/labels/val2017/

Use the fixed 1,500-frame KITTI validation replay formed from sorted training frame indices 5981 through 7480 inclusive. Download KITTI 2D object-detection data from the KITTI benchmark site, and COCO val2017 images from COCO, with the matching Ultralytics YOLO-format labels.

Set the root explicitly before running:

# macOS/Linux
export RAMS_DATA_ROOT="$HOME/rams/data"
# Windows PowerShell: $env:RAMS_DATA_ROOT = 'D:\data'

The complete data preparation, calibration, phase sequence, retained outputs, and interpretation constraints are in REPRODUCIBILITY.md.

Run the paper suite

These are generic source-checkout examples. Use the exact Windows, macOS, or Jetson commands in the relevant package guide for a paper-facing run.

Run a smoke check first. It uses a small real replay when the datasets are present, validates the requested backend and paths, and must not be reported.

python scripts/run_paper_suite.py --platform windows --backend onnx --smoke --device <device-label>

For a full run, first document the device configuration and TDP assumptions:

cp configs/energy_profile.example.json configs/energy_profile_<device>.json

Then run calibration and the full phased suite. The default --phase all performs runtime phases 1 through 4, tier accuracy, policy accuracy, and retention sensitivity. It uses 10 independent blocks of 200 frames unless overridden explicitly.

python scripts/run_paper_suite.py --phase calibration --platform <platform> --device <device-label>
python scripts/run_paper_suite.py --phase all --platform <platform> --backend <backend> --device <device-label> --energy-profile configs/energy_profile_<device>.json

Use onnx for the Windows CPU reference path, onnx for the macOS CPU reference path, and tensorrt only with engines built on the exact target Jetson. The full commands and platform requirements are in the operator packages below.

Platform packages

The archives are self-contained code and instructions for coauthors. They exclude datasets, model weights, ONNX exports, TensorRT engines, environments, and result records.

Archive	Target route	Operator guide
RAMS_Windows_validation.zip	Windows CPU ONNX	README inside the archive and Windows runbook
RAMS_macOS_validation.zip	Apple Silicon CPU ONNX	macOS runbook
RAMS_Jetson_validation.zip	Jetson TensorRT	Jetson runbook

Download exactly one archive for the target computer, save it in that computer's Downloads folder, extract it, then start with the README.md at the root of the extracted rams_validation folder. Each package explains the required software, model creation, dataset download, safe KITTI split creation, smoke check, calibration, phased run, full run, and result handoff.

The documentation is deliberately split by responsibility:

Need	Read this
Download, installation, data preparation, and commands on a target computer	The README.md inside that platform archive.
Shared experimental protocol, evidence required for a result, and interpretation limits	REPRODUCIBILITY.md.
Project concepts, API orientation, result locations, and citation	This README.

The Jetson package has a mandatory target-device TensorRT preflight. It must load every device-built engine and run one real inference per tier before a paper phase begins. GitHub-hosted Linux and WSL2 check package compatibility; they cannot validate JetPack or a Jetson TensorRT engine.

Rebuild all current archives from tracked sources with:

python scripts/build_platform_packages.py --platform all

Curated results

Current revised-protocol evidence is organized by processor family:

Directory	Device and backend	Scope
results/m4_air	Apple M4 MacBook Air, 16 GB unified memory, CPU ONNX	Real KITTI replay, calibration, raw records, figures, and provenance.
results/i7_1165G7	Intel Core i7-1165G7, Windows CPU ONNX	Complete revised-protocol suite: calibration, runtime, Pareto, fresh COCO tier evaluation, policy-level KITTI metrics, retention sensitivity, and a separate KITTI Raw temporal carry-over analysis.
results/jetson_agx_orin_64gb_cpu_onnx	NVIDIA Jetson AGX Orin 64GB, CPU ONNXRuntime	Corrected on-device runtime, policy-accuracy, retention, calibration, per-rail energy records, and fresh mapped KITTI and COCO tier accuracy.
results/jetson_agx_orin_64gb_pytorch	NVIDIA Jetson AGX Orin 64GB, PyTorch	Corrected on-device runtime, policy-accuracy, retention, calibration, per-rail energy records, and COCO tier-accuracy context.
results/jetson_agx_orin_64gb_tensorrt	NVIDIA Jetson AGX Orin 64GB, TensorRT	Corrected on-device runtime, policy-accuracy, retention, calibration, per-rail energy records, canonical COCO-label verification, and fresh mapped KITTI and COCO tier accuracy.
results/i7_11800H	Intel Core i7-11800H, Windows CPU ONNX	Completed phased runtime evaluation, COCO validation, KITTI policy metrics, retention analysis, and manifests. The RTX 3050 Ti was not used for inference.
results/i7_11800H_rtx3050ti_cuda_onnx	Intel Core i7-11800H with RTX 3050 Ti, Windows CUDA ONNX	CUDA ONNX evidence with a fresh 1,500-frame mapped KITTI COCOeval run. The device result README reports the per-tier mAP values, protocol, and evidence limits. Do not pool this route with the CPU ONNX evidence.
results/old	Historical devices and protocols	Retained for traceability only. Do not pool with revised-protocol results.

TDP-profile energy fields are telemetry-conditioned estimates, not external power measurements. mAP is reportable only where the records identify a fresh evaluation source, such as ultralytics_val, onnx_cuda_cocoeval, or kitti_native_mapped_cocoeval.

Scope

RAMS evaluates runtime behavior, resource telemetry, and detector outcomes. It does not establish vehicle safety, correct initially missed objects, model closed-loop vehicle dynamics, or measure physical energy consumption. TensorRT engines must never be reused across different Jetson models or software stacks.

Citation

Use CITATION.cff for software citation, or cite the current arXiv record:

@misc{khemani2026rams,
  title={RAMS: Resource-Adaptive Model Switching for Edge Perception},
  author={Khemani, Kushal and Leri, Evan and Hod, Amit and Xu, George},
  year={2026},
  eprint={2606.14716},
  archivePrefix={arXiv},
  primaryClass={cs.DC}
}

License

This project is licensed under the MIT License.