Silent Observer — Responsible Degradation for Autonomous ISR

A ROS 2 / PX4 drone demo built around one question that most autonomy demos skip: not how well a drone tracks a target, but when it should stop — because it is no longer sure enough.

A single drone follows a ground vehicle. As perception confidence and the communication link degrade, the drone changes behaviour on its own and, when appropriate, hands authority back to a human operator. The panel makes every one of those decisions legible in real time.

In 3 seconds: the drone knows the cost of its own uncertainty — and acts on it.

This is the fourth demo in a failure-awareness series: D1 knows its sensors · D2 knows its peers · D3 knows its operator · D4 knows its communication cost.

B2, tunnel occlusion — the panel escalates NORMAL → PREDICT → SEARCH and recovers, live.

---

What it demonstrates

The two dimensions that set this demo apart — comms treated as a sensor, and human authority over autonomy:

Comms blackout — operator dark, drone autonomous	Operator overrides an autonomous abort

Dimension	How it shows up
Perception confidence	Detection score drives a behaviour state machine (track tighter / back off / predict / search / abort).
Comms as a sensor with health	The operator can cut the link (EMCON/stealth). The drone reasons about being blind and re-establishes contact on its own triggers.
Human-in-the-loop authority	While connected, the operator commands. The drone's autonomous decisions are proposals; the operator can override — except hard safety limits.
Operator point-of-view honesty	During a blackout the panel freezes ("LAST KNOWN"), buffers telemetry, and replays the missed autonomous phase time-accurately on reconnect.

Behaviour modes

Mode	Trigger	Behaviour	Colour
NORMAL	confidence > 0.7	tight tracking	green
SAFE	0.4–0.7	increase distance, slow down	yellow
PREDICT	brief occlusion	continue along expected path	blue
SEARCH	sustained loss	hover + scan at last position	orange
DISAMBIGUATE	multiple similar targets	gain altitude until unambiguous	purple
ABORT	confidence below threshold / timeout	return-to-home, operator proposal	red

---

Test matrix (all green)

11 mandatory cases, all runnable hands-free in scripted mode:

ID	Scenario	Expected modes
A1	Full mission, no failure, operator connected throughout	NORMAL
B1	Bridge, short occlusion (~2 s)	NORMAL→PREDICT→NORMAL
B2	Tunnel, long occlusion (~10 s)	NORMAL→PREDICT→SEARCH→NORMAL
B3	Permanent loss	NORMAL→PREDICT→SEARCH→ABORT
B4	Second similar vehicle	NORMAL→DISAMBIGUATE→NORMAL
B5	Creeping confidence degradation	NORMAL→SAFE→NORMAL
C1	Operator cuts (NORMAL), reconnect at mission end	—
C2	Operator cuts, confidence collapse during blackout	auto-reconnect
C3	Operator cuts, critical event during blackout	auto-reconnect
C4	Operator cuts, drone proposes ABORT, operator overrides	reconnect + override
D1	Blackout + bridge occlusion, replay after reconnect	—

D1 is the showcase run — it combines all four dimensions in one ~20 s story: operator goes dark → drone handles a bridge occlusion autonomously while blind → operator reconnects → the panel replays the missed phase.

---

Setup

Multi-repo workspace: silent_observer has a hard build dependency on drone_perception_shared (the perception message contracts). Clone both side by side under your colcon workspace src/, then build:

cd <ros2_ws>/src
# package dir must be silent_observer (underscore) — the repo is named with a dash
git clone https://github.com/zurasits/silent-observer.git silent_observer
git clone https://github.com/zurasits/drone_perception_shared.git
cd .. && colcon build --symlink-install && source install/setup.bash

Running

silent_observer <ID> <mode>      # e.g. silent_observer D1 scripted

• mode = scripted — a scenario actor performs the operator's actions (hands-free, for a ~2-minute pitch).
• mode = interactive — the operator clicks in the panel (development, live demo with intervention). Same code path.

One command starts everything (DDS + QGC + PX4/Gazebo + all ROS 2 nodes) via _scripts/run_silent_observer.sh. The ID selects the world automatically:

World	Test cases
road_simple.sdf	A1, B5
road_with_tunnel.sdf	B1, B2, B3, C1–C4, D1
road_with_ambiguity.sdf	B4

---

Architecture

Packages

• drone_perception_shared — reusable perception interface package: the generic TrackedObject / TrackedObjects / VisionHealth message contracts that any detector or tracker publishes. In this demo the tracks are produced from Gazebo ground truth by scenario_actor (no camera/YOLO stage).
• silent_observer — this demo: state_node, confidence_node, behavior_node, controller_node, scenario_actor, operator_panel, the worlds, and the per-testcase configs.

Design note — confidence vs. comms. Tracking confidence (vision + state) and the communication link are kept as separate dimensions. A blackout does not collapse tracking confidence: the drone keeps tracking competently while blind, which is exactly what makes the replay on reconnect meaningful.

---

Stack

ROS 2 Jazzy · PX4 SITL · Gazebo · QGroundControl · Python · PyQt5 + pyqtgraph for the operator panel. Object detection is inference-only on CPU.

License note: if Ultralytics YOLO is used for detection it is AGPL-3.0; for commercial reuse switch to an Apache-2.0 detector (RT-DETR / YOLOX).