feat(drone): add RoboMaster TT/Tello integration#1667
feat(drone): add RoboMaster TT/Tello integration#1667jlelong-berkeley wants to merge 1 commit intodimensionalOS:devfrom
Conversation
Greptile SummaryThis PR adds a complete RoboMaster TT / Tello integration to DimOS, covering the UDP SDK adapter ( Key additions and concerns:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Agent as LLM Agent
participant TCM as TelloConnectionModule
participant SDK as TelloSdkClient
participant DTM as DroneTrackingModule
participant Drone as Tello Drone (UDP)
Agent->>TCM: takeoff()
TCM->>SDK: connection.takeoff()
SDK->>Drone: UDP "takeoff"
Drone-->>SDK: "ok"
SDK-->>TCM: True
TCM-->>Agent: "Takeoff command sent"
Agent->>TCM: follow_object("person")
TCM->>TCM: _wait_until_airborne()
TCM->>TCM: follow_object_cmd.publish(JSON)
TCM->>DTM: follow_object_cmd stream
DTM->>DTM: _detect_initial_bbox(frame, "person")
Note over DTM: YOLO local fallback if no ALIBABA_API_KEY
DTM->>DTM: _detection_servoing_loop()
loop 20 Hz tracking loop
DTM->>DTM: _detect_person_local(frame)
DTM->>DTM: _compute_person_follow_command()
DTM->>TCM: cmd_vel (Twist) via movecmd_twist
TCM->>SDK: connection.move_twist(twist)
SDK->>Drone: UDP "rc lr fb ud yw" (no-wait)
DTM->>DTM: tracking_overlay.publish(frame)
end
Agent->>TCM: land()
TCM->>SDK: connection.land()
SDK->>Drone: UDP "land"
Drone-->>SDK: "ok"
SDK-->>TCM: True
TCM-->>Agent: "Land command sent"
Reviews (1): Last reviewed commit: "feat(drone): add RoboMaster TT/Tello int..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| logger.warning("Tello video stalled; reopening capture") | ||
| capture.release() | ||
| capture = cv2.VideoCapture(uri, cv2.CAP_FFMPEG) | ||
| if capture.isOpened(): | ||
| capture.set(cv2.CAP_PROP_BUFFERSIZE, 1) | ||
| no_frame_count = 0 |
There was a problem hiding this comment.
self._video_capture after stream reconnect
When the video stream stalls and is reopened, the local variable capture is reassigned to the new cv2.VideoCapture object, but self._video_capture is never updated to point to it. As a result:
- The
disconnect()method callsself._video_capture.release()on the old, already-released capture object (the one released at line 416) — the new active capture is never properly released. - The new
cv2.VideoCaptureleaks until the thread exits naturally.
| logger.warning("Tello video stalled; reopening capture") | |
| capture.release() | |
| capture = cv2.VideoCapture(uri, cv2.CAP_FFMPEG) | |
| if capture.isOpened(): | |
| capture.set(cv2.CAP_PROP_BUFFERSIZE, 1) | |
| no_frame_count = 0 | |
| capture.release() | |
| capture = cv2.VideoCapture(uri, cv2.CAP_FFMPEG) | |
| if capture.isOpened(): | |
| capture.set(cv2.CAP_PROP_BUFFERSIZE, 1) | |
| self._video_capture = capture | |
| no_frame_count = 0 |
| while attempt <= retries: | ||
| try: | ||
| with self._command_lock: | ||
| self._command_socket.sendto(command.encode("utf-8"), self._tello_addr) | ||
| data, _ = self._command_socket.recvfrom(2048) | ||
| response = data.decode("utf-8", errors="ignore").strip().lower() | ||
| return response | ||
| except TimeoutError: | ||
| attempt += 1 | ||
| logger.warning(f"Tello command timeout [{command}] attempt {attempt}/{retries + 1}") | ||
| except OSError as exc: | ||
| logger.error(f"Tello command socket error [{command}]: {exc}") | ||
| break | ||
| return "error:timeout" |
There was a problem hiding this comment.
_command_lock held during blocking recvfrom starves real-time rc control
Both send_command and send_command_no_wait share the same _command_lock. send_command acquires the lock for the entire sendto + recvfrom cycle — up to command_timeout (default 7 s) per attempt, and up to 3 attempts (21 s total) when the drone is slow to ack.
send_command_no_wait (used by every rc() call in the visual servoing and orbit loops) must wait to acquire that same lock. So if a blocking SDK command (yaw, move_relative, land, takeoff) times out and retries while the tracking loop is pushing 20 Hz rc updates, the control channel is frozen for the full retry window.
A common fix is to split the socket layer into a fast path (fire-and-forget rc commands, no lock contention) and a slow path (blocking commands with their own lock/socket or a command queue). Alternatively, use a non-blocking send in send_command that releases the lock between sendto and recvfrom.
|
|
||
| # Inputs | ||
| movecmd: In[Vector3] | ||
| movecmd_twist: In[Twist] |
There was a problem hiding this comment.
if you name this cmd_vel keyboard controls in rerun will work
|
|
||
| # Outputs | ||
| odom: Out[PoseStamped] | ||
| status: Out[Any] |
There was a problem hiding this comment.
should add types for this stuff
| follow_object_cmd: Out[Any] | ||
|
|
||
| # Parameters | ||
| tello_ip: str |
There was a problem hiding this comment.
we have standard ways for configuring modules (it auto-allows you to use CLI, config files)
https://github.com/dimensionalOS/dimos/blob/dev/docs/usage/configuration.md
| @@ -0,0 +1,557 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
would be nice to move this to dimos/robot/drone/tello/ dir
| @@ -0,0 +1,136 @@ | |||
| # RoboMaster TT / Tello Integration | |||
There was a problem hiding this comment.
this is very tello specific doc so can move it to dimos/robot/drone/tello/readme.md
| odom: Out[PoseStamped] | ||
| status: Out[Any] | ||
| telemetry: Out[Any] | ||
| video: Out[Image] |
There was a problem hiding this comment.
name it color_image pls (makes it compatible with some stuff)
2 participants
Problem
DimOS had no native RoboMaster TT / DJI Tello embodiment layer. Existing drone support is MAVLink/RosettaDrone-oriented, while TT uses a Wi-Fi UDP text SDK (
8889commands,8890state,11111video).This made TT workflows (teleop, agentic commands, tracking/follow) unavailable without custom integration.
Closes DIM-XXX
Solution
Implemented a full TT/Tello integration path and documented operational setup.
dimos/robot/drone/tello_sdk.pydimos/robot/drone/tello_connection_module.pyodom/status/telemetry/videotakeoff,land,move,move_relative,yaw,rc,send_extfollow_object,center_person_by_yaw,orbit_object,observedrone-tello-tt-basicdrone-tello-tt-agenticdimos/robot/all_blueprints.pydimos/robot/drone/drone_tracking_module.pyyaw_forward_constant) and yaw controller tuningdocs/usage/drone_tello_tt.mddocs/usage/README.mdlinkdimos/robot/drone/README.mdupdatesDesign notes/tradeoffs:
Breaking Changes
None intended.
How to Test
Expected:
Tello SDK connected to 192.168.10.1:8889
video stream starts and camera feed appears in viewer.
Run TT agentic:
export OPENAI_API_KEY=... dimos --robot-ip 192.168.10.1 run drone-tello-tt-agentic --disable web-inputThen in another terminal:
Try:
takeoff
relative movement commands
center on person and keep me in frame
follow person meter
land
Validate local detector fallback (no Alibaba key):
unset ALIBABA_API_KEYRun agentic again and confirm tracking module logs fallback to local detector (no startup crash).
Sanity check non-TT drone stack still starts:
Optional static checks:
Contributor License Agreement
I have read and approved the CLA.