Remote desktop: H.264 (Phase 1, server + viewer) — needs testing - #107
Merged
Conversation
Server + browser side of Phase 1 (companion to leuffen-rmm-agent#36), plus the byte-identical vendored agent sync. - server/app/main.py: fan the agent's `video_info` codec-negotiation message out to the remote-desktop viewer (binary frames already pass through). - static/remote.js: WebCodecs decode path. Advertises `codecs:["h264","jpeg"]` when the browser has VideoDecoder; on a `video_info` it configures the decoder and feeds each Annex-B access unit (keyframe detected by NAL type) to it, drawing decoded VideoFrames to the canvas. Waits for the first keyframe, resets per session, and falls back to the existing JPEG path when H.264 isn't negotiated or WebCodecs is absent. - agent/ vendored sync (screen_h264.py new, screen.py, agent.py, requirements) — code-only, no AGENT_VERSION change. Screen-frame/input transport is unchanged; H.264 rides the same binary channel. NOT tested end-to-end — needs a real Windows agent build (with PyAV) + a WebCodecs browser. Deploying is safe (JPEG fallback), but H.264 won't engage until the agent MSI bundles PyAV. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 server + browser side (companion to
leuffen-rmm-agent#36), plus the byte-identical vendored agent sync.main.py— fan the agent'svideo_infocodec-negotiation message out to the viewer (binary frames already pass through untouched).static/remote.js— WebCodecs decode path. Advertisescodecs: ["h264","jpeg"]when the browser hasVideoDecoder; onvideo_infoit configures the decoder and feeds each Annex-B access unit (keyframe detected by NAL type 5/7/8) as anEncodedVideoChunk, drawing decodedVideoFrames to the canvas. Waits for the first keyframe, resets per session, and falls back to the existing JPEG path when H.264 isn't negotiated / WebCodecs is absent.agent/— sync of the Phase 1 agent code (noAGENT_VERSIONchange).Safe to deploy (JPEG fallback is unchanged), but H.264 won't actually engage until the agent MSI is rebuilt with PyAV (agent PR #36). Please validate on a real Windows agent + a Chromium/Edge viewer before merging. The viewer's NAL keyframe detection is unit-tested;
node --check+py_compilepass.Suggested test flow: merge/build agent #36 → deploy this → open Remote control in Chrome → confirm it's sharp/smooth and check the agent log shows
codec=h264; disconnect from network / use Firefox to confirm the JPEG fallback still works.🤖 Generated with Claude Code