Skip to content

Add native RDP server backend - #41

Draft
kogekiplay wants to merge 54 commits into
AlynxZhou:masterfrom
kogekiplay:rdp-backend
Draft

Add native RDP server backend#41
kogekiplay wants to merge 54 commits into
AlynxZhou:masterfrom
kogekiplay:rdp-backend

Conversation

@kogekiplay

@kogekiplay kogekiplay commented Jul 3, 2026

Copy link
Copy Markdown

Summary

This draft PR adds a native RDP server backend for ReFrame, behind an optional Meson feature flag. The goal is to let ReFrame expose the existing capture/conversion/input/session pipeline over RDP, with FreeRDP used as a reference for the protocol implementation.

This is intentionally opened as a draft because the branch is large and the protocol surface is broad. I would like early feedback on the direction, module boundaries, and which pieces should be reduced or split before review.

Implemented

  • Adds a generic RfRemoteServer abstraction so the main server pipeline can use VNC or RDP backends through the same start/stop/update/flush/input/resize/clipboard-style interface.
  • Keeps existing VNC behavior as separate libvncserver/neatvnc modules.
  • Adds [remote] protocol=vnc|rdp configuration, with VNC remaining the default for compatibility.
  • Adds optional RDP build support through Meson, so default non-RDP builds do not require the RDP codec/tooling stack.
  • Implements the native RDP connection path: TPKT/X.224, TLS, MCS/GCC, security exchange, licensing-valid response, Demand Active/Confirm Active, client finalization, input PDUs, and update dispatch.
  • Implements RDPGFX dynamic virtual channel handling, including capability negotiation, surface creation, frame lifecycle, WireToSurface, FrameAck/QoE feedback, fragmentation, and ZGFX payload compression.
  • Implements graphics update paths for AVC420, AVC444/AVC444v2, AV1 i420/i444, Progressive, RemoteFX, PLANAR, NSCodec, and bitmap fallback.
  • Adds codec quality/backpressure policy for bitrate, QP, GOP, keyframe cadence, AVC444 LC/delta behavior, and QoE/FrameAck-based adaptation.
  • Adds keyboard, pointer, wheel, resize ownership, and single-active-client replacement policy.
  • Implements cliprdr text, HTML, and common image clipboard paths, including CF_UNICODETEXT, HTML Format, DIB/DIBV5, BMP, PNG, JPEG, WEBP, and TIFF handling.
  • Implements RDP audio output over rdpsnd, including static channel negotiation, PCM and IMA/DVI ADPCM format selection, Wave/Wave2 PDUs, WaveConfirm/training/quality-mode handling, PipeWire monitor capture, configurable sample rate/channels/frame size, sender queueing, latency statistics, silence suppression for better A/V sync, and audio-codec=auto|pcm|adpcm configuration.
  • Adds unit tests for protocol parsing/writing, MCS, DVC, RDPGFX, input mapping, config parsing, resize policy, AV1/AVC helpers, RemoteFX, NSCodec, cliprdr, clipboard rich-format helpers, RDP audio stream/rdpsnd helpers, ADPCM format/encoding helpers, and core update/quality policies.

Not Implemented / Future Work

  • NLA/CredSSP/NTLMv2 authentication is not implemented yet; TLS is supported.
  • Dynamic Display Control / monitor-layout updates are not complete yet; the current backend handles initial client desktop size and resize ownership policy.
  • Full cursor shape support is incomplete; the backend has a basic/default pointer path but not full Color Pointer / Large Pointer handling.
  • File clipboard support is not implemented yet, including CF_HDROP, FileGroupDescriptor, FileContents, drag/drop-style file streams, and related cancellation/error paths.
  • More complete delayed clipboard rendering and format-on-demand behavior is future work.
  • More exhaustive Windows/FreeRDP clipboard format compatibility, edge formats, and application-private format passthrough are future work.
  • Very large clipboard payload flow control, cancellation, and recovery behavior can be improved further.
  • Additional rdpsnd encoded formats beyond PCM and IMA/DVI ADPCM are future work.
  • Advanced audio device controls, volume synchronization, and richer client audio policy handling are future work.
  • Multimedia redirection/TSMF is not implemented.
  • Drive redirection, printer redirection, smartcard, serial/parallel redirection, audio input redirection, and RemoteApp/RAIL are out of scope for this PR.

Audio Scope

The audio path implemented in this PR covers the main remote-desktop playback case: server-side audio capture from PipeWire and client playback through rdpsnd, with low-latency scheduling, confirmation handling, and PCM/IMA ADPCM output depending on client support. Audio input redirection is intentionally out of scope for this PR because it is more relevant to conferencing or microphone-forwarding workflows than to the core cross-platform remote desktop path.

Test Plan

Automated checks run on the RDP development target:

  • meson test -C build-rdp --print-errorlogs
  • Targeted checks during the latest audio work:
    • meson test -C build-rdp reframe:rdp-rdpsnd reframe:rdp-audio-stream reframe:rdp-core reframe:config --print-errorlogs
  • meson setup build-nordp -Drdp=false
  • meson compile -C build-nordp
    • non-RDP build still compiles without the RDP backend enabled

Manual smoke testing performed during development:

  • Connected with Remote Desktop Manager and compatible RDP clients.
  • Verified RDPGFX Progressive fallback with RDM.
  • Verified AVC444/AVC420 and AV1-oriented paths with compatible RDP client options.
  • Verified that only one active RDP client is kept; a later connection replaces the previous one to avoid mixed-resolution/QoE/input conflicts in the current single-desktop architecture.
  • Verified text, HTML fallback, and common image clipboard transfer in both directions with compatible clients.
  • Verified live rdpsnd audio output with a compatible client, including Wave2 playback, confirmation handling, silence-suppression behavior for improved A/V sync, and compressed ADPCM negotiation fallback through audio-codec=auto.
  • Deployed to a test prefix and exercised live capture/update/input/audio paths against a running ReFrame session.

Notes for Reviewers

  • The PR is large because it contains the protocol backend, codec matrix, clipboard support, and audio output path. I am happy to split it if that makes review easier.
  • The RDP backend is meant to be optional and should not change existing VNC defaults.
  • The current design treats RDP as a new protocol backend rather than fitting it under the existing VNC naming/interfaces.

Comment thread dists/example.conf
# Source capture FPS. RDP uses this same value for frame pacing and encoders.
fps=30

[remote]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is no need to add another section, this option could be part of [reframe] group.

@@ -23,5 +23,6 @@ executable(
sources: sources,
dependencies: dependencies,
include_directories: include_directories,
install_rpath: libdir,

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no reason that we need rpath.

@@ -0,0 +1,248 @@
#include "rf-rdp-audio-stream.h"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least currently we have no way to get audio via streamer, what is this handling?

@AlynxZhou

Copy link
Copy Markdown
Owner

Are you coding with AI/LLM? Check your produced output, I don't think things have to be done in such a complex way, your PR smells like pouring code anywhere, only crazy man will allow such kind of code to be merged. Check your code, don't waste my time to review/refactor your LLM code, YOU should do it.

@@ -0,0 +1,336 @@
#include "rf-clipboard-rich.h"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to handle rich format clipboard, I am even not sure whether VNC supports it, so don't do this, at least for this PR, focus on core RDP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants