Skip to content

feat(mac): register a HiDPI mode ladder so macOS offers scaling - #63

Open
meta-boy wants to merge 2 commits into
tranvuongquocdat:mainfrom
meta-boy:feat/hidpi-scaled-modes
Open

meta-boy wants to merge 2 commits into
tranvuongquocdat:mainfrom
meta-boy:feat/hidpi-scaled-modes

Conversation

@meta-boy

Copy link
Copy Markdown
Contributor

Problem

Turn HiDPI on and the virtual display is genuinely Retina — backingScaleFactor is 2.0 and macOS renders a doubled framebuffer. But System Settings shows no "Larger Text … More Space" control for it, so scaling can only be changed from inside Side Screen, which tears the display down and restarts the stream.

macOS builds that control from a display's desktop-usable HiDPI modes. VirtualDisplayManager.createDisplay registers only two: the 2x anchor and the single logical size picked in the app. One rung is not a ladder, so the control never appears.

Counting modes where pixelWidth > width and isUsableForDesktopGUI() is true, on a 2304x1440 90Hz display:

2304x1440  <- px 4608x2880   usable
1152x720   <- px 2304x1440   NOT usable   <- exactly half the panel, the natural "Default" step
 960x600   <- px 1920x1200   usable       <- macOS generic fallback, not the panel's aspect
 800x600   <- px 1600x1200   usable       <- generic fallback, 4:3 on a 16:10 panel

Three usable rungs, only one of them actually on the panel's aspect ratio. For comparison, the built-in display on the same Mac reports 72.

Fix

Register five logical rungs when HiDPI is on, at 1.0 / 0.875 / 0.75 / 0.625 / 0.5 of the chosen size. The chosen size stays the top rung, so the app still sets the ceiling and macOS scales down from there.

Rung heights are derived from the scaled width rather than scaled independently, so every rung holds the chosen aspect ratio. A rung that drifted off it would letterbox on the client. Sizes are rounded to even numbers because the physical mode is exactly twice the logical one, and duplicates are dropped so a small chosen resolution cannot register the same mode twice.

The non-HiDPI path is unchanged: still a single mode at the requested resolution.

Verification

Measured on a real 2304x1440 90Hz virtual display, before and after:

before: 3 desktop-usable HiDPI rungs
after:  7 desktop-usable HiDPI rungs

  2304x1440 <- px 4608x2880
  2016x1260 <- px 4032x2520
  1728x1080 <- px 3456x2160
  1440x900  <- px 2880x1800
  1152x720  <- px 2304x1440
  960x600, 800x600 (macOS fallbacks, unchanged)

1152x720 is the clearest signal: it existed before but reported isUsableForDesktopGUI() false. Registering it explicitly flips it to true, which is what gives macOS a real "Default" step to anchor the picker on.

Builds clean; 43/43 tests pass, including 8 new ones covering rung ordering, aspect preservation across four panel shapes, even-sizing, duplicate collapsing, and the zero-size guard.

Note

The ladder is most useful alongside a stream size bounded by what the client can decode. At 2304x1440 HiDPI the framebuffer is 4608x2880, which is past what a typical tablet decoder sustains, so without such a bound the higher rungs produce a black screen rather than a sharper picture. That bound is a separate change and not required for this one to be correct.

Refs #41

@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

@meta-boy is attempting to deploy a commit to the tranvuongquocdat2-7001's projects Team on Vercel.

A member of the Team first needs to authorize it.

@tranvuongquocdat

Copy link
Copy Markdown
Owner

Thanks — the ladder logic and tests look right. The one thing I can't verify without a device is switching rungs from System Settings while connected: the framebuffer changes size under an SCStream that's still configured for the old physical size, and we deliberately don't hook didChangeScreenParametersNotification. Did you try that? If it just works (or you add a capture restart on mode change), I'm happy to take this in the next release. Holding it out of 0.11.3 for now; #62 and #64 are going in.

@meta-boy

meta-boy commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — the ladder logic and tests look right. The one thing I can't verify without a device is switching rungs from System Settings while connected: the framebuffer changes size under an SCStream that's still configured for the old physical size, and we deliberately don't hook didChangeScreenParametersNotification. Did you try that? If it just works (or you add a capture restart on mode change), I'm happy to take this in the next release. Holding it out of 0.11.3 for now; #62 and #64 are going in.

Tested on a Pixel 11 rather than a tablet, macOS 26.5.2. Switching rungs mid-session works.

Seven switches from System Settings across two app resolutions, live session both times. No dropped frames, no reconnect, steady 36 to 42 fps.

It holds because of #62. Encode size is clamp(physical, client limit), and every rung the ladder registers shares one aspect ratio, so when the client's limit is below the doubled framebuffer they all clamp to the same size. With the Pixel advertising 2416x1072:

rung 1280x800 -> physical 2560x1600 -> encode 1712x1072
rung 1120x700 -> physical 2240x1400 -> encode 1712x1072
rung  960x600 -> physical 1920x1200 -> encode 1712x1072

SCStream's configured size never changes, so it never goes stale.

Where it does go stale: macOS also lists fallback modes that are off the ladder's aspect. This setup offered 4:3 800x600, and switching to it keeps the old 16:10 config and letterboxes. Black bars left and right, picture still correct. A fresh session at that resolution configures 1424x1072 and fills the frame, so the bars come purely from the stale config. It recovers on the next rung, but you're right that it isn't free.

The same staleness hits the desktop size from #64. The overlay kept reporting "desktop 1920x1200" after macOS had moved the desktop to 960x600, because setDesktopSize only ever receives the app's dropdown value.

One correction: VirtualDisplayManager already observes didChangeScreenParametersNotification. It came in with #39 and stays live for the whole session, it just calls ensurePhysicalDisplayStaysMain(). So this is one more call inside a hook that already fires, not a new observer.

Both fix together, and negotiate() already has the shape: compare encodeSize before and after, call rebuildEncoder() only when it differs. That no-ops in the common case above and fixes the letterbox plus the stale desktop size otherwise. I can add it to this PR, or leave #63 as-is and do it separately.

@tranvuongquocdat

Copy link
Copy Markdown
Owner

Great test, thanks — and you're right about the observer, my mistake. Please add it to this PR: hooking the encode-size compare and the desktop-size refresh into the existing screen-params handler is the right shape, and the ladder is what makes mid-session switching a real path. I'll take it for 0.11.4 once that's in.

A HiDPI virtual display registered only two modes: the 2x anchor and the
single logical size picked in the app. macOS builds its "Larger Text …
More Space" control in System Settings from a display's desktop-usable
HiDPI modes, so one logical mode means that control never appears and
scaling can only be changed from inside Side Screen, which tears the
display down and restarts the stream.

Register five logical rungs instead, at 1.0/0.875/0.75/0.625/0.5 of the
chosen size. The chosen size stays the top rung, so the app still sets
the ceiling and macOS scales down from there.

Measured on a 2304x1440 90Hz virtual display, counting modes where
pixelWidth > width and isUsableForDesktopGUI() is true:

  before: 3 rungs — 2304x1440, plus 960x600 and 800x600, which are
          macOS's generic fallbacks and not on the panel's aspect
  after:  7 rungs — the five registered ones plus those fallbacks

1152x720 is the clearest case: exactly half the panel, and the natural
"Default" step. It existed before but reported isUsableForDesktopGUI()
false; registering it explicitly flips it to true.

Rung heights are derived from the scaled width rather than scaled
independently, so every rung holds the chosen aspect ratio — a rung off
it would letterbox on the client.

Refs tranvuongquocdat#41
The ladder makes mid-session scaling a real path, so the encode setup has
to survive the user switching rungs while a client is attached. macOS
changes the display's mode underneath a live SCStream, which stays
configured for the previous physical size.

Most switches need no work. Every rung the ladder registers shares one
aspect ratio, so when the client's ceiling is below the doubled
framebuffer they all clamp to the same encode size. macOS also lists
fallback modes of its own that are off that aspect, and those do change
it; leaving the old configuration in place letterboxes the picture into
a frame shaped for the previous mode.

VirtualDisplayManager already observes didChangeScreenParametersNotification
(added for tranvuongquocdat#39), so this hooks the existing handler rather than adding
one. That notification fires for every topology change, so the handler
compares against the mode it last saw and stays silent unless this
display actually switched.

ScreenCapture records the size its live encoder was built for, since a
mode change alters the display underneath both encoder and stream and
the stored value is the only way to tell whether the new encode size
differs. When it does, the existing rebuildEncoder() path runs, the same
one negotiate() uses for a codec or ceiling change.

The desktop size refreshes on every mode change, not only when the
encode size moves: it is the rung the user picked, and it was reporting
the app's dropdown value indefinitely.

Measured on a Pixel 7 (client ceiling 2400x1072), Mac at 1280x800 HiDPI:

  rung 1120x700 (16:10)  encode size unchanged at 1712x1072, desktop -> 1120x700
  rung  960x600 (16:10)  encode size unchanged at 1712x1072, desktop ->  960x600
  rung  800x600 (4:3)    encode size 1712x1072 -> 1424x1072, desktop ->  800x600

35 to 39 fps throughout, no dropped frames, no reconnect. Before this
the 4:3 rung kept the 16:10 configuration and letterboxed, and the
overlay kept reporting the resolution picked in the app.
@meta-boy
meta-boy force-pushed the feat/hidpi-scaled-modes branch from 0ca505b to 4d01dcc Compare September 12, 2026 04:08
@meta-boy

meta-boy commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Added, and rebased onto 0.11.3 while I was in there. Two commits now: the ladder, then the mode-change handling.

It hooks the existing #39 observer rather than adding one. That notification fires for every topology change, so the handler compares against the mode it last saw and stays quiet unless this display actually switched.

ScreenCapture now records the size its live encoder was built for. negotiate() can read its "before" size off state it is about to mutate, but a mode change moves the display underneath encoder and stream both, so the stored value is the only way to know what the live encoder was actually built for. When the new size differs it goes through the same rebuildEncoder() path negotiate() already uses.

The desktop size refreshes on every mode change rather than only when the encode size moves. It is the rung the user picked, and it was reporting the app's dropdown value indefinitely.

Re-tested on a Pixel 7 (Sorry I just have a lot of test pixel devices), ceiling 2400x1072, Mac at 1280x800 HiDPI:

rung 1120x700 (16:10)  encode size unchanged at 1712x1072, desktop -> 1120x700
rung  960x600 (16:10)  encode size unchanged at 1712x1072, desktop ->  960x600
rung  800x600 (4:3)    encode size 1712x1072 -> 1424x1072,  desktop ->  800x600

35 to 39 fps throughout, no dropped frames, no reconnect. The 4:3 rung fills the frame now instead of letterboxing, and the overlay reads "1424x1072 (desktop 800x600)" rather than the stale value it showed before.

Worth noting the common case stays free: switching among ladder rungs still does no encoder work, because they all clamp to the same encode size. Only the off-aspect fallback modes macOS adds itself trigger a rebuild.

57/57 tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants