Skip to content

fix(overlay): restore Wayland overlay on Tauri 2.11.5 (Linux) - #1700

Merged
cjpais merged 15 commits into
cjpais:mainfrom
xilec:fix/tauri-2.11.5-overlay-on-linux
Aug 10, 2026
Merged

fix(overlay): restore Wayland overlay on Tauri 2.11.5 (Linux)#1700
cjpais merged 15 commits into
cjpais:mainfrom
xilec:fix/tauri-2.11.5-overlay-on-linux

Conversation

@xilec

@xilec xilec commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Before Submitting This PR

Please confirm you have done the following:

  • I have searched existing issues and pull requests (including closed ones) to ensure this isn't a duplicate
  • I have read CONTRIBUTING.md

If this is a feature or change that was previously closed/rejected:

  • I have explained in the description below why this should be reconsidered
  • I have gathered community feedback (link to discussion below)

Human Written Description

The Tauri 2.11.5 update in commit 5a7c0ea broke
overlay rendering on my system, and based on issue #1696 it is also broken on some
other systems. This PR narrowly fixes the overlay for Linux while preserving the
overlay behavior on other platforms.

Related Issues/Discussions

Fixes #1696

Community Feedback

Bug fix confirmed by the reporter in #1696 (KDE/KWin). Verified independently on
niri (another Wayland compositor) where the overlay was similarly invisible.

Testing

  • Platform: Linux (NixOS, Wayland, niri).
  • Build: cd src-tauri && cargo build on Tauri 2.11.5 (pinned [patch.crates-io]
    tao/tao-macros to rev 07f3742b1833b64be27b1ef991e38d557d4276c9).
  • Manual verification on niri:
    • Overlay appears on the first transcription (previously invisible).
    • Overlay grows correctly with streamed text.
    • Runtime Top↔Bottom position switch works; overlay does not disappear.
    • Repeated show/hide cycles keep working.
  • Non-Linux platforms (Windows/macOS) are unchanged: all new code is under
    #[cfg(target_os = "linux")]; shared code was only moved into
    #[cfg(not(target_os = "linux"))] with no logic change.

Screenshots/Videos (if applicable)

AI Assistance

  • No AI was used in this PR
  • AI was used (please describe below)

If AI was used:

  • Tools used: opencode (AI coding assistant, model hy3-free, Kimi K2.7, reviewer Kimi K3)
  • How extensively: AI assisted with root-cause analysis, the GTK-surface
    helper refactor, and code review. The final fix and its behavior on
    Wayland were verified manually by the human contributor.

Wayland compositors (niri, KWin) only commit a layer surface on an
unmapped→mapped transition. On Tauri 2.11.5 the overlay never appeared,
and switching its position at runtime dropped it permanently.

Linux/Wayland changes (all under cfg(linux) where they touch the surface):
- Track gtk-layer-shell init in LAYER_SHELL_ACTIVE and skip layer-shell
  calls on the non-layer-shell fallback window.
- Prime the surface with a show()+hide() cycle so the first real show()
  maps it fresh instead of being a no-op on an already-mapped surface.
- In show_overlay_state, flip Tauri window visibility (overlay_window.show())
  so emit("show-overlay") reaches the WebView, and set anchors + show
  atomically on the GTK thread, synchronizing via a channel so emit runs
  only after the surface is actually mapped. set_size/set_position stay on
  the non-layer paths only (niri ignores them on mapped layer surfaces).
- Allocate the max overlay size up front on Linux; niri ignores set_size on
  mapped layer surfaces, so the overlay could not grow with streamed text.
- On hide, unmap the GTK surface and flip Tauri visibility so repeated
  shows keep working.
- On runtime position change, remap the surface (unmap → set anchors → map)
  so the compositor commits the new Top/Bottom anchors.

Non-Linux platforms (Windows, macOS) are unchanged from v0.9.3: compact
initial size, set_size/set_position, Windows topmost re-assert, and the
original timing debug log.

Refactor: extract repeated GTK-surface logic into named helpers —
pump_gtk_events(), with_gtk_window(), gtk_show_layer_surface(),
gtk_remap_layer_surface(), position_overlay_window() — folding the
show/hide + main_iteration cycles and run_on_main_thread boilerplate
that were duplicated across prime, show_overlay_state,
update_overlay_position and hide_recording_overlay. Behavior unchanged.

bindings.ts is intentionally left as the v0.9.3 generated content.
@xilec
xilec marked this pull request as ready for review July 16, 2026 21:53
On Wayland layer-shell, only remap the surface when it is currently
mapped; for an unmapped surface just set the anchors so they apply on
the next map. This keeps Tauri's window visibility untouched, so a
mid-recording position switch no longer hides the overlay and an idle
switch no longer flashes it.
@cjpais

cjpais commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Just to confirm this did work probably on your system?

@xilec

xilec commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Yes, the fix works on my system. I tested it as described in the PR's testing section, and I've been using Handy from this PR branch daily. I've also added a few small improvements based on further review.

Resolve overlay.rs conflicts by keeping the branch's GTK layer-shell
show/remap paths on Linux while adopting main's Windows placement
(place_windows_overlay, streaming-size cache) for the non-Linux paths.
Adopt main's main-thread overlay dispatch (cjpais#1810): show_overlay_state
and update_overlay_position now hop to the GTK main thread, keeping the
branch's layer-shell show/remap logic inside the *_on_main bodies.
@cjpais

cjpais commented Jul 31, 2026

Copy link
Copy Markdown
Owner

@xilec one quick question did you ever encounter the bug in #1806

@xilec

xilec commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@cjpais No — the main window has rendered correctly for me on NixOS/niri across all builds after the Tauri 2.11.5 switch. Left a note in #1806: #1806 (comment)

@xilec

xilec commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Hi @cjpais — just checking in on this one. The branch is now up to date with main (including the main-thread overlay dispatch from #1810), and I've re-verified the overlay locally on Wayland/niri: recording and streaming modes show, reposition (top/bottom), and hide correctly.

Is there anything currently blocking this? For example:

  • just a matter of finding time for review,
  • you'd like additional testing from other Linux contributors,
  • you're waiting for the transcribe-cpp 0.2.0 merge to land first,
  • or you'd like a more detailed description of how the new version works — here in the PR or somewhere else?

Happy to do whatever makes this easier to land.

@buz-ch

buz-ch commented Aug 6, 2026

Copy link
Copy Markdown

FWIW, I built this on Opensuse Tumbleweed and it fixes the issue on KDE Wayland. Been running it since yesterday, it seems perfectly stable.

Thanks a lot!

@cjpais

cjpais commented Aug 6, 2026

Copy link
Copy Markdown
Owner

I'm doing a review over this and there's possibly some changes I want to make. I'm just like a bit concerned about the changes as written

I think the feedback from the issue itself might be worth taking into account, like the other user who offered his perspective on implementation, it may be able to simplify this implementation and maybe be a bit more robust

It's just hard for me to review these Linux things unless I have time to really sit down in front of a Linux machine and make sure that it's solid basically if you could just take a look at the feedback from that other user, I would be grateful to hear some perspective

@xilec

xilec commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Took a look at @arimxyer's feedback as you asked — and went a bit further: I tested a cherry-pick of a7dea9c8 onto current main on niri.

Scenarios covered: first show, streaming growth (256x46 → 400x120 on a mapped surface), runtime Top↔Bottom switch including mid-recording, and repeated show/hide cycles. Everything works, so his set_size_request + layer-shell margin approach is now verified on two compositors (his KWin + niri here).

His analysis is correct, and it also names the problem this PR was working around: the fixed max-size window and "never resize a mapped layer surface" were my way around the same 0x0 surface he diagnosed properly.

On the extra mechanics in this PR (surface priming, remap on position change, sync before the show-overlay emit): I added those in July against failures I was hitting on niri at the time. In this round of testing the same scenarios worked without them — I suppose because #1810 moved all overlay geometry onto the main thread. So I agree the simpler implementation should be sufficient.

One caveat worth flagging from all this testing: there's a pre-existing glitch where the streaming card leaves faded pixels behind the compact overlay — WebKit doesn't clear the now-transparent region on state change. It reproduces on current main, on this PR, and on arimxyer's build alike, so it's orthogonal to the sizing approach. I'll investigate it separately.

orca-paste-1786029200011-1f4d7e75-60f9-4e1a-b13c-42fa2698b266

@arimxyer

arimxyer commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Happy to also contribute to the investigation! @xilec and @cjpais . Also happy to send in a PR if it provides any relief, I originally withheld and just focused on leaving the comment since the PR was already created.

@cjpais

cjpais commented Aug 7, 2026

Copy link
Copy Markdown
Owner

okay @arimxyer @xilec I made the changes directly here, can you please test them and let me know if we are good to merge this based on your testing?

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

🧪 Test Build Ready

Build artifacts for PR #1700 are available for testing.

Download artifacts from workflow run

Artifacts expire after 30 days.

@ctcac00

ctcac00 commented Aug 7, 2026

Copy link
Copy Markdown

I'm also on niri and I've just built this locally and the overlay is working again for me

@xilec

xilec commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@cjpais Tested on niri (multi-monitor Wayland). Almost everything works: first show, streaming growth 256x46 → 400x120 via set_size_request + resize(1, 1), runtime Top↔Bottom switch including mid-recording, correct 40/4px margins, repeated show/hide cycles.

One issue: the cursor-based set_monitor block misplaces the overlay on niri. Handy builds enigo with default features (the x11rb backend), so location() reads the pointer through XWayland — which only tracks the pointer while it's over an X11 window. Over native Wayland windows the coordinates are stale: I kept getting the identical point across app restarts and mouse movement, and monitor_at_point then picked the wrong output (the overlay landed on the monitor with Firefox, an X11 window, instead of the one holding the text-input focus).

I don't see a reliable client-side way to pick the monitor on Wayland: the global pointer position and the keyboard focus are deliberately not exposed to clients, and enigo's own Wayland backend has location() unimplemented for that reason. The alternatives are all dead ends — foreign-toplevel protocols say which app is active but not on which output, and compositor IPC (niri/sway/hyprland) would need per-compositor adapters.

With the set_monitor block removed, the output is left to the compositor — and per the layer-shell protocol, when no output is set the compositor maps the surface on the output the user most recently interacted with. Verified on niri: the overlay then appears on the monitor holding the current text-input focus, which is exactly where a dictation overlay belongs. So this looks like the intended mechanism rather than a fallback.

I'd suggest dropping the set_monitor block (keeping set_size_request + resize(1, 1) and the anchors + margins). Happy to push that change here — or feel free to adjust directly, whatever is easier for you.

@cjpais

cjpais commented Aug 9, 2026

Copy link
Copy Markdown
Owner

@xilec ive applied the fix, can you give it a build and test on your machine please?

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

🧪 Test Build Ready

Build artifacts for PR #1700 are available for testing.

Download artifacts from workflow run

Artifacts expire after 30 days.

@buz-ch

buz-ch commented Aug 9, 2026

Copy link
Copy Markdown

Just built it, works on KDE/Wayland.

Can test multi monitor on Saturday.

@xilec

xilec commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

@cjpais Built and tested on commit 709fb68 on niri (multi-monitor Wayland) — everything works:

  • the overlay now lands on the monitor holding the text-input focus (the case that was broken with set_monitor)
  • first show, streaming growth 256x46 → 400x120, runtime Top↔Bottom switch including mid-recording, repeated show/hide cycles — all good

Good to merge from my side. Thanks for the quick turnaround!

@cjpais

cjpais commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Thanks yall for the confirmation, merging

@cjpais
cjpais merged commit 9bcb6d9 into cjpais:main Aug 10, 2026
2 checks passed
@xilec
xilec deleted the fix/tauri-2.11.5-overlay-on-linux branch August 10, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v0.9.3 regression] Recording overlay invisible on KDE Wayland (KWin) — GTK layer-shell broken after tauri fork → upstream switch

5 participants