[screen-toolkit][FEATURE] Support "Annotate Window" on Niri

[the-simen]

Submission checklist

• I have searched existing issues and confirmed this has not been requested before.
• I have checked existing pull requests for similar changes.
• This is about a community plugin, not about Noctalia itself (those belong in the noctalia repo).

Plugin

screen-toolkit

Feature type

Integration support

Feature summary

The Window tool in Screen Toolkit does not work when running Noctalia with Niri.

All other Screen Toolkit features I have tested work correctly on Niri, including screen recording and markup/annotation. The issue appears to be specific to the Window annotation/capture action.

From the current implementation, annotateWindow() appears to be restricted to Hyprland, and the capture script uses hyprctl activewindow -j to obtain the active window geometry.

For example, the current logic is effectively:

function annotateWindow() {
    if (root.isHyprland)
        root.runAnnotateActiveWindow()
}

and the capture script relies on:

hyprctl activewindow -j

As a result, on Niri the Window action does nothing.

Expected behavior

The Window action should capture the currently focused Niri window and open it in the Screen Toolkit annotation/markup interface, just like it does on Hyprland.

Motivation / use case

Doesn't work on niri

Proposed solution

Niri provides IPC for window information:

niri msg -j windows

The focused window can be identified from the returned JSON and its geometry could potentially be used to construct the capture region.

Alternatively, Niri itself provides:

niri msg action screenshot-window

which can capture the currently focused window.

It would be great if Annotate Window could support Niri using one of these approaches.

Alternatives considered

No response

References / related projects

No response

Additional context

No response

[github-actions]

CC @Ahmed5Emad

[Ahmed5Emad]

Nice I really wanted to fix this anyway but I need someone to test , like change hyprland commands with niri and test it!? , I really don't want to start doing pull requests that , I can't even test , that is why I ignaored , so you have 2 option either test it yourself and give tested code , I will add window manager check then your fix , or you could just add the code and checker and make pull request to it .

I know I am not very helpful 😅 but I really can't test it and I don't want to blindly try to fix it.

[the-simen]

I tested the Niri screenshot API. niri msg action screenshot-window --path ... works correctly and saves the focused window as a PNG, but Niri doesn't provide the window geometry through this action.

I also checked niri msg -j windows. It provides the window's position within its column, but not its actual position on the screen, so it doesn't seem to be usable for this purpose. I couldn't find any commands to get the window's horizontal position in pixels, I think this is due to the specifics of horizontal scrolling in Niri.

I don't know the QML side of the plugin well enough to implement this cleanly. I tried to implement it with AI, but couldn't get it working.

Could you take over the QML integration? I can provide the Niri-specific capture behavior and testing results.

[Ahmed5Emad]

First we use lua now , second from what you said I can't get position, I don't know if I use niri api , if it will be good, like if there's a ready api for that it is better to use it with binds , and from what I checked this plugin sizes so grim can capture and send it to be edited, it mostly use shell for of that no qml or lua

[Ahmed5Emad]

I am telling you to try fixing not because I don't want or can't fix it, I just can't test it , anyway tell your AI to change the capture.sh for capturing window so it uses your niri api instead of grim and then send it to the annotation app, if this work send the file or code here , I will implement it

[the-simen]

Here is the capture.sh file. Screenshots are working, but I ran into an issue with the window picker—I couldn't get it to work. So, the current workflow involves aiming with the mouse—without actually focusing the window (in case hovering triggers focus)—and then activating the plugin.

#!/usr/bin/env bash
# capture.sh <action> [args...]
#
# Actions:
#   annotate-window          — capture the focused window
#                              output: /tmp/screen-toolkit-annotate.png
#                              stdout: "X,Y WxH" geometry string
#   palette   <geometry>     — extract 8 dominant hex colours from a captured region
#                              stdout: one "#RRGGBB" per line
#   qr        <geometry>     — capture a region and decode any QR / barcode found
#                              stdout: decoded text
#
# Exit codes:
#   1 — missing / invalid arguments
#   2 — capture or decode failed
#   3 — missing dependency
#
# Used by: service.luau

set -euo pipefail

ACTION="${1:-}"
GRIM_CURSOR_ARGS=()
[ "${SCREEN_TOOLKIT_CAPTURE_CURSOR:-0}" = "1" ] && GRIM_CURSOR_ARGS+=(-c)

_require() {
    command -v "$1" >/dev/null 2>&1 \
        || { echo "ERROR: missing dependency: $1" >&2; exit 3; }
}

case "$ACTION" in

  annotate-window)
    if NIRI_SOCKET=${NIRI_SOCKET:-$(ls /run/user/$(id -u)/niri*.sock 2>/dev/null | head -1)}; then
      export NIRI_SOCKET
      _require niri
      _tmp="/tmp/screen-toolkit-annotate-tmp-$$.png"
      rm -f "$_tmp"
      niri msg action screenshot-window --path "$_tmp" \
        || { rm -f "$_tmp"; echo "ERROR: niri screenshot-window failed" >&2; exit 2; }
      _i=0; while [ ! -s "$_tmp" ] && [ "$_i" -lt 50 ]; do
        sleep 0.1; _i=$((_i + 1))
      done
      [ -s "$_tmp" ] \
        || { rm -f "$_tmp"; echo "ERROR: niri screenshot timed out" >&2; exit 2; }
      mv "$_tmp" /tmp/screen-toolkit-annotate.png
      printf '%s\n' "0,0 0x0"
    elif [ -n "${HYPRLAND_INSTANCE_SIGNATURE:-}" ]; then
      _require slurp
      _require hyprctl
      _require jq
      _require grim
      PT=$(slurp -p 2>/dev/null) || { echo "ERROR: cancelled" >&2; exit 1; }
      X=$(printf '%s' "$PT" | awk -F'[, ]+' '{print int($1)}')
      Y=$(printf '%s' "$PT" | awk -F'[, ]+' '{print int($2)}')
      WIN=$(hyprctl clients -j 2>/dev/null | jq -c --argjson x "$X" --argjson y "$Y" '
          [ .[] | select(.mapped == true)
            | { at: (.at // [0, 0]), size: (.size // [0, 0]) }
            | select(.at[0] <= $x and (.at[0] + .size[0]) >= $x
                 and .at[1] <= $y and (.at[1] + .size[1]) >= $y) ]
          | sort_by(.size[0] * .size[1]) | first' 2>/dev/null)
      [ -n "$WIN" ] && [ "$WIN" != "null" ] \
          || { echo "ERROR: no window at that point" >&2; exit 2; }
      GEOM=$(printf '%s' "$WIN" | jq -r '"\(.at[0]),\(.at[1]) \(.size[0])x\(.size[1])"' 2>/dev/null)
      [ -n "$GEOM" ] \
          || { echo "ERROR: could not parse window geometry" >&2; exit 2; }
       sleep 0.15
       grim "${GRIM_CURSOR_ARGS[@]}" -g "$GEOM" /tmp/screen-toolkit-annotate.png 2>/dev/null \
          || { echo "ERROR: grim capture failed" >&2; exit 2; }
      printf '%s\n' "$GEOM"
    else
      echo "ERROR: no supported compositor detected" >&2; exit 2
    fi
    ;;

  palette)
    GEOMETRY="${2:-}"
    [ -n "$GEOMETRY" ] || { echo "ERROR: palette: missing <geometry>" >&2; exit 1; }
    _require grim
    _require magick

    FILE="/tmp/screen-toolkit-palette.png"

     sleep 0.15
     grim "${GRIM_CURSOR_ARGS[@]}" -g "$GEOMETRY" "$FILE" 2>/dev/null \
        || { echo "ERROR: palette: grim capture failed" >&2; exit 2; }

    magick "$FILE" -alpha off +dither -colors 8 -unique-colors txt:- 2>/dev/null \
        | grep -v '^#' \
        | grep -oP '#[0-9a-fA-F]{6}' \
        | head -8
    ;;

  qr)
    GEOMETRY="${2:-}"
    [ -n "$GEOMETRY" ] || { echo "ERROR: qr: missing <geometry>" >&2; exit 1; }
    _require grim
    _require zbarimg

     sleep 0.15
     grim "${GRIM_CURSOR_ARGS[@]}" -g "$GEOMETRY" /tmp/screen-toolkit-qr.png 2>/dev/null \
        || { echo "ERROR: qr: grim capture failed" >&2; exit 2; }

    zbarimg -q --raw /tmp/screen-toolkit-qr.png 2>/dev/null
    ;;

  *)
    echo "ERROR: unknown action '${ACTION}'. Expected: annotate-window | palette | qr" >&2
    exit 1
    ;;

esac

[Ahmed5Emad]

Ok I will add this to plugin for niri , I just wish it doesn't break and cupture on niri 😅

[the-simen]

It should work, but I couldn't solve the window picker issue. Since it's impossible to retrieve a window based on coordinates, the IPS only passes column data rather than the horizontal coordinate. You'll simply have to try not to overlap with an adjacent window; however, if there are many floating windows—or a 3-window grid on a single viewport—it won't be possible to capture screenshots of certain windows.

[Ahmed5Emad]

I added it after it is merged test it, if not working reopen the issue