Skip to content

feat: Backend prerequisite changes for the new dashboard. - #686

Draft
GabuTheDev wants to merge 16 commits into
tosuapp:masterfrom
GabuTheDev:feat/dash-backend
Draft

feat: Backend prerequisite changes for the new dashboard.#686
GabuTheDev wants to merge 16 commits into
tosuapp:masterfrom
GabuTheDev:feat/dash-backend

Conversation

@GabuTheDev

@GabuTheDev GabuTheDev commented Aug 6, 2026

Copy link
Copy Markdown
Member

This PR aims to improve some server-side code and introduce needed features for the future dashboard.

Caution

This PR is work-in-progress and not ready for a review. Things may change.

Most code should be self-explainatory and easily readable, but to summarize:

Changes

1. WebSocket Cleaning & The Sockets Registry (4acb6ed, cbf71d1)

The entire WebSocket class was rewritten and transformed into a lightweight, decoupled WebSocketChannel class.

Key changes include:

  • Removed direct dependencies on InstanceManager and hardcoded state-getter function names (getState, getStateV2, etc.) from the class constructor.
  • Replaced monkey-patching native WebSocket instances with a structured WsConnection stored in a Map<string, WsConnection>.
  • Eliminated the internal while (true) infinite polling loops inside the class. Pushing data is now driven explicitly by caller-managed broadcast(data) and dispatchCommand(...) methods.
  • Updated broadcast() to cache JSON strings so payloads are serialized once per tick rather than re-strigified per connected client.
  • Replaced individual uppercase server properties with a unified sockets: Record<string, WebSocketChannel> registry and map-driven HTTP upgrade routing.

why?

  • In the old impl., each Websocket instance was tightly coupled to backend game state polling; every channel spawn it's own un-cancelable while (true) loop that reached into InstanceManager directly to fetch data.
  • Native ws objects were mutated directly with custom properties, which is sloppy and can introduce runtime type leaks.
  • Serialization ran in a loop per client, wasting CPU cycles when multiple clients were subscribed to the same stream.

my solution

  • Refactored WebSocketChannel into a pure, generic transport layer responsible only for connection management, command dispatching, and data broadcasting.
  • Moved polling responsibility outside the channel (managed dynamically via Task.recur), allowing the channel to broadcast pre-serialized JSON efficiently across all open sockets.

2. Modernized HTTP Server & Router Architecture (cbf71d1, fb49c13)

The core HttpServer class and router setup were completely rewritten:

  • Replaced method-keyed route arrays (routes: Record<string, Route[]>) with a unified route stack. String route paths are pre-compiled on registration into { pattern, keys } via pathToRegex(), eliminating redundant runtime regex compilation.
  • For POST, PUT, and PATCH requests, body streaming enforces a hard 10MB limit (maxBodyLength). If the incoming data stream exceeds 10MB, the server immediately emits 413 Payload Too Large and aborts the socket via req.destroy().
  • Replaced index-based middleware recursion with a standard next(err?:unknown) pipeline (Express Server standard, or so I have found on da internet). Middleware exceptions are caught cleanly and passed to next(exc) to render a 500 error instead of silently swallowing failures.

why this

  • The legacy router executed route.path.exec(pathname) up to three separate times per route match during request dispatching (once to test the regex, once to inspect groups, and once to pull values), creating unnecessary CPU overhead on every request.
  • Request body aggregation streamed chunks directly into string memory without payload limits, exposing the server to memory exhaustion or denial-of-service if large payloads were posted.
  • The middleware pipeline used a fragile manual index counter next(index + 1) that caught synchronous errors but didn't pass them downstream, risking hanging requests on failure.

3. The new /api/overlays API.

As of last year, the word counters (refering to pp counters) was abandoned in favor of overlays. I took the rename as a chance to also rewrite the internal API to a more robust RESTful API.

So, I introduced a centralized OverlaysService. The new REST API provides standardized management routes.
Here's a table to showcase the old-to-new mapping:

old new
GET / GET /api/overlays?status=installed
GET /available GET /api/overlays?status=available
GET /api/counters/search/:query GET /api/overlays
GET /api/counters/download/:url?name=... POST /api/overlays/:id/download
GET /api/counters/open/:name POST /api/overlays/:id/open
GET /api/counters/delete/:name DELETE /api/overlays/:id
GET /api/counters/settings/:name GET /api/overlays/:id/settings
POST /api/counters/settings/:name POST /api/overlays/:id/settings

whyyyy

  • Previously, overlay management logic was scattered across ad-hoc helper scripts, server-side template renderers (buildLocalCounters).
  • There was no single service layer or unified data model representing overlay state, status (installed vs available), metadata, or settings.

3. Removal of ?l= & Upgrade to Overlay Runtime Injection

Removed the legacy ?l= query flag parsing and window.COUNTER_PATH script string appending in favor of a modern runtime injection system.

Key changes include:

  • Renamed addCounterMetadata to injectOverlayRuntime(html, token) and updated static file serving to inject <head> runtime metadata instead of appending scripts to the end of the HTML body.
  • Generated scoped WebSocket session tokens (window.TOSU_TOKEN) per overlay directory via createOverlayToken() instead of passing raw directory paths in URL query strings (?l=folder_name).
  • Automatically injected CSS rules (background: transparent !important) into served overlay HTML files so they render seamlessly when embedded in dashboard previews.
  • Injected iframe checks (if (window.top !== window.self)) to suppress overlay console.log noise in the main dashboard web developer console.

4. Introduction of Task Asynchronous Task Utilities

Added a static utility class that standardizes recurring background loops, conditional polling, and debouncing.

idk what to write here. I guess usage?

Task.recur(getInterval, callback).

  • Schedules an async execution loop using non-blocking dynamic intervals.
  • Re-evaluates getInterval() on every cycle to dynamically adjust polling speed.
  • returns a TaskHandle with a .stop() teardown method. (is also self stoppable / loop can stop itself)

Task.once(predicate, action, checkIntervalMs)

  • Periodically polls a condition predicate (predicate()). Once the condition evaluates to truthy, it automatically terminates polling and executes the target action(result).

Task.debounce(delayMs, fn)

  • A simple debounce method. Nothing new here.
  • I know there always was a debounce function, but I'd rather have everything.

6. Downloader Engine Modernization & Streaming Checksums

I researched a bit and it seems like most of our issues (For example the SLL error) are because http.get is shit and deprecated. I replaced it with the native fetch function + Node stream pipelines.

  • Updated verifyDownload to calculate file hashes using a stream pipeline instead of reading the whole file into RAM.

@GabuTheDev GabuTheDev self-assigned this Aug 6, 2026
@GabuTheDev GabuTheDev added area:server The nodejs backend of tosu. breaking Introduces changes that break backward compatibility or alter existing behavior. changelog worthy Has meaning for users; should be mentioned in next changelog. labels Aug 6, 2026
@GabuTheDev GabuTheDev added the Skip CI Optional label to temporary disable CI jobs. label Aug 7, 2026
import fs from 'fs';
import path from 'path';

const OVERLAYS_API_URL = 'https://tosu.app/api.json';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should an array of urls to support mirrors incase if some countries have banned tosu.app ip's

Comment on lines +80 to +86
const width = parseInt(String(w), 10);
const height = parseInt(String(h), 10);

return {
width: Number.isNaN(width) ? null : width,
height: Number.isNaN(height) ? null : height
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. wtf is this
  2. use isRealNumber


private async fetchRepoOverlays(): Promise<Overlay[]> {
try {
const res = await fetch(OVERLAYS_API_URL);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add support for multiple hosts and timeout of 5_000ms

@storycraft storycraft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Never mix sync io in async code. Legacy codes were partially allowed because of architecture issues, but new codes must not.
  2. The new /api/overlays endpoints would be better without s.
    1. /api/overlays/:id/settings -> /api/overlay/:id/settings
    2. OverlaysService -> OverlayService

I think few small changes like downloader.ts can be cherry-picked and be done in another pr so this pr gets smaller.

return { stop };
}

public static debounce<T extends (...args: any[]) => void>(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can imagine debounce and Task.debounce being mixed everywhere in the future. If there are type issue in debounce function, please fix it instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:server The nodejs backend of tosu. breaking Introduces changes that break backward compatibility or alter existing behavior. changelog worthy Has meaning for users; should be mentioned in next changelog. Skip CI Optional label to temporary disable CI jobs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants