PicoBot

PicoBot is a perception-driven game bot. A Raspberry Pi Pico running the TinyUSB firmware in firmware/phase-e/k75 acts as a real HID device and relays keyboard/mouse inputs to a computer — inputs arrive exactly like a physical keyboard. On top of that transport, a smart bot watches the game's minimap, navigates between anchor points, and fires skills off per-skill cooldowns, mimicking how a real player works a farming rotation.

The first version of the firmware ran CircuitPython (a COM port and the Adafruit HID library); it is kept on the legacy/circuitpython branch, next to the original Python host on legacy/python.

This project is purely for learning purposes. Picobot was built for my personal botting needs in a Maplestory private server. Botting is a punishable offense, please use the program at your own risk.

Installation

The host is written in Rust (rust/). The original Python host is kept on the legacy/python branch.

1. Install Rust (https://rustup.rs) and Node.js.
2. Build the dashboard: cd web; npm install; npm run build.
3. Build the host: cd rust; cargo build --release (the first build downloads the ONNX runtime used for title OCR).
4. Title OCR needs PaddleOCR's recogniser, PP-OCRv6_rec_small.onnx: put it in models/ in the project folder (the file ships in the rapidocr Python package's models/ folder). Without it, map identity falls back to the pin.

Running

cd rust
cargo run --release -p picobot-host -- --root ..
# explicit overrides:  ... -- --root .. --port hid --window "Eluna (x64)"
# auto-detect the Pico: ... -- --root .. --port auto

--root is the project folder (where config.json, maps/, the reach files and web/dist live). The built binary works the same way: rust\target\release\picobot.exe --root . from the project root.

This starts the headless host: Pico link + WebSocket (default :8765) + HTTP dashboard (default :8000). Open http://localhost:8000 — the dashboard is the UI. The Pico link and game window can be picked from the dashboard's Connection panel (selects, or Auto to probe for the Pico); both are remembered in config.json so subsequent runs need no flags. CLI flags override the remembered values and are persisted the same way.

Telegram alerts use bot_token and chat_id in config.json; picobot --root . --notify-test sends one test alert and exits.

The Python host on legacy/python reads the same config.json, maps/ and reach files and uses the same ports and COM port — run one host at a time.

The dashboard at a glance

Mobile-first: on a phone it's four screens behind a bottom nav; on a desktop, two columns. See docs/dashboard.md.

• Home — live view, a class picker, one big Start/Stop.
• Control — rune solving: hazard banner, view, arrow pad and keys, plus a collapsed bot status. A rune or another player vibrates the phone and badges this tab.
• Setup — readiness checklist plus pages for the map, its layout, class (with skills and move keys), measuring moves, tuning (attacks, patrol with anchor stats, safety) and connection.
• Log — levelled events (debug < info < warn < error) with severity filters; HID/serial chatter sits at debug.

Drawing platforms and placing anchors happens on the desktop, with the tools under the view.

How it works

• Farming: anchors are patrol checkpoints — with ≥2 the bot plans a nearest-neighbour route from the player's position and weave-attacks between every checkpoint; single anchors weave in place. Movement geometry (platforms, walls, floor) is hand-drawn — authoritative, since auto-detecting translucent minimap lines proved too fragile.
• Map identity: map changes are detected from the loading blackout, the minimap panel is located by its white frame, and the map is named by its OCR'd title (voted, fuzzy-matched against map_name, read on a worker thread). Translucent UI can't trigger any of these.
• Safety: rune markers, other players, window-focus loss, or a map transfer (loading screen) pause the bot (Telegram alert if configured); remote input solves checks from a phone over Tailscale.

Documentation

Doc	Covers
docs/architecture.md	Components, layers, data flow, directory map
docs/map-detection.md	Blackout trigger, panel detection, title OCR, pins
docs/layout.md	Layout drawing, anchor placement, move measurement, map format
docs/bot-behavior.md	Patrol routes, weaving, skills, safety
docs/dashboard.md	Panels, views, event levels, WS protocol
docs/configuration.md	config.json reference
docs/development.md	Setup, tests, conventions, debug frame captures
docs/learnings.md	What was tried, what we learned
docs/future-plans.md	Decided ideas that are on hold

Remote connections over mobile data (Tailscale)

• Install Tailscale on the desktop host and your phone, sign in to the same tailnet, and enable MagicDNS.
• Open http://<host>.tail-xxxx.ts.net:8000 on the phone — the dashboard shows the live feed and the remote input pad works anywhere. WireGuard encryption means no port forwarding and no TLS needed.