Otter

English · 繁體中文

A memory-first personal agent that lives on your machine.

Otter is a long-running personal AI agent built around one idea: memory is the product, not a feature. It remembers what you tell it across restarts, distills conversations into durable long-term memory, learns lessons from its own mistakes, works a night shift while you sleep, and reports back in the morning — all running locally, talking to you over Telegram, a terminal TUI, or a local web GUI.

It is the flagship/reference agent for memory-river, a framework-neutral long-term memory engine, and runs on the pi-coding-agent runtime with DeepSeek as the default brain.

Status: early (0.x), a personal project shaped by daily real use. APIs and behavior move. Single-owner by design — this is an agent you run for yourself, not a multi-tenant service.

What it does

Memory that actually closes the loop. Every turn, relevant memories are recalled and injected into context automatically. When a conversation goes idle, it is distilled into semantic capsules and written into a LanceDB store with three tiers (short-term capsules, mid-term granules, long-term originals) and an aging model — memories decay, get demoted, and can be archived instead of growing forever. Sessions survive restarts: the agent resumes where it left off. The agent also has explicit memory tools (memory_recall, memory_store, memory_update, memory_rehydrate) and is taught a gap-aware discipline for using them: recall before answering, rehydrate originals when summaries aren't enough, never bluff from a partial match.

A night shift. On a schedule (default around 03:00), Otter runs autonomous rounds under a hard token budget: it distills hindsight lessons from the day's real errors (so the same mistake surfaces as advice before a similar action next time), visually patrols a watchlist of web pages and diffs what changed, and consolidates memory. In the morning it sends a brief of what it did and what it found.

Eyes and hands, gated. The brain is text-only DeepSeek, but Otter reads images and PDFs through a vision tool (Gemini), captures full-page screenshots of URLs through a PixelRAG sidecar, does autonomous multi-step web research through a browser-use sidecar (domain-fenced at the browser layer, not by trusting the LLM), and searches the web via your SearXNG instance. On your machine it has real shell/file tools — behind a four-tier tool guard: read-only is free, memory writes are audited, general writes are audited and blocked in autonomous rounds, and dangerous operations either require a confirmation button in Telegram or are hard-blocked outright (mkfs, dd to devices, destructive rm, …). Tier T1/T2/T3 decisions land in an audit log (read-only T0 actions don't). The hard-block list is a best-effort net over shell command text, not a formal sandbox — see the security section.

Hygiene by default. Text is run through a best-effort, pattern-based secret scanner before it enters derived memory (it catches common key formats — not everything). Memory directories are written 0700/0600. Telegram delivery is durable: offsets commit only after a reply is actually delivered, and after a crash the pending prompt is replayed exactly once. Note that the crash-replay record and the tool-audit log store text before redaction, so treat the whole runtime state tree as private.

A persona you own. Character, owner profile, and memory doctrine live in three markdown files under persona/. Your real profile (persona/USER.md) is gitignored; a template ships in its place.

Architecture

flowchart LR
    subgraph channels [Channels]
        TG[Telegram]
        TUI[Terminal TUI]
        GUI[Local web GUI]
    end
    subgraph agent [Otter daemon]
        BRAIN["pi runtime + DeepSeek brain"]
        GUARD[Tool guard T0–T3]
        SCHED[Night scheduler]
    end
    subgraph memory [memory-river]
        RECALL[autoRecall each turn]
        DISTILL[idle distillation]
        STORE[(LanceDB store)]
    end
    channels --> BRAIN
    BRAIN --> GUARD
    SCHED --> BRAIN
    RECALL --> BRAIN
    BRAIN --> DISTILL
    DISTILL --> STORE
    STORE --> RECALL

Loading

The memory layer is a pure pi extension over @memory-river/core: recall hooks into context assembly, distillation hooks into idle/compaction, and the store is plain LanceDB on disk (~/.otter/memory-river by default) with a local Ollama embedder — no cloud memory service involved.

Requirements

• Node.js ≥ 22.13 (the locked dependency set requires it)
• Ollama with the embedding model: ollama pull hf.co/Qwen/Qwen3-Embedding-0.6B-GGUF (memory search runs 100% locally)
• DeepSeek API key for the brain (provider is configurable via OTTER_AGENT_PROVIDER)
• Optional: Gemini API key (vision/document reading), Telegram bot (live channel), SearXNG endpoint (web search), Python ≥ 3.12 for the screenshot & browser-research sidecars — these are off until you set them up (run scripts/setup-pixelrag.sh and create the browser-research venv); without setup, those specific tools are simply unavailable and the rest of the agent runs fine.

Quick start (terminal)

git clone https://github.com/Hsi431/otter-agent ~/otter-agent
cd ~/otter-agent
npm install
ollama pull hf.co/Qwen/Qwen3-Embedding-0.6B-GGUF

export DEEPSEEK_API_KEY=sk-...
npm run tui

Talk to it. Tell it something worth remembering, quit, start it again, and ask what you told it — the answer comes back from the store, not from luck.

To make it yours, copy the persona template and fill it in:

cp persona/USER.template.md persona/USER.md

All configuration is environment variables; see .env.example for the full annotated list. (For historical reasons the DeepSeek key also falls back to ~/.openclaw/openclaw.json if the env var is unset — harmless if you don't have OpenClaw, but set DEEPSEEK_API_KEY explicitly and you never touch that path.)

Running the Telegram service

1. Create a bot with @BotFather, get the token and your numeric chat id.

2. Create ~/.config/otter-agent/telegram.env (no export prefixes — systemd reads it):

   OTTER_TELEGRAM_BOT_TOKEN=123456:ABC...
   OTTER_TELEGRAM_SERVICE_LIVE_ALLOWED_CHAT_ID=<your chat id>
   GEMINI_API_KEY=...        # optional, enables vision
   OTTER_GUI=1               # optional, local GUI on 127.0.0.1:8787

   chmod 600 it.

3. Install the systemd user unit (the unit expects the repo at ~/otter-agent):

   cp systemd/otter-telegram-live.service ~/.config/systemd/user/
   systemctl --user daemon-reload
   systemctl --user enable --now otter-telegram-live
   journalctl --user -u otter-telegram-live -f

The service answers only the allowed chat id — everyone else is ignored. /status in the chat shows daemon health without waking the brain.

Night shift and the morning brief are on by default once the service runs (tune with OTTER_NIGHT_AT, OTTER_BRIEF_AT, OTTER_NIGHT_TOKEN_BUDGET). To give the visual patrol something to watch, just tell Otter to keep an eye on a page, or edit ~/.config/otter-agent/watchlist.json.

Security posture, honestly

Otter is a single-owner agent that runs with your privileges. Its threat model is "help you," not "resist a determined attacker who controls its inputs." Read this before you run it:

• It runs real shell and file tools on your machine. The guard tiers and hard-block list raise the bar (and autonomous night rounds use a much stricter whitelist), but a model with bash can obviously read a file and send it out, or write files through shell redirection — that's the nature of a capable agent, not a defect. The hard-block list (dd, mkfs, destructive rm) is a best-effort filter over command text; it is not a formal sandbox and can be worked around with enough shell creativity. Run Otter under a dedicated user, or on a machine you're comfortable giving an LLM shell access to.
• Prompt injection through fetched content is unsolved. Everything the browser/vision sidecars bring back is treated as untrusted evidence, but content you tell Otter to read can try to steer it. Because it acts with your privileges, assume a hostile page could make it act — the guard tiers limit the blast radius, they don't eliminate it.
• SSRF: the text web_fetch path validates and pins DNS to block private-network access. The screenshot (PixelRAG) and browser-research sidecars do not yet pin their own connections, so don't point them at hostnames you don't trust on a machine with sensitive internal services.
• The web GUI binds to loopback and has no authentication. /prompt is a live command endpoint, not just a dashboard — any local process (or a web page that can reach loopback) can drive the agent. Don't port-forward it; treat localhost as trusted only if it is.
• Memory, media, and the crash-replay/audit state are plain data on disk (memory dirs are 0700/0600), not encrypted at rest. The whole ~/.otter and ~/.local/state/otter-agent tree is private-by-permission; back it up accordingly.

Development

npm test          # full suite, no network or API keys needed
npm run memory:loop-check   # inspect the distillation pipeline state

Tests run against fake providers and in-memory transports; the suite is the contract for every subsystem (memory loop, Telegram durability, tool guard, scheduler, vision plumbing).

License

Apache-2.0. Built on pi-coding-agent, memory-river, PixelRAG, and browser-use.