Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 21 additions & 10 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,28 @@ BIRDNETGO_URL=http://192.168.1.10:8080
# contributed art for. This fills in the rest: the refresh service holds
# BirdNET-Go's detection SSE stream and, the moment a species it can't download
# is heard, generates its kachō-e cutout. Until art exists the species is hidden
# (the collage only shows illustrated birds). Set a Google AI (Gemini) API key to
# enable generation; leave it unset to rely on downloads only. Uses the paid
# Gemini image API with YOUR key.
# (the collage only shows illustrated birds). Set an API key to enable
# generation; leave it unset to rely on downloads only. You pay for what it
# generates, with YOUR key.
# Persist the art with the named volume in docker-compose.yml so upgrades don't
# re-spend those API calls.
# GEMINI_API_KEY=
# GENERATE_API_KEY=

# Seconds to wait between image-API calls, to stay within the Gemini free-tier
# rate limit (default 6). This is the throughput knob: art is acquired one pose at
# a time, so this gap is the only thing pacing the calls. Lower it on a paid tier,
# raise it if you get throttled, 0 to remove the gap entirely. Needs GEMINI_API_KEY.
# Where those calls go. Any OpenAI-compatible chat/completions endpoint works,
# so this is also how you point Saezuri at a model on your own hardware
# (e.g. http://192.168.1.10:1234/v1 for LM Studio). Default: OpenRouter.
# GENERATE_API_URL=https://openrouter.ai/api/v1

# Which model draws the birds. The default is the one the illustration style was
# tuned against. Anything else is your experiment — the prompt asks for the bird
# on a flat magenta ground that Saezuri then cuts away, and a model that ignores
# that instruction produces a magenta rectangle instead of a cutout.
# GENERATE_MODEL=google/gemini-2.5-flash-image

# Seconds to wait between model calls, to stay within your provider's rate limit
# (default 6). This is the throughput knob: art is acquired one pose at a time, so
# this gap is the only thing pacing the calls. Lower it if your plan allows, raise
# it if you get throttled, 0 to remove the gap entirely. Needs GENERATE_API_KEY.
# GENERATE_SLEEP=6

# Prompt addenda for species the model keeps getting wrong, as a JSON object keyed
Expand All @@ -43,10 +54,10 @@ BIRDNETGO_URL=http://192.168.1.10:8080
# SPECIES_NOTES=/usr/share/nginx/html/assets/illustrations/_species-notes.json

# --- Free pre-made illustrations (on by default) ---
# A zero-cost alternative (or complement) to on-demand Gemini generation: the moment
# A zero-cost alternative (or complement) to on-demand generation: the moment
# BirdNET-Go reports a species, the refresh service downloads its ready-made kachō-e
# cutout from the saezuri-illustrations repo (via the jsDelivr CDN) into the volume —
# no API key needed. Composes with GEMINI_API_KEY: download is tried first, and a
# no API key needed. Composes with GENERATE_API_KEY: download is tried first, and a
# species the repo doesn't have still falls back to generation. Art persists in the
# volume, so it's fetched once. To DISABLE downloads, set ILLUSTRATIONS_REPO empty.
# ILLUSTRATIONS_REPO=vrwrts/saezuri-illustrations
Expand Down
8 changes: 6 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,12 @@ Where things live, so a change lands in the right place fast.
- **Art acquisition:** `src/server/generate.ts` (`Generator`). The unit of work is a **pose**,
not a species: a perched-only species already renders (`resolveArt`), so perched is what
lifts it off the fallback silhouette and flight only changes the 15% that roll for it.
Two independent lanes — repo download (concurrent, cheap) and Gemini generation (serial) —
so free art never queues behind a paid render. Precedence is fixed and deliberate: file on
Two independent lanes — repo download (concurrent, cheap) and model generation (serial) —
so free art never queues behind a paid render. The model is not baked in: the vendored
pipeline calls an OpenAI-compatible `chat/completions` endpoint, so `GENERATE_API_URL` /
`GENERATE_MODEL` reach OpenRouter by default and a local server just as well. The app itself
never calls a model — it reads `GENERATE_API_KEY` only as a feature gate and passes the
environment through to the pipeline. Precedence is fixed and deliberate: file on
disk > illustrations repo > local generation > fallback silhouette. The repo is the state of
the art, so `species-notes.json` (`notes.ts`) tunes the *generation* fallback only and never
replaces downloaded art; a changed note re-renders a pose only when `source: 'generated'`.
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ RUN mkdir -p /opt/canvas && cd /opt/canvas \
# We fetch only the pipeline.tar.gz asset (not the whole art repo), so the build
# stays lean. Bumping PIPELINE_VERSION (a normal app commit) adopts a new pipeline.
# NOTE: the illustrations repo must have published this release first.
ARG PIPELINE_VERSION=v1.1.0
ARG PIPELINE_VERSION=v2.0.0
RUN apk add --no-cache curl \
&& mkdir -p /build \
&& curl -fSL "https://github.com/vrwrts/saezuri-illustrations/releases/download/${PIPELINE_VERSION}/pipeline.tar.gz" \
Expand All @@ -38,7 +38,7 @@ RUN apk add --no-cache curl \
# alpine base: the cutout step is matte.py (numpy + scipy + Pillow, all with musl
# wheels), so the image needs no glibc-only wheels or baked matting model. The
# generator toolchain + Node service ship in every image; a display-only
# container (no GEMINI_API_KEY) still publishes, it just skips generation.
# container (no GENERATE_API_KEY) still publishes, it just skips generation.
FROM nginx:alpine AS runtime

# Python (pipeline: numpy / scipy / Pillow) + Node (the refresh service).
Expand Down
46 changes: 34 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,10 @@ annotated copy of every setting.
| `ILLUSTRATIONS_REPO` | `vrwrts/saezuri-illustrations` | Source repo for the free pre-made cutouts, downloaded per detected species. Set it **empty** to turn downloading off entirely. |
| `ILLUSTRATIONS_REF` | `main` | Branch or release tag to pull art from. Pin a tag for a fixed art set. |
| `ILLUSTRATIONS_BASE_URL` | derived jsDelivr URL | Overrides the whole download base URL, and wins over the two above. For testing against a local file server. |
| `GEMINI_API_KEY` | unset | Google AI (Gemini) key. Set it to *also* generate art for species the repo lacks (see below); unset relies on downloads only. |
| `GENERATE_SLEEP` | `6` | Seconds between image-API calls, to stay under the Gemini free tier. **The throughput knob**: lower it on a paid tier, raise it if you get throttled, `0` to remove the gap. |
| `GENERATE_API_KEY` | unset | API key for the image model. Set it to *also* generate art for species the repo lacks (see below); unset relies on downloads only. |
| `GENERATE_API_URL` | `https://openrouter.ai/api/v1` | Any OpenAI-compatible `chat/completions` base URL — including one on your own network, which is how you generate art locally. |
| `GENERATE_MODEL` | `google/gemini-2.5-flash-image` | Which model draws the birds. The default is the one the style was tuned against; see the caveat below before changing it. |
| `GENERATE_SLEEP` | `6` | Seconds between model calls, to stay under your provider's rate limit. **The throughput knob**: lower it if your plan allows, raise it if you get throttled, `0` to remove the gap. |
| `SPECIES_NOTES` | `_species-notes.json` beside the art | Prompt addenda for species that keep coming out wrong (see below). Layered over the set bundled with the pipeline. |

### Reference calls
Expand Down Expand Up @@ -172,12 +174,13 @@ rewrites `/etc/nginx/nginx.conf`, which an unprivileged container cannot do.
## On-demand generation (optional)

The free downloads above only cover species someone has contributed art for. To *also* fill in
anything the repo doesn't have — generated fresh in the same kachō-e style — set `GEMINI_API_KEY`:
anything the repo doesn't have — generated fresh in the same kachō-e style — set
`GENERATE_API_KEY`:

```bash
docker run -d -p 8090:8080 \
-e BIRDNETGO_URL=http://<birdnet-go-host>:8080 \
-e GEMINI_API_KEY=<your-google-ai-key> \
-e GENERATE_API_KEY=<your-openrouter-key> \
-v saezuri-illustrations:/data/illustrations \
ghcr.io/vrwrts/saezuri:latest
```
Expand All @@ -192,12 +195,31 @@ stop being a silhouette, so that render lands and shows up before the flight pos
The free downloads and the paid generation run independently, so a species whose art is already in
the repo appears immediately rather than queueing behind someone else's render.

### Choosing a model

Saezuri talks to an OpenAI-compatible `chat/completions` endpoint, so you are not tied to one
vendor. `GENERATE_API_URL` defaults to [OpenRouter](https://openrouter.ai), which fronts most
image-output models behind one account, and `GENERATE_MODEL` picks one from it. Point
`GENERATE_API_URL` at a local server instead and generation never leaves your network:

```bash
-e GENERATE_API_URL=http://192.168.1.10:1234/v1 \
-e GENERATE_MODEL=<the-model-your-server-serves> \
-e GENERATE_API_KEY=<whatever-your-server-accepts>
```

One real constraint on the model: **it has to honour the magenta background.** No image model
emits transparency, so the prompt asks for the bird on a flat `#FF00FF` ground and Saezuri cuts
that away afterwards. A model that ignores the instruction gives you a magenta rectangle rather
than a bird. `google/gemini-2.5-flash-image` is the default because the style was tuned against
it; anything else is worth checking on a species or two before you leave it running.

Things to know:

- **It uses the paid Gemini image API with _your_ key** — you pay for what it generates.
Only detected species the repo doesn't already have are generated (typically a handful).
Generation is paced by `GENERATE_SLEEP` (default 6s) to stay under the free tier; that gap is
the only throughput control, since the limit here is the API's request rate.
- **You pay for what it generates, with _your_ key.** Only detected species the repo doesn't
already have are generated (typically a handful). Generation is paced by `GENERATE_SLEEP`
(default 6s); that gap is the only throughput control, since the limit here is the provider's
request rate.
- **A pose the model declines is left alone for a day** rather than re-attempted every
refresh, so a stubborn species can't quietly drain quota. See
[Free illustrations](#free-illustrations) for how gaps are remembered.
Expand All @@ -206,7 +228,7 @@ Things to know:
- **The generator is bundled in every image** — vendored at build time from the
[saezuri-illustrations](https://github.com/vrwrts/saezuri-illustrations) pipeline at a pinned
version (numpy/scipy cutout, no ML model, so the `nginx:alpine` image stays modest).
`GEMINI_API_KEY` unset simply means the worker never generates; the container is otherwise identical.
`GENERATE_API_KEY` unset simply means the worker never generates; the container is otherwise identical.
- **A species that keeps coming out wrong** needs a better prompt, not more attempts — see
[Species notes](#species-notes).
- **Licensing.** Generating art locally for your own display is personal use. The style
Expand Down Expand Up @@ -241,7 +263,7 @@ How it behaves:
upstream](https://github.com/vrwrts/saezuri-illustrations) rather than keeping the fix local — the
pipeline ships its own `species-notes.json` that yours is layered over, and that is the file to
send a PR to.
- **It needs `GEMINI_API_KEY`.** A note is an instruction to the generator; with no key there is
- **It needs `GENERATE_API_KEY`.** A note is an instruction to the generator; with no key there is
nothing to instruct.

## Free illustrations
Expand All @@ -264,8 +286,8 @@ How it behaves:
it is first heard and kept in the volume, so a restart re-downloads nothing. A species only
appears once it has both. A fresh display fills in over the first hours
as birds are heard (not all at t=0).
- **Composes with on-demand generation.** Download is tried first (free); if `GEMINI_API_KEY`
is set, a species the repo *doesn't* have still falls back to Gemini generation.
- **Composes with on-demand generation.** Download is tried first (free); if `GENERATE_API_KEY`
is set, a species the repo *doesn't* have still falls back to local generation.
- **Requires `BIRDNETGO_URL`** — the refresh service (which fetches art and builds the manifest)
only runs when it's set.
- **Offline-safe / disable.** A failed fetch is non-fatal (silhouette until art exists). Set
Expand Down
25 changes: 21 additions & 4 deletions addon/DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,29 @@ generic silhouette, still labelled and still sized by their real count.
| **Illustrations repository** | `vrwrts/saezuri-illustrations` | Where cutouts are downloaded from. Empty turns downloading off. |
| **Illustrations branch** | `main` | Branch or tag to download from. |
| **Illustrations base URL** | derived | Overrides the two above with a direct URL. |
| **Gemini API key** | unset | Optional. Set it to *also* generate art, in the same style, for species nobody has contributed yet. |
| **Pause between generations** | `6` | Seconds between generated illustrations. Lower on a paid tier, raise if rate-limited. |
| **Image model API key** | unset | Optional. Set it to *also* generate art, in the same style, for species nobody has contributed yet. |
| **Image model endpoint** | OpenRouter | Advanced. Any OpenAI-compatible endpoint, including one on your own network. |
| **Image model** | `google/gemini-2.5-flash-image` | Advanced. Which model draws the birds. |
| **Pause between generations** | `6` | Seconds between generated illustrations. Lower if your plan allows, raise if rate-limited. |
| **Species notes** | none | Per-bird prompt corrections, see below. |

Generation costs money at Google's rates and is entirely optional. Everything works
without a key.
Generation costs money at your provider's rates and is entirely optional. Everything
works without a key.

The key is an [OpenRouter](https://openrouter.ai) key unless you change the endpoint.
Setting **Image model endpoint** to a server on your own network — for example
`http://192.168.1.10:1234/v1` for LM Studio — keeps generation local. Whatever model you
choose has to honour the prompt's flat magenta background, which the app then cuts away;
a model that ignores it gives you a magenta rectangle instead of a bird, which is why the
default is the model the style was tuned against.

### Upgrading from an earlier version

Two options were removed: **Gemini API key** (now **Image model API key**) and
**Generated illustrations per cycle** (unused since 0.15). Home Assistant validates your
saved configuration against the app's options, so if either is still in yours the app
will refuse to start until you remove it. Do that in the app's Configuration tab, then
set the new key.

Illustrations are generated one pose at a time, perched first. A bird stops being a
silhouette as soon as its perched illustration lands, so it appears without waiting
Expand Down
10 changes: 5 additions & 5 deletions addon/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,17 @@ schema:
illustrations_repo: str?
illustrations_ref: str?
illustrations_base_url: str?
gemini_api_key: password?
generate_api_key: password?
# Any OpenAI-compatible chat/completions endpoint, including one on your own
# network. Left empty: OpenRouter. See DOCS.md.
generate_api_url: str?
generate_model: str?
generate_sleep: int(0,600)
# "Scientific name|note" per entry, e.g.
# Turdus merula|Solid glossy black, orange-yellow bill and eye-ring.
# run.sh materialises these into the notes file the generator reads.
species_notes:
- str?
# Ignored since 0.15: art is acquired one pose at a time, so there is no batch
# to cap. Kept in the schema only so an existing configuration still validates;
# remove it from your options at your convenience.
generate_max_per_cycle: int(1,64)?
call_providers: str?
calls_max_per_cycle: int(1,64)
frame_width: int(100,4000)
Expand Down
7 changes: 5 additions & 2 deletions addon/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,9 @@ export_opt BIRDNETGO_TOKEN birdnetgo_token
export_opt ILLUSTRATIONS_REPO illustrations_repo
export_opt ILLUSTRATIONS_REF illustrations_ref
export_opt ILLUSTRATIONS_BASE_URL illustrations_base_url
export_opt GEMINI_API_KEY gemini_api_key
export_opt GENERATE_API_KEY generate_api_key
export_opt GENERATE_API_URL generate_api_url
export_opt GENERATE_MODEL generate_model
export_opt GENERATE_SLEEP generate_sleep
export_species_notes
export_opt CALL_PROVIDERS call_providers
Expand Down Expand Up @@ -201,8 +203,9 @@ redacted() { [ -n "${1:-}" ] && echo '<set>' || echo '<unset>'; }

log "BIRDNETGO_URL=${BIRDNETGO_URL}"
log "BIRDNETGO_TOKEN=$(redacted "${BIRDNETGO_TOKEN:-}")"
log "GEMINI_API_KEY=$(redacted "${GEMINI_API_KEY:-}")"
log "GENERATE_API_KEY=$(redacted "${GENERATE_API_KEY:-}")"
for _name in ILLUSTRATIONS_REPO ILLUSTRATIONS_REF ILLUSTRATIONS_BASE_URL \
GENERATE_API_URL GENERATE_MODEL \
GENERATE_SLEEP SPECIES_NOTES CALL_PROVIDERS CALLS_MAX_PER_CYCLE \
FRAME_WIDTH FRAME_HEIGHT FRAME_BG FRAME_SHADOW FRAME_WINDOWS \
SPECIES_DICT_LOCALES PUBLISH_DEBOUNCE_MS AGING_INTERVAL_MS \
Expand Down
29 changes: 19 additions & 10 deletions addon/translations/en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,28 +26,37 @@ configuration:
description: >-
Overrides the repository and branch above with a URL to download cutouts
from directly.
gemini_api_key:
name: Gemini API key
generate_api_key:
name: Image model API key
description: >-
Optional. Set it to also generate art for species nobody has contributed
an illustration for yet.
an illustration for yet. An OpenRouter key unless you change the endpoint
below. You pay for what it generates.
generate_api_url:
name: Image model endpoint
description: >-
Advanced. Any OpenAI-compatible endpoint, including one running on your
own network — for example http://192.168.1.10:1234/v1. Empty uses
OpenRouter.
generate_model:
name: Image model
description: >-
Advanced. Which model draws the birds. Empty uses
google/gemini-2.5-flash-image, the one the illustration style was tuned
against. A model that ignores the prompt's flat magenta background gives
you a magenta rectangle instead of a cutout bird.
generate_sleep:
name: Pause between generations
description: >-
Seconds to wait between generated illustrations. Lower it if you are on a
paid Gemini tier, raise it if you get rate-limited.
Seconds to wait between generated illustrations. Lower it if your plan
allows faster calls, raise it if you get rate-limited.
species_notes:
name: Species notes
description: >-
For birds that keep coming out wrong. One entry per bird, written as
"Scientific name|what it should look like" — for example "Turdus
merula|Solid glossy black, orange-yellow bill and eye-ring". Only affects
illustrations this app generates itself; downloaded ones are left alone.
generate_max_per_cycle:
name: Generated illustrations per cycle (unused)
description: >-
No longer used — illustrations are now generated one at a time. Safe to
remove from your configuration.
call_providers:
name: Recording archives
description: >-
Expand Down
Loading
Loading