diff --git a/.env.example b/.env.example index b82a8de..381226e 100644 --- a/.env.example +++ b/.env.example @@ -31,11 +31,16 @@ BIRDNETGO_URL=http://192.168.1.10:8080 # GEMINI_API_KEY= # Seconds to wait between image-API calls, to stay within the Gemini free-tier -# rate limit. Default matches the pipeline (6s). Only used with GEMINI_API_KEY. +# rate limit (default 6). This is the throughput knob: art is acquired one pose at +# a time, so this gap is the only thing pacing the calls. Lower it on a paid tier, +# raise it if you get throttled, 0 to remove the gap entirely. Needs GEMINI_API_KEY. # GENERATE_SLEEP=6 -# Cap species generated per pipeline run (default 4). Only used with GEMINI_API_KEY. -# GENERATE_MAX_PER_CYCLE=4 +# Prompt addenda for species the model keeps getting wrong, as a JSON object keyed +# by scientific name or slug. Defaults to _species-notes.json beside the art, so it +# lives in the persisted volume. Layered over the set bundled with the pipeline; +# only affects art Saezuri generates, never art downloaded from the repo. +# SPECIES_NOTES=/usr/share/nginx/html/assets/illustrations/_species-notes.json # --- Free pre-made illustrations (on by default) --- # A zero-cost alternative (or complement) to on-demand Gemini generation: the moment diff --git a/CLAUDE.md b/CLAUDE.md index a2fd6b6..9b6d9eb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,6 +150,19 @@ Where things live, so a change lands in the right place fast. gates/aggregates species, and publishes `/snapshot.json`, `/layout-manifest.json`, `/calls-manifest.json`, and the e-ink PNG frames (`render.ts`, reusing `computeLayout`). Run it with `npm run refresh:dev`. +- **Art acquisition:** `src/server/generate.ts` (`Generator`). The unit of work is a **pose**, + not a species: a perched-only species already renders (`resolveArt`), so perched is what + lifts it off the fallback silhouette and flight only changes the 15% that roll for it. + Two independent lanes — repo download (concurrent, cheap) and Gemini generation (serial) — + so free art never queues behind a paid render. Precedence is fixed and deliberate: file on + disk > illustrations repo > local generation > fallback silhouette. The repo is the state of + the art, so `species-notes.json` (`notes.ts`) tunes the *generation* fallback only and never + replaces downloaded art; a changed note re-renders a pose only when `source: 'generated'`. + `_art-state.json` in `assetsDir` records per pose: repo 404s (7-day TTL, so the repo isn't + re-probed every publish), provenance, and the note version. It is a cache, never a source of + truth — a bad entry must degrade to "probe again", never to "skip forever". The generate lane + owns the rate limit (`GENERATE_SLEEP`): the pipeline is invoked once per pose, so its own + inter-call sleep never fires. - **Reference calls:** `src/server/calls.ts` (`CallLibrary`) queues a lookup per newly-heard species, mirroring `generate.ts`; `callProviders/` holds one provider per archive behind a common interface — `find()` resolves null for "nothing here" (cacheable) and **throws** for diff --git a/Dockerfile b/Dockerfile index 69a705f..108367d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -24,7 +24,7 @@ RUN mkdir -p /opt/canvas && cd /opt/canvas \ # We fetch only the pipeline.tar.gz asset (not the whole art repo), so the build # stays lean. Bumping PIPELINE_VERSION (a normal app commit) adopts a new pipeline. # NOTE: the illustrations repo must have published this release first. -ARG PIPELINE_VERSION=v1.0.0 +ARG PIPELINE_VERSION=v1.1.0 RUN apk add --no-cache curl \ && mkdir -p /build \ && curl -fSL "https://github.com/vrwrts/saezuri-illustrations/releases/download/${PIPELINE_VERSION}/pipeline.tar.gz" \ diff --git a/README.md b/README.md index 0d4ef71..0d81d2c 100644 --- a/README.md +++ b/README.md @@ -66,8 +66,8 @@ annotated copy of every setting. | `ILLUSTRATIONS_REF` | `main` | Branch or release tag to pull art from. Pin a tag for a fixed art set. | | `ILLUSTRATIONS_BASE_URL` | derived jsDelivr URL | Overrides the whole download base URL, and wins over the two above. For testing against a local file server. | | `GEMINI_API_KEY` | unset | Google AI (Gemini) key. Set it to *also* generate art for species the repo lacks (see below); unset relies on downloads only. | -| `GENERATE_MAX_PER_CYCLE` | `4` | Cap on species generated per pipeline run. | -| `GENERATE_SLEEP` | pipeline default (`6`) | Seconds between image-API calls, to stay under the Gemini free tier. Read by the vendored pipeline, not by Saezuri — the default lives there. | +| `GENERATE_SLEEP` | `6` | Seconds between image-API calls, to stay under the Gemini free tier. **The throughput knob**: lower it on a paid tier, raise it if you get throttled, `0` to remove the gap. | +| `SPECIES_NOTES` | `_species-notes.json` beside the art | Prompt addenda for species that keep coming out wrong (see below). Layered over the set bundled with the pipeline. | ### Reference calls @@ -183,26 +183,67 @@ docker run -d -p 8090:8080 \ ``` The refresh service holds BirdNET-Go's detection SSE stream; the moment a new species is heard it -first tries the free download, and if the repo doesn't have it, generates a perched + flight cutout -(via the bundled pipeline), then refreshes the layout manifest the frontend polls. Silhouettes turn -into real birds on their own within seconds to hours. +first tries the free download, and if the repo doesn't have it, generates the cutout (via the +bundled pipeline), then refreshes the layout manifest the frontend polls. Silhouettes turn into +real birds on their own within seconds to hours. + +Art is acquired **one pose at a time**, perched first. A species needs only its perched cutout to +stop being a silhouette, so that render lands and shows up before the flight pose is even started. +The free downloads and the paid generation run independently, so a species whose art is already in +the repo appears immediately rather than queueing behind someone else's render. Things to know: - **It uses the paid Gemini image API with _your_ key** — you pay for what it generates. - Only detected species not already downloaded are generated (typically a handful). Generation is - paced by `GENERATE_SLEEP` (default 6s) to stay under the free tier and capped per cycle by - `GENERATE_MAX_PER_CYCLE`. + Only detected species the repo doesn't already have are generated (typically a handful). + Generation is paced by `GENERATE_SLEEP` (default 6s) to stay under the free tier; that gap is + the only throughput control, since the limit here is the API's request rate. +- **A pose the model declines is left alone for a day** rather than re-attempted every + refresh, so a stubborn species can't quietly drain quota. See + [Free illustrations](#free-illustrations) for how gaps are remembered. - **Persist the art** with the named volume above so container upgrades don't re-spend those API calls. The manifest is rebuilt from the volume at startup. - **The generator is bundled in every image** — vendored at build time from the [saezuri-illustrations](https://github.com/vrwrts/saezuri-illustrations) pipeline at a pinned version (numpy/scipy cutout, no ML model, so the `nginx:alpine` image stays modest). `GEMINI_API_KEY` unset simply means the worker never generates; the container is otherwise identical. +- **A species that keeps coming out wrong** needs a better prompt, not more attempts — see + [Species notes](#species-notes). - **Licensing.** Generating art locally for your own display is personal use. The style derives from the CC-BY-NC-SA lineage (see below) — confirm the obligations before publishing generated images. +## Species notes + +Sometimes a species comes out wrong no matter how many times you regenerate it — the model's prior +is simply off, and re-rolling the dice won't fix it. A *note* is a sentence or two appended to that +species' prompt only: + +```json +{ + "Turdus merula": "Solid glossy black, no pale markings; bill and eye-ring bright orange-yellow.", + "parus-major": "Black crown and throat stripe, bright white cheeks, yellow underparts." +} +``` + +Save it as `_species-notes.json` beside the art (inside the persisted volume, so it survives +upgrades), or point `SPECIES_NOTES` anywhere you like. A key may be either the scientific name or +its slug — the slug is what you see in the illustration filenames. Keys beginning with `_` are +comments. + +How it behaves: + +- **Edits apply on their own.** When you change a species' note, its art is re-rendered on the next + cycle; you don't need to delete anything or restart the container. +- **It only affects art Saezuri generated.** A cutout downloaded from the illustrations repo is + left alone, because that repo is the state of the art and everyone benefits from it being right. + If a note fixes a species the repo gets wrong, [contribute it + upstream](https://github.com/vrwrts/saezuri-illustrations) rather than keeping the fix local — the + pipeline ships its own `species-notes.json` that yours is layered over, and that is the file to + send a PR to. +- **It needs `GEMINI_API_KEY`.** A note is an instruction to the generator; with no key there is + nothing to instruct. + ## Free illustrations You don't have to pay for generation to get real art. **On by default**, the moment BirdNET-Go @@ -233,7 +274,7 @@ How it behaves: - **Gaps it can't fill are remembered, not retried forever.** A cutout the repo doesn't have (or that generation declined) is logged once and left alone for a while — a week for a repo miss, a day for a generation miss — instead of being re-requested on every refresh. The record lives in - `_misses.json` in the illustrations volume; delete it to retry everything immediately. + `_art-state.json` in the illustrations volume; delete it to retry everything immediately. - **Self-healing.** Delete a cutout from the volume and the service notices, re-downloads it (or regenerates it), and rebuilds the manifest — which is also how you replace art you don't like. This works for a single pose of a pair, and for a bird that hasn't been heard in weeks: every diff --git a/addon/DOCS.md b/addon/DOCS.md index f2952b6..7a84ec1 100644 --- a/addon/DOCS.md +++ b/addon/DOCS.md @@ -72,12 +72,36 @@ generic silhouette, still labelled and still sized by their real count. | **Illustrations branch** | `main` | Branch or tag to download from. | | **Illustrations base URL** | derived | Overrides the two above with a direct URL. | | **Gemini API key** | unset | Optional. Set it to *also* generate art, in the same style, for species nobody has contributed yet. | -| **Generated illustrations per cycle** | `4` | How many to generate at a time. | -| **Pause between generations** | `6` | Seconds between generated illustrations. | +| **Pause between generations** | `6` | Seconds between generated illustrations. Lower on a paid tier, raise if rate-limited. | +| **Species notes** | none | Per-bird prompt corrections, see below. | Generation costs money at Google's rates and is entirely optional. Everything works without a key. +Illustrations are generated one pose at a time, perched first. A bird stops being a +silhouette as soon as its perched illustration lands, so it appears without waiting +for the flight one. + +### Species notes + +Some birds come out wrong no matter how often they are regenerated — the model's idea +of them is simply off, and trying again won't help. A note is a short description +added to that bird's prompt only. One entry per bird: + +``` +Turdus merula|Solid glossy black, orange-yellow bill and eye-ring. +Parus major|Black crown and throat stripe, white cheeks, yellow underparts. +``` + +Use the scientific name before the pipe (the slug, like `turdus-merula`, also works). +Change a note and that bird is redrawn on the next cycle — nothing to restart. + +Notes only affect illustrations this app generates itself. One downloaded from the +illustrations repository is left as it is, because that repository is the shared set +everyone draws from. If a note fixes a bird the repository gets wrong, please +[contribute it there](https://github.com/vrwrts/saezuri-illustrations) so every +installation benefits. + ### Reference recordings When a species is heard, Saezuri looks up a freely-licensed recording of its call and diff --git a/addon/config.yaml b/addon/config.yaml index 0917a91..16987c6 100644 --- a/addon/config.yaml +++ b/addon/config.yaml @@ -21,7 +21,6 @@ ports: ports_description: "8080/tcp": Direct web access, only needed for an e-ink panel fetching /24h.png options: - generate_max_per_cycle: 4 calls_max_per_cycle: 4 frame_width: 800 frame_height: 480 @@ -40,8 +39,16 @@ schema: illustrations_ref: str? illustrations_base_url: str? gemini_api_key: password? - generate_max_per_cycle: int(1,64) generate_sleep: int(0,600) + # "Scientific name|note" per entry, e.g. + # Turdus merula|Solid glossy black, orange-yellow bill and eye-ring. + # run.sh materialises these into the notes file the generator reads. + species_notes: + - str? + # Ignored since 0.15: art is acquired one pose at a time, so there is no batch + # to cap. Kept in the schema only so an existing configuration still validates; + # remove it from your options at your convenience. + generate_max_per_cycle: int(1,64)? call_providers: str? calls_max_per_cycle: int(1,64) frame_width: int(100,4000) diff --git a/addon/run.sh b/addon/run.sh index c1bd535..9b3dcfa 100755 --- a/addon/run.sh +++ b/addon/run.sh @@ -40,6 +40,35 @@ export_opt() { fi } +# The generator reads prompt addenda from a JSON file, but a Home Assistant user has +# no way to edit a file inside the app's own /data. So the option is a list of +# "key|note" strings — editable in the app's configuration UI, and the same +# pipe-separated convention the pipeline already uses for "scientific|common" — and +# this turns it into the file. Split on the FIRST pipe only, since a note may +# contain one. +export_species_notes() { + opt_present species_notes || return 0 + _notes_file=/data/species-notes.json + jq '[.species_notes[]? + | select(type == "string") + | (. | index("|")) as $i + | select($i != null) + | { key: (.[:$i] | sub("^\\s+"; "") | sub("\\s+$"; "")), + value: (.[$i+1:] | sub("^\\s+"; "") | sub("\\s+$"; "")) } + | select(.key != "" and .value != "") + ] | from_entries' "$OPTIONS" > "$_notes_file" + _count=$(jq 'length' "$_notes_file") + if [ "$_count" -gt 0 ]; then + export SPECIES_NOTES="$_notes_file" + log "species_notes: ${_count} note(s) -> ${_notes_file}" + else + # Nothing usable: leave SPECIES_NOTES unset so the app falls back to its own + # default path, rather than pointing it at an empty file. + rm -f "$_notes_file" + log "species_notes: no usable entries (expected \"Scientific name|note\")" + fi +} + # FRAME_SHADOW is read as "anything but 0", so a YAML bool has to become 1 or 0; # exporting the string "false" would silently enable the shadow. export_bool_opt() { @@ -60,8 +89,8 @@ export_opt ILLUSTRATIONS_REPO illustrations_repo export_opt ILLUSTRATIONS_REF illustrations_ref export_opt ILLUSTRATIONS_BASE_URL illustrations_base_url export_opt GEMINI_API_KEY gemini_api_key -export_opt GENERATE_MAX_PER_CYCLE generate_max_per_cycle export_opt GENERATE_SLEEP generate_sleep +export_species_notes export_opt CALL_PROVIDERS call_providers export_opt CALLS_MAX_PER_CYCLE calls_max_per_cycle export_opt FRAME_WIDTH frame_width @@ -174,7 +203,7 @@ log "BIRDNETGO_URL=${BIRDNETGO_URL}" log "BIRDNETGO_TOKEN=$(redacted "${BIRDNETGO_TOKEN:-}")" log "GEMINI_API_KEY=$(redacted "${GEMINI_API_KEY:-}")" for _name in ILLUSTRATIONS_REPO ILLUSTRATIONS_REF ILLUSTRATIONS_BASE_URL \ - GENERATE_MAX_PER_CYCLE GENERATE_SLEEP CALL_PROVIDERS CALLS_MAX_PER_CYCLE \ + GENERATE_SLEEP SPECIES_NOTES CALL_PROVIDERS CALLS_MAX_PER_CYCLE \ FRAME_WIDTH FRAME_HEIGHT FRAME_BG FRAME_SHADOW FRAME_WINDOWS \ SPECIES_DICT_LOCALES PUBLISH_DEBOUNCE_MS AGING_INTERVAL_MS \ SUMMARY_INTERVAL_MS FRAME_HTML_DIR CACHE_DIR; do diff --git a/addon/translations/en.yaml b/addon/translations/en.yaml index bab9d52..0ada609 100644 --- a/addon/translations/en.yaml +++ b/addon/translations/en.yaml @@ -31,12 +31,23 @@ configuration: description: >- Optional. Set it to also generate art for species nobody has contributed an illustration for yet. - generate_max_per_cycle: - name: Generated illustrations per cycle - description: How many illustrations to generate at a time. generate_sleep: name: Pause between generations - description: Seconds to wait between generated illustrations. + description: >- + Seconds to wait between generated illustrations. Lower it if you are on a + paid Gemini tier, raise it if you get rate-limited. + species_notes: + name: Species notes + description: >- + For birds that keep coming out wrong. One entry per bird, written as + "Scientific name|what it should look like" — for example "Turdus + merula|Solid glossy black, orange-yellow bill and eye-ring". Only affects + illustrations this app generates itself; downloaded ones are left alone. + generate_max_per_cycle: + name: Generated illustrations per cycle (unused) + description: >- + No longer used — illustrations are now generated one at a time. Safe to + remove from your configuration. call_providers: name: Recording archives description: >- diff --git a/docker-compose.yml b/docker-compose.yml index 6a29f60..859a69f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -22,10 +22,13 @@ services: # display-only: the refresh service still publishes, it just skips # generation, so new species stay hidden until art exists. GEMINI_API_KEY: ${GEMINI_API_KEY:-} - # Throttle between image-API calls (unset => pregen's 6s free-tier default). + # Seconds between image-API calls (default 6, the Gemini free-tier pace). The + # throughput knob: art is acquired one pose at a time and this gap is what keeps + # us under the request-rate limit. GENERATE_SLEEP: ${GENERATE_SLEEP:-} - # Cap species generated per pipeline run. - GENERATE_MAX_PER_CYCLE: ${GENERATE_MAX_PER_CYCLE:-4} + # Prompt addenda for species that keep coming out wrong. Defaults to + # _species-notes.json beside the art, i.e. inside the persisted volume. + SPECIES_NOTES: ${SPECIES_NOTES:-} # --- Free pre-made illustrations (see .env.example) --- # Per detected species, the refresh service downloads a ready-made cutout from # the saezuri-illustrations repo (via jsDelivr) — no Gemini key needed. Defaults diff --git a/src/server/generate.test.ts b/src/server/generate.test.ts index c4814fc..64b2bb2 100644 --- a/src/server/generate.test.ts +++ b/src/server/generate.test.ts @@ -4,12 +4,15 @@ import { join } from 'node:path' import { afterEach, beforeEach, describe, expect, it, type Mock, vi } from 'vitest' // Mock the pipeline subprocess: record argv, resolve immediately (code 0). Lets us -// assert whether --rebuild (download-only) or --generate (Gemini) was invoked without -// needing Python. +// assert what was asked of the pipeline without needing Python. A hook lets a test +// simulate the render actually landing (or failing to). const spawnCalls: string[][] = [] +const spawnAtMs: number[] = [] +let onSpawn: ((args: string[]) => void | Promise) | undefined vi.mock('node:child_process', () => ({ spawn: (_bin: string, args: string[]) => { spawnCalls.push(args) + spawnAtMs.push(Date.now()) const handlers: Record void> = {} const child = { on(ev: string, cb: (arg?: unknown) => void) { @@ -17,7 +20,7 @@ vi.mock('node:child_process', () => ({ return child }, } - queueMicrotask(() => handlers.close?.(0)) + void Promise.resolve(onSpawn?.(args)).then(() => handlers.close?.(0)) return child }, })) @@ -31,55 +34,99 @@ const PNG = Buffer.from( ) const BASE = 'http://fixtures.test' +const STATE_FILE = '_art-state.json' let assetsDir: string -let onGenerated: Mock<() => Promise> - -// Pose stems requested since the last stubFetch, in order. -const fetched: string[] = [] +let notesPath: string +let onGenerated: Mock<() => void> +let fetchMock: Mock // fetch stub: 200 for exactly the listed pose filenames (e.g. 'turdus-merula' for the // perched pose, 'turdus-merula-2' for flight), 404 for everything else. function stubFetch(presentPoses: string[]) { - fetched.length = 0 - vi.stubGlobal( - 'fetch', - vi.fn(async (url: string) => { - const m = url.match(/\/illustrations\/(.+)\.png$/) - const name = m?.[1] - if (name) fetched.push(name) - if (name && presentPoses.includes(name)) { - return { ok: true, arrayBuffer: async () => PNG.buffer.slice(0) } as unknown as Response - } - return { ok: false, status: 404 } as Response - }), - ) + fetchMock = vi.fn(async (url: string) => { + const name = url.match(/\/illustrations\/(.+)\.png$/)?.[1] + if (name && presentPoses.includes(name)) { + // Slice by offset+length, not `.buffer`: a Buffer from a small base64 string + // sits in Node's shared pool, so `.buffer` is the whole pool and the bytes + // written would not be a valid PNG. + const body = PNG.buffer.slice(PNG.byteOffset, PNG.byteOffset + PNG.byteLength) + return { ok: true, arrayBuffer: async () => body } as unknown as Response + } + return { ok: false, status: 404 } as Response + }) + vi.stubGlobal('fetch', fetchMock) } -function makeGen(opts: { enabled: boolean; downloadBaseUrl: string }) { - onGenerated = vi.fn<() => Promise>(async () => {}) +/** Names the download path asked the repo for, in call order. */ +const fetched = () => + fetchMock.mock.calls.map(([url]) => (url as string).match(/\/illustrations\/(.+)\.png$/)?.[1]) + +/** argv of every `--generate` invocation, in call order. */ +const generateCalls = () => spawnCalls.filter((a) => a.includes('--generate')) + +function makeGen(opts: { + enabled: boolean + downloadBaseUrl: string + generateGapMs?: number + notesPaths?: string[] + isDescribed?: (key: string) => boolean +}) { + onGenerated = vi.fn<() => void>(() => {}) return new Generator({ pythonBin: 'python3', workerScript: '/fake/worker.py', assetsDir, cacheDir: join(assetsDir, '.cache'), - maxPerCycle: 4, enabled: opts.enabled, downloadBaseUrl: opts.downloadBaseUrl, + notesPaths: opts.notesPaths ?? [notesPath], + generateGapMs: opts.generateGapMs ?? 0, onGenerated, + isDescribed: opts.isDescribed, }) } const idle = (g: Generator) => vi.waitFor(() => { - // biome-ignore lint/suspicious/noExplicitAny: reach into private drain state for the test + // biome-ignore lint/suspicious/noExplicitAny: reach into private lane state for the test const any = g as any - expect(any.busy).toBe(false) - expect(any.queued.size).toBe(0) + expect(any.dlBusy).toBe(false) + expect(any.genBusy).toBe(false) + expect(any.dlQueue.size).toBe(0) + expect(any.genQueue.size).toBe(0) }) +/** Make the pipeline "succeed" by writing the pose file it was asked for. */ +function landRenders() { + onSpawn = async (args) => { + const i = args.indexOf('--generate') + if (i === -1) return + const sci = args[i + 1]?.split('|')[0] ?? '' + const slug = sci.toLowerCase().replace(/[^a-z0-9]+/g, '-') + const suffix = args[args.indexOf('--poses') + 1] === '2' ? '-2' : '' + await writeFile(join(assetsDir, `${slug}${suffix}.png`), PNG) + } +} + +interface PoseStateJson { + downloadMissAt?: number + generateMissAt?: number + source?: string + noteVer?: string +} + +const readState = async () => + JSON.parse(await readFile(join(assetsDir, STATE_FILE), 'utf8')) as Record + +const writeState = (state: Record) => + writeFile(join(assetsDir, STATE_FILE), JSON.stringify(state)) + beforeEach(async () => { spawnCalls.length = 0 + spawnAtMs.length = 0 + onSpawn = undefined assetsDir = await mkdtemp(join(tmpdir(), 'saezuri-gen-')) + notesPath = join(assetsDir, '_species-notes.json') }) afterEach(async () => { vi.unstubAllGlobals() @@ -88,7 +135,7 @@ afterEach(async () => { describe('Generator art acquisition', () => { it('downloads a complete pair and rebuilds the manifest (no Gemini key)', async () => { - stubFetch(['turdus-merula', 'turdus-merula-2']) // both poses available in the repo + stubFetch(['turdus-merula', 'turdus-merula-2']) const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) @@ -96,48 +143,50 @@ describe('Generator art acquisition', () => { const files = await readdir(assetsDir) expect(files).toContain('turdus-merula.png') expect(files).toContain('turdus-merula-2.png') - // Complete via download → manifest rebuilt, never Gemini generation. + // Downloaded art still needs its silhouette built, but never Gemini generation. expect(spawnCalls.some((a) => a.includes('--rebuild'))).toBe(true) - expect(spawnCalls.some((a) => a.includes('--generate'))).toBe(false) - expect(onGenerated).toHaveBeenCalledTimes(1) + expect(generateCalls()).toHaveLength(0) + expect(onGenerated).toHaveBeenCalled() }) - it('generates only the missing pose when the repo has a partial pair (keyed)', async () => { - stubFetch(['turdus-merula']) // repo has the perched pose only; flight 404s + it('generates only the pose the repo is missing', async () => { + stubFetch(['turdus-merula']) // perched only; flight 404s + landRenders() const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - // Perched downloaded for free, but the pair is incomplete → Gemini fills the gap - // (pregen skips the pose already on disk). expect(await readdir(assetsDir)).toContain('turdus-merula.png') - const gen = spawnCalls.find((a) => a.includes('--generate')) - expect(gen).toBeDefined() + // Exactly one generation, for the flight pose only. + expect(generateCalls()).toHaveLength(1) + const gen = generateCalls()[0] expect(gen).toContain('Turdus merula|Eurasian Blackbird') - expect(onGenerated).toHaveBeenCalledTimes(1) + expect(gen.slice(gen.indexOf('--poses'))).toContain('2') }) - it('is a no-op for an absent species with no Gemini key', async () => { - stubFetch([]) // everything 404s - const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) + it('asks the pipeline for one pose per invocation, perched first', async () => { + stubFetch([]) // repo has nothing, so both poses are generated + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(await readdir(assetsDir)).not.toContain('turdus-merula.png') - expect(spawnCalls).toHaveLength(0) // nothing to rebuild, nothing to generate - expect(onGenerated).not.toHaveBeenCalled() + const poses = generateCalls().map((a) => a[a.indexOf('--poses') + 1]) + expect(poses).toEqual(['1', '2']) + // One species per call, never a batch of several. + for (const call of generateCalls()) { + expect(call.filter((a) => a.includes('|'))).toHaveLength(1) + } }) - it('falls back to Gemini generation for an absent species when keyed', async () => { - stubFetch([]) // repo has nothing - const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + it('is a no-op for an absent species with no Gemini key', async () => { + stubFetch([]) + const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - const gen = spawnCalls.find((a) => a.includes('--generate')) - expect(gen).toBeDefined() - expect(gen).toContain('Turdus merula|Eurasian Blackbird') - expect(onGenerated).toHaveBeenCalledTimes(1) + expect(await readdir(assetsDir)).not.toContain('turdus-merula.png') + expect(generateCalls()).toHaveLength(0) }) it('re-downloads only the pose deleted from an existing pair', async () => { @@ -148,7 +197,7 @@ describe('Generator art acquisition', () => { g.enqueueRepairs([{ slug: 'turdus-merula', sci: 'Turdus merula', com: 'Eurasian Blackbird' }]) await idle(g) - expect(fetched).toEqual(['turdus-merula-2']) + expect(fetched()).toEqual(['turdus-merula-2']) expect(await readdir(assetsDir)).toContain('turdus-merula-2.png') expect(spawnCalls.some((a) => a.includes('--generate'))).toBe(false) }) @@ -159,67 +208,385 @@ describe('Generator art acquisition', () => { g.enqueueRepairs([{ slug: 'turdus-merula' }]) await idle(g) - expect(fetched).toEqual(['turdus-merula', 'turdus-merula-2']) + expect(fetched()).toEqual(['turdus-merula', 'turdus-merula-2']) expect(spawnCalls.some((a) => a.includes('--generate'))).toBe(false) }) + it('no-ops enqueue when both sources are disabled', async () => { + stubFetch(['turdus-merula']) + const g = makeGen({ enabled: false, downloadBaseUrl: '' }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + expect(spawnCalls).toHaveLength(0) + expect(fetchMock).not.toHaveBeenCalled() + expect(onGenerated).not.toHaveBeenCalled() + }) +}) + +describe('Generator miss backoff', () => { it('remembers a repo miss and stops re-requesting it, across restarts', async () => { - stubFetch([]) + stubFetch([]) // repo has nothing const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(fetched).toEqual(['turdus-merula', 'turdus-merula-2']) + expect(fetched()).toEqual(['turdus-merula', 'turdus-merula-2']) - const misses = JSON.parse(await readFile(join(assetsDir, '_misses.json'), 'utf8')) - expect(Object.keys(misses.download).sort()).toEqual(['turdus-merula', 'turdus-merula-2']) + // Same process, next publish: enqueueMissingAssets re-asks and must cost nothing. + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect(fetched()).toEqual(['turdus-merula', 'turdus-merula-2']) - fetched.length = 0 + const state = await readState() + expect(typeof state['turdus-merula'].downloadMissAt).toBe('number') + expect(typeof state['turdus-merula-2'].downloadMissAt).toBe('number') + + // A fresh Generator over the same directory is a restart. + fetchMock.mockClear() const restarted = makeGen({ enabled: false, downloadBaseUrl: BASE }) restarted.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(restarted) - expect(fetched).toEqual([]) + expect(fetchMock).not.toHaveBeenCalled() }) it('retries a repo miss once its backoff has expired', async () => { const stale = Date.now() - 8 * 24 * 60 * 60 * 1000 // older than the 7d download TTL - await writeFile( - join(assetsDir, '_misses.json'), - JSON.stringify({ download: { 'turdus-merula': stale, 'turdus-merula-2': stale } }), - ) + await writeState({ + 'turdus-merula': { downloadMissAt: stale }, + 'turdus-merula-2': { downloadMissAt: stale }, + }) stubFetch(['turdus-merula', 'turdus-merula-2']) const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(fetched).toEqual(['turdus-merula', 'turdus-merula-2']) + expect(fetched()).toEqual(['turdus-merula', 'turdus-merula-2']) expect(await readdir(assetsDir)).toContain('turdus-merula-2.png') - const misses = JSON.parse(await readFile(join(assetsDir, '_misses.json'), 'utf8')) - expect(misses.download).toEqual({}) // both poses landed + // Both poses landed, so neither source's verdict stands any more. + const state = await readState() + expect(state['turdus-merula'].downloadMissAt).toBeUndefined() + expect(state['turdus-merula-2'].downloadMissAt).toBeUndefined() }) - it('backs off a species the model declines rather than regenerating every sweep', async () => { + it('backs off a pose the model declines rather than regenerating every sweep', async () => { stubFetch([]) // repo has nothing; the mocked worker exits 0 without writing anything const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(spawnCalls.filter((a) => a.includes('--generate'))).toHaveLength(1) + expect(generateCalls()).toHaveLength(2) // one per pose g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(spawnCalls.filter((a) => a.includes('--generate'))).toHaveLength(1) + expect(generateCalls()).toHaveLength(2) // backed off, not retried - const misses = JSON.parse(await readFile(join(assetsDir, '_misses.json'), 'utf8')) - expect(Object.keys(misses.generate).sort()).toEqual(['turdus-merula', 'turdus-merula-2']) + const state = await readState() + expect(typeof state['turdus-merula'].generateMissAt).toBe('number') + expect(typeof state['turdus-merula-2'].generateMissAt).toBe('number') }) - it('no-ops enqueue when both sources are disabled', async () => { + it('carries the legacy _misses.json backoff over on upgrade', async () => { + // Deployments upgrading from the previous format must not re-probe the repo for + // every species it is already known to lack. + const recent = Date.now() - 60_000 + await writeFile( + join(assetsDir, '_misses.json'), + JSON.stringify({ + download: { 'turdus-merula': recent, 'turdus-merula-2': recent }, + generate: { 'parus-major': recent }, + }), + ) + stubFetch([]) + const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + expect(fetchMock).not.toHaveBeenCalled() + expect(typeof (await readState())['parus-major'].generateMissAt).toBe('number') + }) + + it('does not record a miss when the probe fails transiently', async () => { + fetchMock = vi.fn(async () => { + throw new Error('ECONNRESET') + }) + vi.stubGlobal('fetch', fetchMock) + const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + // Nothing recorded, so the next cycle tries again rather than writing it off. + const state = await readState().catch(() => ({}) as Awaited>) + expect(state['turdus-merula']?.downloadMissAt).toBeUndefined() + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect(fetchMock.mock.calls.length).toBeGreaterThan(2) + }) + + it('treats a malformed state file as empty rather than skipping every pose', async () => { + await writeFile(join(assetsDir, STATE_FILE), 'not json at all') stubFetch(['turdus-merula']) - const g = makeGen({ enabled: false, downloadBaseUrl: '' }) + const g = makeGen({ enabled: false, downloadBaseUrl: BASE }) g.enqueue('Turdus merula', 'Eurasian Blackbird') await idle(g) - expect(spawnCalls).toHaveLength(0) - expect(await readdir(assetsDir)).not.toContain('turdus-merula.png') - expect(onGenerated).not.toHaveBeenCalled() + expect(fetched()).toContain('turdus-merula') + expect(await readdir(assetsDir)).toContain('turdus-merula.png') + }) +}) + +describe('Generator lanes', () => { + it('acquires repo art while the generate lane is busy', async () => { + stubFetch(['parus-major', 'parus-major-2']) + // Block the generate lane until we release it. + let release: () => void = () => {} + const blocked = new Promise((r) => { + release = r + }) + onSpawn = async (args) => { + if (args.includes('--generate')) await blocked + } + + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') // repo has nothing → generate lane + g.enqueue('Parus major', 'Great Tit') // repo has both → download lane + + // The repo-supplied species lands without waiting for the render. + await vi.waitFor(async () => { + const files = await readdir(assetsDir) + expect(files).toContain('parus-major.png') + expect(files).toContain('parus-major-2.png') + }) + expect(generateCalls().length).toBeGreaterThan(0) // render genuinely still in flight + + release() + await idle(g) + }) + + // Real timers, not fake ones: the lanes await real fs reads and writes, which fake + // timers do not drive, so advancing them races the state file instead of the gap. + it('paces successive generations by the configured gap', async () => { + const GAP_MS = 120 + stubFetch([]) + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE, generateGapMs: GAP_MS }) + const startedMs = Date.now() + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + expect(generateCalls()).toHaveLength(2) + // Each call waits out a gap: the first after enqueue, the second after the first. + expect(spawnAtMs[0] - startedMs).toBeGreaterThanOrEqual(GAP_MS) + expect(spawnAtMs[1] - spawnAtMs[0]).toBeGreaterThanOrEqual(GAP_MS) + }) + + it('does not pace when the gap is zero', async () => { + stubFetch([]) + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE, generateGapMs: 0 }) + const startedMs = Date.now() + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + expect(generateCalls()).toHaveLength(2) + // Generous on purpose: this only has to distinguish "no gap" from the 6s-per-pose + // production default, so it must not fail merely because the machine is loaded. + expect(Date.now() - startedMs).toBeLessThan(2_000) + }) +}) + +describe('Generator species notes', () => { + const noteFile = (notes: Record) => writeFile(notesPath, JSON.stringify(notes)) + + it('re-renders generated art when its note changes', async () => { + stubFetch([]) // nothing in the repo → both poses generated locally + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + const before = generateCalls().length + expect((await readState())['turdus-merula'].source).toBe('generated') + + await noteFile({ 'Turdus merula': 'darker bill, rounder head' }) + const g2 = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g2.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g2) + + const forced = generateCalls() + .slice(before) + .filter((a) => a.includes('--force')) + expect(forced.length).toBeGreaterThan(0) + expect(forced[0]).toContain('Turdus merula|Eurasian Blackbird') + }) + + it('leaves repo art alone when a note is added for it', async () => { + stubFetch(['turdus-merula', 'turdus-merula-2']) + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect((await readState())['turdus-merula'].source).toBe('repo') + const before = generateCalls().length + + await noteFile({ 'turdus-merula': 'darker bill' }) + const g2 = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g2.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g2) + + // The repo is authoritative: a note must not silently replace its art. + expect(generateCalls()).toHaveLength(before) + }) + + it('does not re-render when the note is unchanged', async () => { + await noteFile({ 'Turdus merula': 'darker bill' }) + stubFetch([]) + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + const before = generateCalls().length + + const g2 = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g2.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g2) + expect(generateCalls()).toHaveLength(before) + }) + + it('retries a forced re-render that produced nothing', async () => { + stubFetch([]) + landRenders() + const g = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + // A note arrives, and the forced render fails, leaving the old file untouched. + await writeFile(notesPath, JSON.stringify({ 'Turdus merula': 'darker bill' })) + onSpawn = async (args) => { + if (args.includes('--force')) return // the render fails; the old file survives + } + const g2 = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g2.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g2) + + // The failure must not be recorded as success under the new note, or it would + // never be attempted again. + expect((await readState())['turdus-merula'].noteVer).toBeUndefined() + + // So a later cycle tries again. + const before = generateCalls().length + const g3 = makeGen({ enabled: true, downloadBaseUrl: BASE }) + g3.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g3) + expect(generateCalls().length).toBeGreaterThan(before) + }) + + it('passes every notes layer to the pipeline', async () => { + stubFetch([]) + landRenders() + const bundled = join(assetsDir, 'bundled-notes.json') + await writeFile(bundled, '{}') + const g = makeGen({ + enabled: true, + downloadBaseUrl: BASE, + notesPaths: [bundled, notesPath], + }) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + + const gen = generateCalls()[0] + expect(gen.filter((a) => a === '--notes')).toHaveLength(2) + expect(gen).toContain(bundled) + expect(gen).toContain(notesPath) + }) +}) + +describe('Generator corrupt-art diagnosis', () => { + const logged: string[] = [] + beforeEach(() => { + logged.length = 0 + vi.spyOn(console, 'log').mockImplementation((m: unknown) => { + logged.push(String(m)) + }) + }) + afterEach(() => { + vi.restoreAllMocks() + }) + + const complaints = () => logged.filter((l) => l.includes('layout manifest keeps rejecting')) + + // Only Date is faked: faking timers wholesale breaks the real fs the lanes await. + const advance = (ms: number) => vi.setSystemTime(new Date(Date.now() + ms)) + + it('reports art the manifest keeps rejecting, once the manifest has had time', async () => { + vi.useFakeTimers({ toFake: ['Date'] }) + try { + stubFetch(['turdus-merula', 'turdus-merula-2']) + // The manifest never describes it — what a corrupt PNG looks like from here. + const g = makeGen({ enabled: false, downloadBaseUrl: BASE, isDescribed: () => false }) + + // Cycle 1 downloads the art, so it sights nothing yet. + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect(complaints()).toHaveLength(0) + + // A burst of cycles right after it lands must stay quiet: the manifest is simply + // behind, which is indistinguishable from corruption at this point. + for (let i = 0; i < 3; i++) { + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + } + expect(complaints()).toHaveLength(0) + + // Still undescribed a couple of minutes later — now it is a real problem. + advance(150_000) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect(complaints().length).toBeGreaterThan(0) + + // Reported once, not on every cycle from here on. + const after = complaints().length + advance(150_000) + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + expect(complaints()).toHaveLength(after) + } finally { + vi.useRealTimers() + } + }) + + it('stays quiet for art the manifest does describe', async () => { + stubFetch(['turdus-merula', 'turdus-merula-2']) + const g = makeGen({ enabled: false, downloadBaseUrl: BASE, isDescribed: () => true }) + for (let i = 0; i < 3; i++) { + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + } + expect(complaints()).toHaveLength(0) + }) + + it('stays quiet while the manifest is merely catching up', async () => { + stubFetch(['turdus-merula', 'turdus-merula-2']) + // Absent on the first look, described by the time it is asked again — the normal + // lag between art landing and the next publish. + let described = false + const g = makeGen({ + enabled: false, + downloadBaseUrl: BASE, + isDescribed: () => described, + }) + vi.useFakeTimers({ toFake: ['Date'] }) + try { + g.enqueue('Turdus merula', 'Eurasian Blackbird') // downloads + await idle(g) + g.enqueue('Turdus merula', 'Eurasian Blackbird') // first sighting, undescribed + await idle(g) + described = true + // Long enough that a time-gated warning would have fired had it stayed absent. + advance(150_000) + for (let i = 0; i < 3; i++) { + g.enqueue('Turdus merula', 'Eurasian Blackbird') + await idle(g) + } + expect(complaints()).toHaveLength(0) + } finally { + vi.useRealTimers() + } }) }) diff --git a/src/server/generate.ts b/src/server/generate.ts index 91e8353..c0d3477 100644 --- a/src/server/generate.ts +++ b/src/server/generate.ts @@ -1,30 +1,40 @@ import { spawn } from 'node:child_process' -import { existsSync } from 'node:fs' +import { existsSync, statSync } from 'node:fs' import { mkdir, readFile, rename, writeFile } from 'node:fs/promises' import { join } from 'node:path' import { FLIGHT_SUFFIX } from '../domain/asset.ts' import { slugify } from '../domain/slug.ts' +import { loadNotes, noteFor, noteVersion, type SpeciesNotes } from './notes.ts' import type { ArtRepair } from './reconcile.ts' -// Art queue: turns "this species was heard but has no cutout" into art, via two -// sources that compose. For each heard species we FIRST try to download a ready-made -// cutout from the saezuri-illustrations repo (free, no key); only species the repo -// doesn't have fall back to on-demand Gemini generation (worker.py --generate), and -// only when a key is set. Deduped by slug, serialized, capped per batch. After a batch -// the manifest is rebuilt (so downloaded art is described even with no key) and -// onGenerated republishes. A no-op when neither source is available. +// Art queue: turns "this species has no cutout" into art. reconcile.ts decides *what* +// is missing; this decides how to get it. +// +// The unit of work is a POSE, not a species. A species with only its perched cutout is +// already fully renderable (see resolveArt), so the perched render is what lifts it off +// the fallback silhouette while the flight render only changes the 15% of appearances +// that roll for it — batching the pair would make the valuable render wait behind the +// cheap one. +// +// Two sources compose, in a fixed precedence: a ready-made cutout from the +// saezuri-illustrations repo (free, no key) first, and on-demand Gemini generation only +// for what the repo lacks. The repo is meant to be the state of the art, so it always +// wins; species notes tune the generation fallback and never override the repo. +// +// They run as independent LANES so a cheap repo download never queues behind an +// expensive render. The generate lane is serial and paces its own calls, because the +// constraint there is the image API's rate limit, not worker count. -type ArtRequest = ArtRepair - -// '' = base (perched) pose, FLIGHT_SUFFIX = flight pose. -const POSE_SUFFIXES = ['', FLIGHT_SUFFIX] +/** '' = base (perched) pose, FLIGHT_SUFFIX = flight pose. Perched first: it is the + * pose that makes a species visible, so it should never wait behind flight. */ +const POSE_SUFFIXES = ['', FLIGHT_SUFFIX] as const const MISS_SOURCES = ['download', 'generate'] as const type MissSource = (typeof MISS_SOURCES)[number] -// How long a source is left alone after it failed to supply a pose. Without this, -// every publish re-requests a 404 for every never-illustrated species — which is most -// of them — and the sweep now spans every species we've ever heard. +// How long a source is left alone after it failed to supply a pose. Without this, every +// publish re-requests a 404 for every never-illustrated species — which is most of them +// — and the sweep spans every species we've ever heard. // download: the repo does gain art, just rarely; same TTL as the call archives. // generate: shorter, because a pose the model declined is worth retrying sooner than // one the repo simply lacks — but not every publish, which spends real quota. @@ -33,235 +43,493 @@ const MISS_TTL_MS: Record = { generate: 24 * 60 * 60 * 1000, } -/** Leading underscore keeps it out of the manifest scan, matching the - * `_fallback.png` convention in the illustrations dir. */ -const MISSES_FILE = '_misses.json' +/** Concurrent repo downloads. Network-only and cheap, so a small fixed fan-out; there + * is no rate limit to respect here, unlike the generate lane. */ +const DOWNLOAD_CONCURRENCY = 4 + +/** Leading underscore keeps these out of the manifest's `*.png` scan, matching the + * `_fallback.png` convention. The legacy file held only the per-source misses; this + * one also carries provenance and the note each pose was drawn under, so it is read + * once to seed the new file rather than being kept in sync. */ +const STATE_FILE = '_art-state.json' +const LEGACY_MISSES_FILE = '_misses.json' + +const TAG = 'saezuri-generate' +const log = (msg: string) => console.log(`${TAG}: ${msg}`) +const logErr = (e: unknown) => + console.error(`${TAG}: ${e instanceof Error ? e.message : String(e)}`) + +const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)) + +/** One pose of one species: the unit both lanes queue and dedupe on. `sci` is absent + * when the slug came off disk and we could not name its bird — downloadable by slug, + * but never generated, because we won't spend a Gemini call on a guessed name. */ +interface PoseRequest extends ArtRepair { + suffix: string + /** Re-render even though the file exists — set only when a note changed. */ + force?: boolean +} + +/** What we know about a pose we have already acted on. A cache, never a source of + * truth: disk decides whether art exists, and a missing or unreadable entry must + * degrade to "ask again", never to "skip this pose forever". */ +interface PoseState { + /** When the repo last returned a clean 404 for this pose. */ + downloadMissAt?: number + /** When a generation run last finished without producing this pose. */ + generateMissAt?: number + /** Where the file on disk came from. Gates note-driven regeneration: repo art is + * never replaced by a local render. */ + source?: 'repo' | 'generated' + /** The note this pose was generated under, so a changed note can re-render it. */ + noteVer?: string +} + +const MISS_FIELD: Record = { + download: 'downloadMissAt', + generate: 'generateMissAt', +} export interface GeneratorOptions { pythonBin: string workerScript: string assetsDir: string cacheDir: string - /** Max species per pipeline invocation (0 = no cap). */ - maxPerCycle: number /** GEMINI_API_KEY present — otherwise on-demand generation is skipped. */ enabled: boolean /** Base URL for downloading pre-made cutouts (repo root; the illustrations dir is * appended). Empty ⇒ download disabled. e.g. https://cdn.jsdelivr.net/gh/@ */ downloadBaseUrl: string - /** Called after each batch completes (reload manifest + republish). */ + /** Notes files, layered in order (bundled pipeline file first, operator's over it). */ + notesPaths: readonly string[] + /** Gap between image-API calls. The generate lane owns the rate limit: it asks the + * pipeline for one pose at a time, so the pipeline's own inter-call sleep never + * applies here. */ + generateGapMs: number + /** Called after art lands. Debounced by the caller — never awaited in a lane. */ onGenerated: () => void | Promise + /** Whether the layout manifest currently describes a pose. Supplied by the caller + * because it already holds the manifest. Used only to diagnose art that is on disk + * yet never published, which otherwise fails completely silently. */ + isDescribed?: (stem: string) => boolean } -const TAG = 'saezuri-generate' -const log = (msg: string) => console.log(`${TAG}: ${msg}`) -const logErr = (e: unknown) => - console.error(`${TAG}: ${e instanceof Error ? e.message : String(e)}`) +const poseName = (suffix: string) => (suffix === FLIGHT_SUFFIX ? 'flight' : 'perched') +const describe = (pose: PoseRequest) => `${pose.sci ?? pose.slug} ${poseName(pose.suffix)}` export class Generator { - private queued = new Map() - private inFlight = new Set() - private busy = false - /** Pose file stem -> when that source last failed to supply it. Keyed per pose, not - * per species, so a half-supplied pair backs off only on the pose that's absent; and - * kept per source, because "the repo doesn't have it" says nothing about whether the - * model can draw it. */ - private misses: Record> = { - download: new Map(), - generate: new Map(), - } - private missesLoaded = false - private missesDirty = false + private dlQueue = new Map() + private dlInFlight = new Set() + private dlBusy = false + + private genQueue = new Map() + private genInFlight = new Set() + private genBusy = false + + private state = new Map() + private stateLoaded = false + private stateDirty = false + private notes: SpeciesNotes = {} + private notesLoadedAt = 0 + /** Keys already reported, so a warning isn't repeated every cycle. */ + private warned = new Set() + /** When a pose was first seen on disk but absent from the manifest. */ + private firstUndescribedMs = new Map() constructor(private opts: GeneratorOptions) {} - /** Enqueue a species for art acquisition. No-ops when neither art source is - * available (no download URL and no Gemini key), already queued, or in flight. */ + private get anySource(): boolean { + return this.opts.enabled || Boolean(this.opts.downloadBaseUrl) + } + + /** Enqueue a heard species for art acquisition. */ enqueue(sci: string, com: string): void { const slug = slugify(sci) if (slug) this.request({ slug, sci, com }) } - /** Enqueue repairs planned by reconcile.ts. A repair with no `sci` is a cutout - * found on disk whose species we couldn't name: downloadable by slug, but never - * generated — we won't spend a Gemini call on a name we guessed. */ + /** Enqueue repairs planned by reconcile.ts. */ enqueueRepairs(repairs: readonly ArtRepair[]): void { for (const r of repairs) this.request(r) } - private request(req: ArtRequest): void { - if (!this.opts.enabled && !this.opts.downloadBaseUrl) return - if (this.inFlight.has(req.slug) || this.queued.has(req.slug)) return - this.queued.set(req.slug, req) - void this.drain() + /** Cheap and idempotent: both lanes dedupe queued and in-flight stems, and each lane + * short-circuits on a pose already on disk or recently missed — so re-running this + * every publish costs no network. */ + private request(req: ArtRepair): void { + if (!this.anySource || !req.slug) return + for (const suffix of POSE_SUFFIXES) { + const stem = `${req.slug}${suffix}` + if (this.dlInFlight.has(stem) || this.dlQueue.has(stem)) continue + if (this.genInFlight.has(stem) || this.genQueue.has(stem)) continue + const pose: PoseRequest = { ...req, suffix } + if (this.opts.downloadBaseUrl) this.dlQueue.set(stem, pose) + else this.genQueue.set(stem, pose) + } + void this.drainDownloads() + void this.drainGenerate() } - private take(n: number): ArtRequest[] { - const out: ArtRequest[] = [] - for (const [slug, sp] of this.queued) { - this.queued.delete(slug) - out.push(sp) - if (out.length >= n) break + private posePath(stem: string): string { + return join(this.opts.assetsDir, `${stem}.png`) + } + + /** Hand a pose to the generate lane. Claimed into the receiving lane's in-flight set + * BEFORE the caller releases it, so a publish landing in between can't see the pose + * as unowned and re-enqueue it. + * + * A repair with no `sci` stops here: it is downloadable by slug, but naming its bird + * is guesswork and a Gemini call on a guessed name is worse than no art. */ + private handToGenerate(stem: string, pose: PoseRequest): void { + if (!this.opts.enabled || !pose.sci) return + if (this.genInFlight.has(stem) || this.genQueue.has(stem)) return + this.genQueue.set(stem, pose) + void this.drainGenerate() + } + + // ---- download lane ------------------------------------------------------- + + private async drainDownloads(): Promise { + if (this.dlBusy) return + this.dlBusy = true + try { + await this.loadState() + while (this.dlQueue.size > 0) { + const batch: Array<[string, PoseRequest]> = [] + for (const entry of this.dlQueue) { + this.dlQueue.delete(entry[0]) + this.dlInFlight.add(entry[0]) + batch.push(entry) + if (batch.length >= DOWNLOAD_CONCURRENCY) break + } + let landed = 0 + try { + const got = await Promise.all(batch.map(([s, p]) => this.acquireFromRepo(s, p))) + landed = got.filter(Boolean).length + await this.saveState() + } catch (e) { + logErr(e) + } finally { + for (const [stem] of batch) this.dlInFlight.delete(stem) + } + if (landed > 0) { + // Downloaded art still needs its silhouette built: unlike the generate lane, + // nothing has run the pipeline for it, and the browser only sees art the + // manifest describes. Once per batch, not per pose, since it rescans the dir. + await this.safeRebuild() + void this.opts.onGenerated() + } + } + } finally { + this.dlBusy = false + } + } + + /** True when a new file landed from the repo. A miss or an error is non-fatal — the + * pose just falls through to the generate lane. */ + private async acquireFromRepo(stem: string, pose: PoseRequest): Promise { + if (await this.settleExisting(stem, pose)) return false + + if (this.backedOff('download', stem)) { + this.handToGenerate(stem, pose) + return false + } + + try { + const res = await fetch(`${this.opts.downloadBaseUrl}/illustrations/${stem}.png`) + if (!res.ok) { + this.recordMiss('download', stem, 'not in the illustrations repo; retrying in 7d') + this.handToGenerate(stem, pose) + return false + } + await mkdir(this.opts.assetsDir, { recursive: true }) + await writeAtomic(this.posePath(stem), Buffer.from(await res.arrayBuffer())) + this.forgetMisses(stem) + this.setState(stem, { source: 'repo' }) + log(`${describe(pose)}: repo`) + return true + } catch (e) { + // Transient by assumption (DNS, reset, CDN blip): leave the miss unrecorded so + // the next cycle retries, exactly as the call library does for a thrown lookup. + logErr(e) + return false } - return out } - private async drain(): Promise { - if (this.busy) return - this.busy = true + // ---- generate lane ------------------------------------------------------- + + private async drainGenerate(): Promise { + if (this.genBusy) return + this.genBusy = true try { - await this.loadMisses() - while (this.queued.size > 0) { - const cap = this.opts.maxPerCycle > 0 ? this.opts.maxPerCycle : this.queued.size - const batch = this.take(cap) - for (const s of batch) this.inFlight.add(s.slug) + await this.loadState() + while (this.genQueue.size > 0) { + const entry = this.genQueue.entries().next() + if (entry.done) break + const [stem, pose] = entry.value + this.genQueue.delete(stem) + this.genInFlight.add(stem) + let landed = false try { - // 1) Free pre-made art first (no key needed). A species is only "done" via - // download once BOTH poses are present; a partial pair still needs generation. - const needGen: ArtRequest[] = [] - let downloadedAny = false - for (const s of batch) { - let complete = this.isComplete(s.slug) - if (!complete && this.opts.downloadBaseUrl) { - const { got, complete: c } = await this.downloadArt(s.slug) - if (got > 0) downloadedAny = true - complete = c - } - // No `sci` means the slug came off disk unnamed — downloadable, never generated. - if (!complete && s.sci && this.worthGenerating(s.slug)) needGen.push(s) - } - // 2) Generate whatever the repo didn't fully supply, only if a Gemini key is - // set. spawnGenerate ends with build_masks, so it rebuilds the manifest; - // pregen skips the pose(s) already on disk, so it only fills the gap. - let generated = false - if (this.opts.enabled && needGen.length > 0) { - await this.spawnGenerate(needGen) - generated = true - this.recordGenerationGaps(needGen) - } - // 3) If we only downloaded, rebuild the manifest so the new art is described - // (no Gemini key needed). Then republish either way, BEFORE clearing - // inFlight, so a fresh detection sees the art and isn't re-enqueued. - if (downloadedAny || generated) { - if (downloadedAny && !generated) await this.rebuildManifest() - await this.opts.onGenerated() - } + landed = await this.generatePose(stem, pose) + await this.saveState() } catch (e) { - // A pipeline failure is transient by assumption (rate limit, restart), so - // no miss is recorded and the next sweep retries — mirrors CallProvider. + // A pipeline failure is transient by assumption (rate limit, restart), so no + // miss is recorded and the next sweep retries — mirrors CallProvider. logErr(e) } finally { - for (const s of batch) this.inFlight.delete(s.slug) - await this.saveMisses() + this.genInFlight.delete(stem) } + if (landed) void this.opts.onGenerated() } } finally { - this.busy = false + this.genBusy = false } } - private posePath(stem: string): string { - return join(this.opts.assetsDir, `${stem}.png`) + /** True when a new file landed from generation. */ + private async generatePose(stem: string, pose: PoseRequest): Promise { + if (!this.opts.enabled || !pose.sci) return false + if (!pose.force) { + if (await this.settleExisting(stem, pose)) return false + if (this.backedOff('generate', stem)) return false + } + + const note = noteFor(await this.getNotes(), pose.sci) + // Pace the calls here rather than in the pipeline: it is invoked once per pose, so + // its own inter-call sleep never fires. + if (this.opts.generateGapMs > 0) await sleep(this.opts.generateGapMs) + + // A forced re-render overwrites a file that is already there, so its presence + // afterwards proves nothing — the mtime is what distinguishes a new render from the + // old one surviving a failed attempt. Getting this wrong would record the failure + // as success under the new note and never retry it. + const beforeMs = pose.force ? mtimeMs(this.posePath(stem)) : undefined + const startedMs = Date.now() + await this.spawn([ + this.opts.workerScript, + '--generate', + `${pose.sci}|${pose.com ?? pose.sci}`, + '--poses', + pose.suffix === FLIGHT_SUFFIX ? '2' : '1', + '--assets-dir', + this.opts.assetsDir, + '--cache-dir', + this.opts.cacheDir, + ...this.opts.notesPaths.flatMap((p) => ['--notes', p]), + ...(pose.force ? ['--force'] : []), + ]) + + const secs = Math.round((Date.now() - startedMs) / 1000) + const afterMs = mtimeMs(this.posePath(stem)) + if (afterMs === undefined || afterMs === beforeMs) { + this.recordMiss('generate', stem, `generation produced nothing (${secs}s); retrying in 24h`) + return false + } + this.forgetMisses(stem) + this.setState(stem, { source: 'generated', noteVer: noteVersion(note) }) + log(`${describe(pose)}: generated (${secs}s${note ? ', +note' : ''})`) + return true + } + + /** A failed rebuild must not abort the lane — the art is on disk either way, and the + * next publish or heal will rebuild. */ + private async safeRebuild(): Promise { + try { + await this.rebuildManifest() + } catch (e) { + logErr(e) + } + } + + // ---- shared decisions ---------------------------------------------------- + + /** Decide what to do about a pose whose file is already on disk. Returns true when + * the pose needs no further work in the calling lane. */ + private async settleExisting(stem: string, pose: PoseRequest): Promise { + if (!existsSync(this.posePath(stem))) return false + + // Whatever a source concluded before, the pose is here now. + this.forgetMisses(stem) + + const st = this.state.get(stem) + // Art that predates this bookkeeping: assume the repo, the safer of the two, so an + // unknown file is never force-replaced by a local render. + const source = st?.source ?? 'repo' + if (st?.source === undefined) this.setState(stem, { source }) + + const note = pose.sci ? noteFor(await this.getNotes(), pose.sci) : undefined + if (source === 'generated' && st?.noteVer !== noteVersion(note)) { + if (this.opts.enabled && pose.sci) { + log(`${describe(pose)}: note changed, re-rendering`) + this.handToGenerate(stem, { ...pose, force: true }) + return true + } + } else if (source === 'repo' && note !== undefined && !this.warned.has(`note:${stem}`)) { + this.warned.add(`note:${stem}`) + log( + `${describe(pose)}: has a note but its art came from the illustrations repo, which ` + + `takes precedence — if the repo's art is wrong, contribute the note upstream`, + ) + } + + this.checkDescribed(stem, pose) + return true } - private isComplete(slug: string): boolean { - return POSE_SUFFIXES.every((suffix) => existsSync(this.posePath(`${slug}${suffix}`))) + /** Surface the one failure that is otherwise completely silent: a pose whose file is + * on disk but which the manifest build rejects (a truncated download, a corrupt + * PNG). Nothing downloads it — the file is there — and nothing generates it — the + * pair looks complete — so it is re-requested forever with no progress and no log. + * + * Gated on elapsed time rather than a number of sightings. The caller tests a + * manifest it reloads only when publishing, so a pose that just landed is legitimately + * absent from it for a while — and a burst of detections could otherwise rack up + * sightings within that window and cry corruption about perfectly good art. */ + private checkDescribed(stem: string, pose: PoseRequest): void { + const isDescribed = this.opts.isDescribed + if (!isDescribed || this.warned.has(`undescribed:${stem}`)) return + if (isDescribed(stem)) { + this.firstUndescribedMs.delete(stem) + return + } + const firstMs = this.firstUndescribedMs.get(stem) + if (firstMs === undefined) { + this.firstUndescribedMs.set(stem, Date.now()) + return + } + if (Date.now() - firstMs < UNDESCRIBED_GRACE_MS) return + this.warned.add(`undescribed:${stem}`) + log( + `${describe(pose)}: ${stem}.png is on disk but the layout manifest keeps rejecting it — ` + + `the file is probably corrupt; delete it to re-acquire`, + ) + } + + /** Notes are re-read periodically rather than cached for the process lifetime, so an + * operator editing the file sees the effect without a restart. */ + private async getNotes(): Promise { + const now = Date.now() + if (now - this.notesLoadedAt < NOTES_TTL_MS) return this.notes + this.notes = await loadNotes(this.opts.notesPaths) + this.notesLoadedAt = now + return this.notes } + // ---- miss bookkeeping ---------------------------------------------------- + /** True when this source recently failed to supply this pose. */ private backedOff(source: MissSource, stem: string): boolean { - const at = this.misses[source].get(stem) + const at = this.state.get(stem)?.[MISS_FIELD[source]] return at !== undefined && Date.now() - at < MISS_TTL_MS[source] } private recordMiss(source: MissSource, stem: string, reason: string): void { // Log only the transition into backoff, so a permanent gap says its piece once // rather than on every sweep. - if (!this.misses[source].has(stem)) log(`${stem}.png: ${reason}`) - this.misses[source].set(stem, Date.now()) - this.missesDirty = true + if (this.state.get(stem)?.[MISS_FIELD[source]] === undefined) log(`${stem}.png: ${reason}`) + this.setState(stem, { [MISS_FIELD[source]]: Date.now() }) } /** The pose landed, so neither source's verdict stands any more. */ private forgetMisses(stem: string): void { - for (const source of MISS_SOURCES) { - if (this.misses[source].delete(stem)) this.missesDirty = true - } + const st = this.state.get(stem) + if (!st) return + if (st.downloadMissAt === undefined && st.generateMissAt === undefined) return + this.setState(stem, { downloadMissAt: undefined, generateMissAt: undefined }) } - /** True while at least one absent pose is still worth asking the model for. Without - * this, a species the model keeps declining would be regenerated every publish. */ - private worthGenerating(slug: string): boolean { - return POSE_SUFFIXES.some((suffix) => { - const stem = `${slug}${suffix}` - return !existsSync(this.posePath(stem)) && !this.backedOff('generate', stem) - }) + // ---- state persistence --------------------------------------------------- + + private setState(stem: string, patch: PoseState): void { + this.state.set(stem, { ...this.state.get(stem), ...patch }) + this.stateDirty = true } - /** After a generation run, any pose still absent is one the model didn't produce - * (worker.py exits 0 on a partial run so the batch isn't aborted). */ - private recordGenerationGaps(batch: readonly ArtRequest[]): void { - for (const s of batch) { - for (const suffix of POSE_SUFFIXES) { - const stem = `${s.slug}${suffix}` - if (existsSync(this.posePath(stem))) this.forgetMisses(stem) - else this.recordMiss('generate', stem, 'generation produced nothing; retrying in 24h') + private async loadState(): Promise { + if (this.stateLoaded) return + this.stateLoaded = true + try { + const raw = await readFile(join(this.opts.assetsDir, STATE_FILE), 'utf8') + const data: unknown = JSON.parse(raw) + if (data === null || typeof data !== 'object' || Array.isArray(data)) return + for (const [stem, value] of Object.entries(data)) { + if (value === null || typeof value !== 'object') continue + const v = value as PoseState + this.state.set(stem, { + downloadMissAt: typeof v.downloadMissAt === 'number' ? v.downloadMissAt : undefined, + generateMissAt: typeof v.generateMissAt === 'number' ? v.generateMissAt : undefined, + source: v.source === 'repo' || v.source === 'generated' ? v.source : undefined, + noteVer: typeof v.noteVer === 'string' ? v.noteVer : undefined, + }) } + return + } catch { + // Absent or unreadable: fall through to the legacy file, then start empty. Costs + // one round of repo probes, which is why this must never be authoritative. } + await this.seedFromLegacyMisses() } - /** Download whichever poses the repo has for a slug into assetsDir, skipping poses - * already on disk or recently missed. Returns how many new files landed (`got`) and - * whether the species is now `complete` (both poses present). A miss/error is - * non-fatal — the species just falls through to generation. Writes atomically. */ - private async downloadArt(slug: string): Promise<{ got: number; complete: boolean }> { - await mkdir(this.opts.assetsDir, { recursive: true }) - let got = 0 - for (const suffix of POSE_SUFFIXES) { - const stem = `${slug}${suffix}` - const dest = this.posePath(stem) - if (existsSync(dest)) continue - if (this.backedOff('download', stem)) continue - try { - const res = await fetch(`${this.opts.downloadBaseUrl}/illustrations/${stem}.png`) - if (!res.ok) { - this.recordMiss('download', stem, 'not in the illustrations repo; retrying in 7d') - continue + /** Carry over the backoff recorded by the previous format, so upgrading doesn't + * re-probe the repo for every species we already know it lacks. */ + private async seedFromLegacyMisses(): Promise { + try { + const raw = await readFile(join(this.opts.assetsDir, LEGACY_MISSES_FILE), 'utf8') + const data = JSON.parse(raw) as Partial>> + let seeded = 0 + for (const source of MISS_SOURCES) { + for (const [stem, at] of Object.entries(data[source] ?? {})) { + if (typeof at !== 'number') continue + this.state.set(stem, { ...this.state.get(stem), [MISS_FIELD[source]]: at }) + seeded++ } - const buf = Buffer.from(await res.arrayBuffer()) - const tmp = `${dest}.tmp` - await writeFile(tmp, buf) - await rename(tmp, dest) - this.forgetMisses(stem) - got++ - log(`downloaded ${stem}.png`) - } catch (e) { - logErr(e) // network error — transient, no miss recorded; fall through to generation } + if (seeded > 0) { + this.stateDirty = true + log(`carried over ${seeded} recorded gap(s) from ${LEGACY_MISSES_FILE}`) + } + } catch { + // No legacy file either: start empty. } - return { got, complete: this.isComplete(slug) } } - private spawnGenerate(batch: ArtRequest[]): Promise { - log(`generating ${batch.map((s) => s.sci).join(', ')}`) - const args = [ + private async saveState(): Promise { + if (!this.stateDirty) return + this.stateDirty = false + try { + await mkdir(this.opts.assetsDir, { recursive: true }) + await writeAtomic( + join(this.opts.assetsDir, STATE_FILE), + JSON.stringify(Object.fromEntries(this.state)), + ) + } catch (e) { + logErr(e) // non-fatal: we just re-probe after a restart + } + } + + // ---- pipeline invocations ------------------------------------------------ + + /** Ensure a manifest (and the fallback silhouette) exist without generating art, so + * the browser always fetches a real manifest file. */ + rebuildManifest(): Promise { + return this.spawn([ this.opts.workerScript, - '--generate', - ...batch.map((s) => `${s.sci}|${s.com ?? s.sci}`), + '--rebuild', '--assets-dir', this.opts.assetsDir, '--cache-dir', this.opts.cacheDir, - ] - return this.spawn(args) + ]) } - /** Ensure a manifest (and the fallback silhouette) exist without generating art, - * so the browser always fetches a real manifest file. */ - rebuildManifest(): Promise { + /** Rebuild the manifest, first cutting out any render an older pipeline version left + * on the magenta ground. Those files are invisible to everything else: nothing + * re-mattes them and the download path won't overwrite them. */ + repairAndRebuildManifest(): Promise { return this.spawn([ this.opts.workerScript, - '--rebuild', + '--repair', '--assets-dir', this.opts.assetsDir, '--cache-dir', @@ -269,43 +537,6 @@ export class Generator { ]) } - private async loadMisses(): Promise { - if (this.missesLoaded) return - this.missesLoaded = true - try { - const raw = await readFile(join(this.opts.assetsDir, MISSES_FILE), 'utf8') - const data = JSON.parse(raw) as Partial>> - for (const source of MISS_SOURCES) { - for (const [stem, at] of Object.entries(data[source] ?? {})) { - if (typeof at === 'number') this.misses[source].set(stem, at) - } - } - } catch { - // Absent or unreadable: start empty and rebuild it. Deleting the file is also - // the documented way to force an immediate retry of every known gap. - } - } - - private async saveMisses(): Promise { - if (!this.missesDirty) return - this.missesDirty = false - try { - await mkdir(this.opts.assetsDir, { recursive: true }) - const path = join(this.opts.assetsDir, MISSES_FILE) - const tmp = `${path}.tmp` - await writeFile( - tmp, - JSON.stringify({ - download: Object.fromEntries(this.misses.download), - generate: Object.fromEntries(this.misses.generate), - }), - ) - await rename(tmp, path) - } catch (e) { - logErr(e) // non-fatal: we just re-attempt after a restart - } - } - private spawn(args: string[]): Promise { return new Promise((resolve, reject) => { const child = spawn(this.opts.pythonBin, args, { stdio: 'inherit', env: process.env }) @@ -316,3 +547,28 @@ export class Generator { }) } } + +/** How long a pose may sit on disk undescribed by the manifest before that is treated + * as corruption rather than as the manifest not having caught up. Comfortably longer + * than a manifest rebuild, and shorter than the aging tick that re-asks for it. */ +const UNDESCRIBED_GRACE_MS = 60_000 + +/** How long a loaded notes set is reused before being re-read from disk. Short enough + * that an edit is picked up promptly, long enough that a burst of poses doesn't re-read + * the file per pose. */ +const NOTES_TTL_MS = 10_000 + +/** Modification time in ms, or undefined when the file isn't there. */ +function mtimeMs(path: string): number | undefined { + try { + return statSync(path).mtimeMs + } catch { + return undefined + } +} + +async function writeAtomic(path: string, data: Buffer | string): Promise { + const tmp = `${path}.tmp` + await writeFile(tmp, data) + await rename(tmp, path) +} diff --git a/src/server/notes.test.ts b/src/server/notes.test.ts new file mode 100644 index 0000000..1b9c750 --- /dev/null +++ b/src/server/notes.test.ts @@ -0,0 +1,103 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { loadNotes, noteFor, noteVersion } from './notes.ts' + +let dir: string +const write = async (name: string, body: unknown) => { + const path = join(dir, name) + await writeFile(path, typeof body === 'string' ? body : JSON.stringify(body)) + return path +} + +beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'saezuri-notes-')) +}) +afterEach(async () => { + await rm(dir, { recursive: true, force: true }) +}) + +describe('loadNotes', () => { + it('is empty with no paths', async () => { + expect(await loadNotes([])).toEqual({}) + }) + + it('ignores a missing file', async () => { + expect(await loadNotes([join(dir, 'nope.json')])).toEqual({}) + }) + + it('drops comment keys and non-string values', async () => { + const p = await write('n.json', { + 'Turdus merula': 'keep me', + _comment: 'drop me', + 'Parus major': { not: 'a string' }, + 'Erithacus rubecula': 42, + }) + expect(await loadNotes([p])).toEqual({ 'Turdus merula': 'keep me' }) + }) + + it('layers later files over earlier ones per key', async () => { + const bundled = await write('bundled.json', { a: 'from bundled', b: 'only bundled' }) + const operator = await write('operator.json', { a: 'from operator', c: 'only operator' }) + expect(await loadNotes([bundled, operator])).toEqual({ + a: 'from operator', + b: 'only bundled', + c: 'only operator', + }) + }) + + it('is idempotent when a layer is passed twice', async () => { + // The pipeline prepends the bundled layer itself, so the service passing it too + // means it appears twice. That has to be a no-op. + const bundled = await write('bundled.json', { a: 'bundled' }) + const operator = await write('operator.json', { a: 'operator' }) + expect(await loadNotes([bundled, bundled, operator])).toEqual( + await loadNotes([bundled, operator]), + ) + }) + + it('skips a malformed file instead of failing the load', async () => { + const bad = await write('bad.json', '{ not json') + const good = await write('good.json', { a: 'kept' }) + expect(await loadNotes([bad, good])).toEqual({ a: 'kept' }) + }) + + it('skips a file that is not a JSON object', async () => { + const arr = await write('arr.json', ['not', 'an', 'object']) + const good = await write('good.json', { a: 'kept' }) + expect(await loadNotes([arr, good])).toEqual({ a: 'kept' }) + }) +}) + +describe('noteFor', () => { + it('resolves a scientific-name key', () => { + expect(noteFor({ 'Turdus merula': 'n' }, 'Turdus merula')).toBe('n') + }) + + it('resolves a slug key', () => { + expect(noteFor({ 'turdus-merula': 'n' }, 'Turdus merula')).toBe('n') + }) + + it('prefers the scientific name over the slug', () => { + expect(noteFor({ 'Turdus merula': 'sci', 'turdus-merula': 'slug' }, 'Turdus merula')).toBe( + 'sci', + ) + }) + + it('is undefined for a species with no note', () => { + expect(noteFor({ 'Parus major': 'n' }, 'Turdus merula')).toBeUndefined() + }) +}) + +describe('noteVersion', () => { + it('distinguishes no note from any note', () => { + expect(noteVersion(undefined)).toBeUndefined() + expect(noteVersion('')).toBeDefined() + }) + + it('is stable for the same note and differs for a changed one', () => { + expect(noteVersion('darker bill')).toBe(noteVersion('darker bill')) + expect(noteVersion('darker bill')).not.toBe(noteVersion('darker bill.')) + }) +}) diff --git a/src/server/notes.ts b/src/server/notes.ts new file mode 100644 index 0000000..b88fb00 --- /dev/null +++ b/src/server/notes.ts @@ -0,0 +1,58 @@ +import { readFile } from 'node:fs/promises' +import { slugify } from '../domain/slug.ts' +import { fnv1a } from '../lib/hash.ts' + +// Per-species prompt addenda. The pipeline is what actually injects these into the +// image prompt (pregen.py: load_species_notes / note_for); the service reads them +// only to notice that a note CHANGED, which is what lets it re-render art it +// generated under an older note. Both sides must resolve a species to the same +// note, so the layering order and the scientific-name-then-slug lookup here are +// kept in parity with pregen.py deliberately — change one and change the other. + +const TAG = 'saezuri-notes' +const logErr = (msg: string) => console.error(`${TAG}: ${msg}`) + +export type SpeciesNotes = Readonly> + +/** Load and layer notes files, later paths winning per key — the bundled pipeline + * file first, an operator's own file over it. A missing file contributes nothing; + * a malformed one is reported and skipped rather than taking generation down with + * it, since these are hand-edited. */ +export async function loadNotes(paths: readonly string[]): Promise { + const merged: Record = {} + for (const path of paths) { + if (!path) continue + let raw: string + try { + raw = await readFile(path, 'utf8') + } catch { + continue // absent is the normal case for the operator layer + } + try { + const parsed: unknown = JSON.parse(raw) + if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) { + logErr(`ignoring ${path}: not a JSON object`) + continue + } + for (const [key, value] of Object.entries(parsed)) { + // `_`-prefixed keys are comments, matching the convention used for + // `_fallback.png` and the call library's `_misses.json`. + if (!key.startsWith('_') && typeof value === 'string') merged[key] = value + } + } catch (e) { + logErr(`ignoring ${path}: ${e instanceof Error ? e.message : String(e)}`) + } + } + return merged +} + +/** Resolve a species' note by scientific name, falling back to its slug. */ +export function noteFor(notes: SpeciesNotes, scientificName: string): string | undefined { + return notes[scientificName] ?? notes[slugify(scientificName)] +} + +/** Short token identifying the note a pose was generated under. `undefined` for no + * note, so "never had one" and "had one that was removed" compare as different. */ +export function noteVersion(note: string | undefined): string | undefined { + return note === undefined ? undefined : fnv1a(note).toString(36) +} diff --git a/src/server/refresh.ts b/src/server/refresh.ts index 78fa0ce..b5ddda9 100644 --- a/src/server/refresh.ts +++ b/src/server/refresh.ts @@ -37,6 +37,9 @@ const CALL_PROVIDER_FACTORIES: Record CallProvider> = { const ASSET_HEAL_DEBOUNCE_MS = 2_000 const ASSET_WATCH_REARM_MS = 1_000 +// Trailing window for republishing after art lands. See scheduleArtPublish. +const ART_PUBLISH_DEBOUNCE_MS = 3_000 + // Resolve a comma-separated env override to the subset of `all` it names (in // `all`'s order). Unknown codes are dropped; an empty or all-unknown list falls // back to the full set. @@ -71,7 +74,8 @@ interface Config { assetsDir: string callsDir: string cacheDir: string - maxPerCycle: number + notesPaths: string[] + generateGapMs: number callProviders: CallProviderName[] callsMaxPerCycle: number frameWidth: number @@ -89,6 +93,16 @@ function intEnv(name: string, def: number): number { return Number.isFinite(n) && n > 0 ? n : def } +/** Read a duration given in seconds (possibly fractional) as milliseconds. Unlike + * intEnv, zero is meaningful here — it is how an operator on a paid tier turns the + * inter-call gap off entirely. */ +function secondsEnvMs(name: string, defSeconds: number): number { + const raw = (process.env[name] ?? '').trim() + if (!raw) return defSeconds * 1000 + const n = Number(raw) + return Number.isFinite(n) && n >= 0 ? Math.round(n * 1000) : defSeconds * 1000 +} + function readConfig(): Config { const baseUrl = (process.env.BIRDNETGO_URL ?? '').trim() if (!baseUrl) throw new Error('BIRDNETGO_URL is required') @@ -106,6 +120,18 @@ function readConfig(): Config { (illustrationsRepo ? `https://cdn.jsdelivr.net/gh/${illustrationsRepo}@${illustrationsRef}` : '') + const workerScript = (process.env.WORKER_SCRIPT ?? '/opt/saezuri/pipeline/worker.py').trim() + const assetsDir = join(htmlDir, 'assets', 'illustrations') + // Prompt addenda, layered: the pipeline's own species-notes.json carries what the + // community has learned, and an operator's file refines it per key. The operator's + // has to sit on the writable volume — beside the art is where they will look for it. + // Both layers are passed to the pipeline verbatim; it prepends the bundled one + // itself, and merging a layer twice is a no-op, so the duplicate is harmless and + // keeps this list identical to the one the service hashes for note changes. + const notesPaths = [ + join(workerScript, '..', 'species-notes.json'), + (process.env.SPECIES_NOTES ?? '').trim() || join(assetsDir, '_species-notes.json'), + ] return { baseUrl, token: (process.env.BIRDNETGO_TOKEN ?? '').trim() || undefined, @@ -116,11 +142,15 @@ function readConfig(): Config { geminiEnabled: Boolean((process.env.GEMINI_API_KEY ?? '').trim()), downloadBaseUrl, pythonBin: (process.env.PYTHON_BIN ?? 'python3').trim(), - workerScript: (process.env.WORKER_SCRIPT ?? '/opt/saezuri/pipeline/worker.py').trim(), - assetsDir: join(htmlDir, 'assets', 'illustrations'), + workerScript, + assetsDir, callsDir: join(htmlDir, 'assets', 'calls'), cacheDir: (process.env.CACHE_DIR ?? '/var/cache/saezuri').trim(), - maxPerCycle: intEnv('GENERATE_MAX_PER_CYCLE', 4), + notesPaths, + // The generate lane asks the pipeline for one pose per invocation, so the + // pipeline's own inter-call sleep never fires and this is the only thing keeping + // us under the image API's rate limit. + generateGapMs: secondsEnvMs('GENERATE_SLEEP', 6), // Unlike the other CSV settings, an explicitly empty CALL_PROVIDERS means // "off" rather than "all" — it is the way to stop the service reaching out // to third-party archives at all. @@ -146,6 +176,7 @@ class Refresher { private sciBySlug: ReadonlyMap = new Map() private lastSummaryMs = 0 private pendingPublish?: ReturnType + private pendingArtPublish?: ReturnType private generator: Generator private callLibrary: CallLibrary private lastFrameSig = new Map() @@ -161,11 +192,16 @@ class Refresher { workerScript: cfg.workerScript, assetsDir: cfg.assetsDir, cacheDir: cfg.cacheDir, - maxPerCycle: cfg.maxPerCycle, enabled: cfg.geminiEnabled, downloadBaseUrl: cfg.downloadBaseUrl, - // Reload the manifest + republish so the freshly-acquired art appears. - onGenerated: () => this.safe('publish', () => this.publish()), + notesPaths: cfg.notesPaths, + generateGapMs: cfg.generateGapMs, + // Reload the manifest + republish so the freshly-acquired art appears. Debounced + // rather than immediate: acquisition is now one pose at a time, and a publish + // rebuilds every mask and re-renders every frame, so a fast run of poses must + // coalesce into one rebuild instead of paying that per pose. + onGenerated: () => this.scheduleArtPublish(), + isDescribed: (key) => key in this.manifest.masks, }) this.callLibrary = new CallLibrary({ callsDir: cfg.callsDir, @@ -315,6 +351,18 @@ class Refresher { } } + /** Coalesce a run of acquired poses into one trailing publish. Separate from + * schedulePublish because art lands on a much faster cadence than detections and + * wants a much shorter window: long enough that a burst of repo downloads rebuilds + * the manifest once, short enough that a slow render still shows up promptly. */ + private scheduleArtPublish(): void { + if (this.pendingArtPublish) return + this.pendingArtPublish = setTimeout(() => { + this.pendingArtPublish = undefined + void this.safe('publish', () => this.publish()) + }, ART_PUBLISH_DEBOUNCE_MS) + } + /** Coalesce a burst of detections into one trailing publish per debounce * window, so a dawn chorus doesn't rewrite the snapshot on every event. */ private schedulePublish(): void { @@ -334,7 +382,11 @@ class Refresher { log( `starting; publishing to ${this.cfg.htmlDir} (art source: ${artSource}, call source: ${callSource})`, ) - await this.safe('rebuild', () => this.generator.rebuildManifest()) + // Repair before the first publish: an older pipeline wrote the magenta render under + // its real filename and matted it in a second pass, so a run killed in between left + // a rectangle nothing else would ever revisit. Rebuilds the manifest as well, so it + // stands in for the plain rebuild — and has to run before the count below reads it. + await this.safe('repair', () => this.generator.repairAndRebuildManifest()) await this.safe('unpaired', async () => { const unpaired = unpairedSlugs(await loadManifest(this.cfg.htmlDir)) if (unpaired.length > 0) log(`${unpaired.length} half-illustrated species on disk`)