feat(picturebooks): second StoryWeaver batch — 154 more Kannada picture books - #98
Conversation
…re books Expands the C-20 picture-book pipeline (#86) from 40 to 194 books, fetched through the same scripts/fetch-storyweaver.ts pipeline: top-rated by Ratings, PUBLISHER-level, non-GIF, deduplicated against the first batch, and skipped automatically wherever the attribution page does not declare CC BY 4.0 (one candidate skipped this round for that reason). Manifest rebuilt via npm run data:picturebooks; npm run data:validate, npm test, npm run typecheck, and npm run build all pass; spot-checked the hub, reader, and credits page for a new book in the browser. Signed-off-by: Devu Dilip <devu.dilip@gmail.com> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Deploying sirigannada with
|
| Latest commit: |
1174946
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c19e4bc0.sirigannada.pages.dev |
| Branch Preview URL: | https://claude-storyweaver-book-inte.sirigannada.pages.dev |
…nnada books extractParagraphs read only the first div.content block before the page number. StoryWeaver's full-bleed layouts (sp_h_i100 / sp_v_i100) leave that block empty and place the words in positioned content blocks after it, so 29 books shipped with no text at all. The extractor now collects every content block in DOM order; page text was refetched for those 29 (images untouched): 21 recovered their Kannada text, 8 are genuine wordless picture books and stay as they are. Five slugs resolved to other-language originals (the numeric StoryWeaver id decides the story): 219-anveshaneya-kathegalu and 219-panchatantra-kathegalu (the same Marathi book twice), 12908-summane-kootiruvude-heege and 16394-hoo-mattu-jenu-nona (Hindi), 12817-naanu-baalak-illa-baalike (English). Removed from the sources, assets and catalogue. 189 books remain; every title and page text is Kannada. Signed-off-by: Devu Dilip <devu.dilip@gmail.com>
|
Review fix pushed (1174946), on top of the batch commit: Extractor bug, 29 books shipped with no text. Five books were not Kannada (the numeric id decides which story the API returns): 219-anveshaneya-kathegalu and 219-panchatantra-kathegalu (same Marathi book twice), 12908 and 16394 (Hindi), 12817 (English). Removed from sources, assets and the catalogue. 189 books remain; a script scan finds every title and page text in Kannada script. Verified: Follow-ups, not in this PR: the manifest is 633 KB because each entry carries its full provenance block and is fetched on first paint of home and library; and each batch adds ~270 MB to the repo. |
…rving them cache-first forever (#101) (#102) Every /data/** request was cache-first, and DATA_CACHE has not been bumped since v5, so an installed app kept the manifest it cached on install: after #98 shipped 154 more picture books, existing installs still listed 40. The catalogue files in PRECACHE_DATA (manifests, proverbs, game data) are now stale-while-revalidate in the same cache: instant from cache, refreshed in the background, new books on the next open. Shards, book text, images and audio stay cache-first. Existing DATA_CACHE entries survive; the changed sw.js also re-runs install, which overwrites the precached catalogues right away. Signed-off-by: Devu Dilip <devu.dilip@gmail.com>
1 participant
Summary
scripts/fetch-storyweaver.tspipeline: top-rated by Ratings,publisher.type === "PUBLISHER", non-GIF — same filter as the first batch, deduplicated against every slug already indata/picturebooks-src/22-chuskit-shalege-hoguthale) was auto-skipped: its attribution page doesn't declare CC BY 4.0data/picturebooks-src/catalogue.txtextended with the new batch's slug list for provenancenpm run data:picturebooks), all books re-validated (npm run data:validate)Verification
npm run typecheck— cleannpm test— 662/662 passingnpm run data:validate— 194 picture books validated, 0 errorsnpm run build— succeeds, generates reader + credits static pages for all 194 books/creditspage render the full CC BY 4.0 attribution (source URL, licence, translator/author/illustrator, per-illustration credits)Note
This roughly quadruples the picture-book asset payload (~279MB across images/audio, same non-LFS approach as the first batch) — flagging for awareness since it's a meaningful jump in repo size, not because anything is broken.
🤖 Generated with Claude Code