Conversation
…gration Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AX5fZLCKjxyASaMAxMxxum
…Turso)
Phase 0: swap the DB access mechanism only — business logic, schema, and the
OLX collection method are unchanged. libSQL is SQLite-compatible (single code
path: file: locally, Turso URL in prod), enabling Render + Turso deploy.
- db.ts: createClient + initDb (executeMultiple schema.sql) + thin async
helpers dbGet/dbAll/dbRun; drop the historical migration scaffold
(addColumnIfMissing/migrateListingsTable/backfill/PRAGMA/WAL) — schema.sql
already holds every column.
- env.ts: load server/.env before db.ts reads TURSO_* (imported first in db.ts).
- Whole DB layer is async: every .get/.all/.run -> await dbGet/dbAll/dbRun;
interactive db.transaction('write') for read->decide->write (upsert,
statusEngine, analysis/relevance/aiPicks commit); db.batch('write') for pure
write sets (cascade delete, sort swaps, filtered_out recompute, migratePostedAt).
- index.ts: host 0.0.0.0, CORS origin from WEB_ORIGIN, await initDb before listen.
- CLI entrypoints (scan.ts, migratePostedAt.ts) call initDb.
- .env.example: TURSO_DATABASE_URL/TURSO_AUTH_TOKEN/WEB_ORIGIN.
- docs: architecture.md/structure.md updated; render-turso-phase0.md marked done.
Verified locally against libSQL file:: build green; /health ok; live OLX scan
(GraphQL) -> upsert+dedup; coverage-window disable; manual PATCH override;
CLI scan; cascade-delete/recompute via batch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AX5fZLCKjxyASaMAxMxxum
Beginner-friendly walkthrough: Turso DB setup, Render Web Service (backend) with build/start commands and env vars, Render Static Site (frontend) with /api/* rewrite (same-origin, no CORS), verification, free-tier cold-start notes, and troubleshooting. No code changes — frontend keeps relative /api fetch; the Render rewrite proxies it to the backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AX5fZLCKjxyASaMAxMxxum
Gate access to the dashboard behind Google login (allowlist of one email) without turning the app multi-user — no users table, no user_id, no data partitioning. Auth is a lock on the front door plus a session cookie. Server (server/src/auth/): - config.ts: env (GOOGLE_CLIENT_ID/ALLOWED_EMAILS/SESSION_SECRET), cookie flags (prod cross-site Secure+SameSite=None, local http Lax), fail-fast assertAuthConfigured, AUTH_DISABLED dev bypass. - plugin.ts: fastify-plugin (non-encapsulated) registering @fastify/cookie and @fastify/jwt, verifyGoogleIdToken via google-auth-library, global onRequest lock on /api/* (skips /health, /api/auth/*, CORS preflight). - routes.ts: POST /api/auth/google, GET /api/auth/me, POST /api/auth/logout. - index.ts: import ./env first, assertAuthConfigured, CORS credentials, register auth before domain routes. Frontend (web/src/auth/): - useAuth.ts: useSession/useLogin/useLogout, reacts to global 401 event. - AuthGate.tsx: login screen with GoogleLogin, wraps the app. - base.ts: credentials: 'include', VITE_API_BASE prefix, 401 -> gate event. - main.tsx: GoogleOAuthProvider. Header: logout button. DB schema untouched. Docs + .env examples updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015wZmcZ6mmoQa5rv9gCa1KX
… cache tuning
Optimize database reads/writes for the Turso (libSQL) backend, where every
execute is a network round-trip.
- normalizer.upsertListings: replace the per-listing loop (EXISTS -> upsert ->
read-back -> UPDATE filtered_out, ~4 round-trips/row) with one bulk SELECT of
existing fields, in-memory COALESCE merge + filtered_out, and a single
db.batch('write'). A scan of N listings now does ~2 round-trips instead of ~4N.
filtered_out is folded into the UPSERT (no separate read/write).
- schema: add idx_listings_search_refresh (coverage window) and
idx_listings_search_lastseen (verify P1).
- scanRunLifecycle: throttle scan-progress writes to >=1s (poll is 1.5s),
avoiding dozens of redundant UPDATEs during deep scans.
- web: QueryClient defaults staleTime 60s + refetchOnWindowFocus false to stop
refetching the full listings list on every window focus.
- docs/architecture.md updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHUDe59mPDD1miT5km3n3u
strict mode bug
Render cannot reach www.googleapis.com cert endpoints (v1 PEM → 403, v3 JWK → non-200), so google-auth-library/jose-JWKS verification fails in production. Verify the ID token primarily via the tokeninfo endpoint on oauth2.googleapis.com (a reachable host; Google checks the signature, we validate aud/iss/email_verified), falling back to local jose+JWKS verification where www.googleapis.com is reachable (e.g. local dev). Якщо на Render увімкнено auto-deploy з гілки — після пушу він сам передеплоїться. Підтверди — і я запушу (або скажи, чи хочеш разом із рештою змін цієї сесії: оптимізація БД + фронтенд-фікси).
…sses On a plain page reload these always-fetched (stats) / on-demand (filter-options) endpoints scanned the search's listings multiple times. On Turso each pass bills N rows read, so for ~400 listings a reload cost ~2600 reads. - /stats: replace 4 separate scans (in_db COUNT, stale COUNT, verify P1 COUNT, verify P2 COUNT) with one aggregate using conditional SUM(CASE...). Reuse P1_CONDITION/P2_CONDITION (now exported from verifyScan.ts) so the verify predicate stays single-sourced. ~1600 -> ~400 reads. - /filter-options: replace 4 scans (DISTINCT city, DISTINCT seller_name, pros, cons) with one SELECT and JS dedup. categories still from category_facet. - SearchStats / FilterOptions contracts unchanged; frontend untouched. Verified on a temp DB: aggregate stats and consolidated filter lists match the old per-query results exactly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHUDe59mPDD1miT5km3n3u
…sses On a plain page reload these always-fetched (stats) / on-demand (filter-options) endpoints scanned the search's listings multiple times. On Turso each pass bills N rows read, so for ~400 listings a reload cost ~2600 reads. - /stats: replace 4 separate scans (in_db COUNT, stale COUNT, verify P1 COUNT, verify P2 COUNT) with one aggregate using conditional SUM(CASE...). Reuse P1_CONDITION/P2_CONDITION (now exported from verifyScan.ts) so the verify predicate stays single-sourced. ~1600 -> ~400 reads. - /filter-options: replace 4 scans (DISTINCT city, DISTINCT seller_name, pros, cons) with one SELECT and JS dedup. categories still from category_facet. - SearchStats / FilterOptions contracts unchanged; frontend untouched. Verified on a temp DB: aggregate stats and consolidated filter lists match the old per-query results exactly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHUDe59mPDD1miT5km3n3u
…q4p' into claude/turso-db-optimization-dv0q4p
new optimization for render-turso
…pass
The /stats endpoint ran a separate SUM(CASE…) scan over all listings on every
search-select to derive in_db/stale_count/verify_candidates — all derivable from
the listings array the client already has. Now /stats returns only last_scan
(~1 read) and the client computes aggregates via computeListingStats().
- server: trim /api/searches/:id/stats to LastScanResponse (last_scan only)
- web: add utils/searchStats.ts; useSearchStats merges last_scan + derived counts
from the ['listings'] cache (deduped with the table — no extra request)
- bonus: verify_candidates now updates live on status edits (same cache)
- expected: /stats 410→~1 read, search-select 1228→~819 (−33%); verify after deploy
docs: plan docs/plans/turso-stats-clientside.md, structure.md
…nder 401)
The blob-download helpers for the AI workflows (analysis package, relevance
package, AI Picks package, and the xlsx/json preview export) used a bare
fetch('/api/...') without the API_BASE prefix or credentials: 'include'. On
Render the web app and API are separate origins, so the SameSite=None session
cookie was not sent and the global JWT lock returned 401. Locally the Vite proxy
made it same-origin, hiding the bug.
Add a shared apiBlob() helper in api/base.ts that mirrors api() (API_BASE prefix,
credentials: 'include', conditional Content-Type, 401 -> re-login event) but
resolves to a Blob, and route all four download functions through it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECwRYEMkkETto1AWc6e7Jm
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Проблема
На деплої Render завантаження пакета ручного режиму з будь-якого AI-workflow (аналіз плюси/мінуси, фільтр релевантності, AI Picks) падало з 401 «Не авторизовано». Локально працювало.
Причина
Авторизація — це глобальний
onRequest-замок (server/src/auth/plugin.ts:105-116), що викликаєrequest.jwtVerify()на кожному/api/*і повертає 401, якщо немає сесійного JWT. JWT лежить у кукіolx_session, виставленій у проді якSameSite=None; Secure(server/src/auth/config.ts:67-82). На Render фронт і API — різні origin (olx-dashboard-webvsolx-dashboard-api), тож браузер шле цю кукі лише коли запит маєcredentials: 'include'.Спільний враппер
api()(web/src/api/base.ts) робить це правильно. Але хелпери завантаження blob обходилиapi()і використовували голийfetch('/api/...')без префіксаAPI_BASEі безcredentials: 'include'→ кукі не йшла → JWT verify падав → 401. Локально Vite-проксі робить запит same-origin, тож кукі слалася за замовчуванням і баг був невидимий.Зміни
web/src/api/base.ts— новий спільний хелперapiBlob(), що дзеркалитьapi()(префіксAPI_BASE,credentials: 'include', умовнийContent-Type, подія ре-логіну на 401), але резолвиться вBlob.apiBlob()+ наявнийdownloadBlob():web/src/api/analysis.ts—fetchAnalyzePackageZip,exportPreview(xlsx/json)web/src/api/relevance.ts—fetchRelevancePackageZipweb/src/api/aiPicks.ts—fetchAiPicksPackageZipБез змін бекенду, схеми чи залежностей. Окрім фікса кукі, тепер протермінована сесія коректно повертає застосунок на логін-гейт замість тихої помилки.
Перевірка
npm run build -w web—tsc -b+ Vite build проходять.web/srcне лишилось голихfetch('/api...— усі виклики йдуть черезapi()/apiBlob().olx_sessionі повертає 200.🤖 Generated with Claude Code
Generated by Claude Code