fix: restaurar max_tokens=512 y dejar de matar generaciones lentas - #127
Merged
Conversation
Reverts the cap to 512 and moves the fix to where the defect actually was. The cap was never wasting anything: across the ten recorded generations in bob/sessions/Sprint_1/baseline_eval.md the longest output was 415 tokens and all ten ended on a complete sentence. Lowering it to 320 would have cut real prose to buy latency headroom. The real problem is that HARD_TIMEOUT_SECONDS was the MVP's *SLA* number (<8s P95) used as a *kill* threshold, so no band existed in which a generation is late but working: a conditioned call measured at 8.4s worst case got retried and ultimately failed rather than returned, surfacing as an intermittent 503. The timeout goes to 15s (~1.8x that worst case). The SLA is unchanged and still tracked — now measured on results rather than enforced by discarding them. The client abort in frontend/src/lib/api.ts goes 10s -> 20s for the same reason: at 10s the browser would cancel requests the backend was about to answer, showing a timeout for a generation that had in fact succeeded. Decision by the repo owner (2026-07-28): the demo is an edited video, not a live timed run, so complete prose is worth more than a shorter worst case. Verified: pytest backend/tests/test_generation_params.py -q -> 3 passed; npx vitest run (frontend) -> 107 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Este commit se subió a
fix/106-max-tokensdespués de que se mergeara el PR #123, así que no llegó amain. Hoymaintiene 320 y timeout de 8 s, que es justo lo contrario de la decisión tomada.Decisión (owner, 2026-07-28)
La demo es un vídeo editado, no una ejecución en vivo cronometrada, así que la prosa completa vale más que un peor caso corto.
Qué cambia
max_tokensvuelve a 512. Medido en las diez generaciones registradas debob/sessions/Sprint_1/baseline_eval.md: la salida más larga fueron 415 tokens y las diez cerraron frase. Bajar a 320 recortaba prosa real, no holgura.HARD_TIMEOUT_SECONDS: 8 s → 15 s. El 8 era el número del SLA usado como umbral de muerte: no existía ninguna banda en la que una generación llegue tarde pero funcione. Una llamada condicionada de 8,4 s se reintentaba y acababa fallando en vez de devolverse, y eso salía como 503 intermitente.frontend/src/lib/api.ts: 10 s → 20 s, para que el navegador no cancele peticiones que el backend estaba a punto de contestar.El SLA de
docs/MVP.mdno cambia: se sigue midiendo, pero sobre resultados en vez de imponerlo descartándolos.Por qué importa ahora
Verificado ayer contra la app arrancada de verdad:
POST /api/generatetardó 9,8 s y devolvió 200 con las dos columnas, RAG y Passport firmado. Con la configuración que hay hoy enmain, esa misma petición habría muerto en el timeout de 8 s y el navegador la habría abortado a los 10.Verificación
🤖 Generated with Claude Code