perf(intake): the funnel stops paying for work it already has - #1483
Open
yanekyuk wants to merge 1 commit into
Open
perf(intake): the funnel stops paying for work it already has#1483yanekyuk wants to merge 1 commit into
yanekyuk wants to merge 1 commit into
Conversation
Three cuts between a person's last answer at /i/new and the proposal they see: 5 model calls per signal become 3, and the blocking stretch 3 becomes 2. The profile bridge folds into the follow-up planner as a nullable field on each question, so personalization is one prompt section rather than a second serial call — and silence on every question is an ordinary success, not a failure to swallow. Merging put optionSchema in the schema twice, which made the converter emit a $ref Gemini rejects: the call was burning its retry and answering from the fallback model. One extra zod instance fixes it, and a spec guard fails on any $ref. The graph stops re-inferring what synthesis just wrote. A caller that supplies a stage's output now skips that stage: seeded inferredIntents route prep straight to verification, and the funnel seeds the synthesized signal. Chat and MCP still infer, because they feed raw utterances in. And authority stops capping the score of signals it was never given a profile to judge. The propose path attaches no profile on purpose; honouring that means leaving authority out of the minimum rather than guessing it. Claude-Session: https://claude.ai/code/session_01Ay9d5zypTYuunAUZX5Vpsw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three cuts on the live
/i/newpath, between a person's last answer and theproposal they see. Per signal: 5 model calls → 3, blocking 3 → 2.
1. The profile bridge folds into the follow-up planner
generateFollowUpsran the planner, then made a second serial call to append atmost one profile-derived option per question. Both calls already received the
same inputs. They are now one structured call whose question carries a nullable
profileBridgeOption.Both prompts' content survives the merge — the core rules, the bridge's
"what makes a bridge useful", and the scuba-divers-plus-pianist example. Only the
call count changed.
Two properties the two-call split enforced structurally are now prompt
constraints, so both are asserted:
shapes (null, a real bridge, a bridge that duplicates a core label) and asserts
2–3 answer-grounded options survive
normalizeOption/normalizeFollowUpQuestionon every question, never displaced and never replaced by the fallback.
on purpose. There is no separate failure to swallow now, so the field is
genuinely nullable: a response with every bridge
nullreturns the questionsnormally, in one call, with no retry. A bridge offered when no brief was
supplied is dropped in code rather than trusted.
The
$reftrap this uncoveredMerging made
optionSchemaappear twice in one schema, so the JSON Schemaconverter emitted the second occurrence as a
$refintodefinitions. Geminirejects that document. The merged call was failing twice on
gemini-2.5-flash, then answering from thegpt-4o-minifallback — 1.5s became6.5s, on a model nobody chose, with no error anywhere. Captured request bodies:
The fix is one extra zod instance for the bridge option so both shapes inline.
A spec guard converts the schema with LangChain's own
toJsonSchemaand fails onany
$refordefinitions— verified to fail when the instance is shared again.2. The graph stops re-inferring what synthesis just wrote
invokeIntentGraphProductionranprep → inference → verification. Its input isthe output of
synthesize: one clean, self-contained, first-person signal. Thereis no messy text left to pull candidates out of, and the call passes
userProfile: '', so inference's other half — enrichment — was switched offanyway.
Chosen shape: supplying a stage's output skips that stage.
shouldRunInferenceroutes straight to
verificationwhenstate.inferredIntentsis alreadynon-empty on entry, and the funnel seeds it with the synthesized signal. No new
operation mode. Seeded state does not collide with
prep:prepwrites onlyactiveIntents/activeIntentIds/trace, and theinferredIntentsreduceris last-write-wins on a channel
prepnever returns.This generalises the way the later cut wants — pre-verified intents can skip to
the reconciler by the same rule.
Chat and MCP are untouched: they feed raw utterances in and still route through
inference.
Known loss, confirmed in code: tombstone extraction disappears from this path.
It was never reachable here —
runSynthesisreads onlydescription,score,and
verificationfrom the result, andproposemode exits atrouteAfterVerificationbefore the reconciler, which is the only thing that actson a tombstone. So nothing that ran is lost; the capability simply stops being
theoretically present.
3. Authority stops scoring signals against a profile that was never supplied
Not a speed fix.
authorityasks whether the speaker's profile supports thespeech act. The funnel passes
userProfile: '', and the graph tookscore = min(authority, sincerity, clarity)— so every funnel-created signal hadits confidence capped by a number the verifier guessed from nothing.
combineFelicityScoresnow leavesauthorityout of the minimum when no profilewas supplied. This honours the comment above
invokeIntentGraphProductionratherthan reversing it: an intent derives only from the person's answers, so it should
not be scored against a profile they were never asked for. The alternative —
passing the global context paragraph
getGlobalContextProductionalreadyfetches — changes what a signal is derived from, which is a product decision, not
a perf one.
This changes stored confidence values. New funnel signals get a
combinedScoreonintent_proposals.analysisthat is no longer capped byauthority; it can only rise. Existing rows keep their old numbers — there is no
backfill. Nothing gates on this score (
isVaguereadsclaritydirectly), so nosignal that used to pass now fails or vice versa; only the recorded number
changes. The raw
felicity_scores.authorityis still stored untouched, and theverifier still emits
SKILL_MISMATCHwhen it is low — both left alonedeliberately, so the verdict stays a faithful record of what the model said.
Measured
Live
gemini-2.5-flash, this branch vs5b1878732, five interleaved sampleseach, medians (same fixture: a two-round interview plus a profile brief):
Baseline samples: 2674/3025/2866/2656/2612 and 5686/4610/5081/4312/4748.
Branch samples: 1684/1639/1877/1736/1616 and 3629/4713/3797/3399/3515.
That is the claimed 5 → 3 calls and blocking 3 → 2. The per-call saving is
smaller than a whole call on the post-answer stretch because verification is the
slowest of the three, and it stays.
Testing
Baselines re-derived on this worktree, not quoted from anywhere:
bun run test: 2388 pass / 0 fail / 211 files before,2399 pass / 0 fail / 212 files after. The runner excludes six live-model
specs; none of them cover the intake orchestrator or the verifier, so nothing
relevant is hidden by that exclusion. The two touched specs were also run in
isolation.
bun run test: 16 fail before, 16 after, 1801 pass bothruns. The named set is identical apart from one timing-sensitive rate-limit
guard (
throws RateLimiterError past the limit, 5001ms in the baseline) thatflipped to passing on a later run. That set is the stable pre-existing dev
failure set — MCP owner-proof, CLI credentials, ToolController, conversation
sessions, rate-limit guard, legacy-negotiation archive — and nothing in it is
reachable from this diff.
negotiation-presenceandchat-sidebar-unread.Neither file, nor the module under test, imports
@indexnetwork/protocol;they are the pre-existing web set and cannot be reached by this diff.
bun run lint(0 errors),typecheck,typecheck:specsall clean.New coverage:
intent.graph.profile-blind.spec.ts(seeded state skips inference,unseeded still infers,
deletestill wins over the seed, and the authorityexemption at both the helper and the node), plus the bridge-property and
$ref-guard tests inintake.orchestrator.spec.ts.packages/protocol23.6.4 → 23.7.0; neither open PR (#1400, #1072) touchespackages/protocol/package.json.Not in this PR
The other four cuts — merge inference+verification, batch the network scoring,
have the verifier emit the clarifier's repair, cap the verifier's reasoning
tokens — are deliberately separate branches and are untouched here.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ay9d5zypTYuunAUZX5Vpsw