docs(sdk): clarify hook value nullability in tailordb examples#1167
Draft
dqn wants to merge 1 commit into
Draft
Conversation
The Hook type signature on the configure layer types value as T | null for both create and update hooks, but every example in the docs sidesteps value or simply passes it through. AI codegen tools (and humans) regularly miss this nullability and write value.toLowerCase() / value.length on a possibly null input, producing TypeErrors at runtime. - Tighten the value bullet to state the T | null type explicitly and call out that update hooks do NOT auto-inject the existing value. - Replace the trivial passthrough example with a transform that uses the documented value ?? "" fallback pattern. - Add a sibling example that derives from user, so the prior intent of showing both forms is preserved.
|
⚡ pkg.pr.new@tailor-platform/sdk@tailor-platform/create-sdk
|
Code Metrics Report (packages/sdk)
Details | | main (6c3a2cb) | #1167 (903307e) | +/- |
|--------------------|----------------|-----------------|------|
| Coverage | 61.9% | 61.9% | 0.0% |
| Files | 363 | 363 | 0 |
| Lines | 12636 | 12636 | 0 |
| Covered | 7827 | 7827 | 0 |
| Code to Test Ratio | 1:0.4 | 1:0.4 | 0.0 |
| Code | 82628 | 82628 | 0 |
| Test | 34422 | 34422 | 0 |SDK Configure Bundle Size
Runtime Performance
Type Performance (instantiations)
Reported by octocov |
dqn
added a commit
that referenced
this pull request
May 13, 2026
…ate cascade) First problem of Phase 2.5 harder tier. Tests whether agents internalize the "update hook value is T | null" precondition (PR #1167 documented it for create; this verifies the same friction angle for update). - `shared/helpers.ts` listProblems regex extended to `^(\d{3}|m\d+|h\d+)-` so the harness discovers h-tier problems alongside legacy and Phase 2. - Reference solution validates with `pnpm challenge --problem h01 --use-solution` (3/3 stages pass, 4/4 tests).
dqn
added a commit
that referenced
this pull request
May 14, 2026
…(Phase 5a) First real AI solve cycle (Phase 4) falsified the LLM-as-judge ambition: judge fired on 1/25 problems, output was a false signal, both true SDK improvements (PRs #1167, #1168) came from analyze --diff + iter variance instead. Phase 5a retires the unused machinery; profile-diff + iter variance become the primary signal in subsequent phases. # What's removed - core/judge.ts (486 lines) + core/judge.test.ts (446 lines) - runJudgePostProcessing / computeJudgeDiff / ImprovementCandidate in cli.ts - ProblemResult.judge, Analytics.affordanceDistribution / improvementCandidatesPath - DiffReport.affordanceDelta + showAffordanceDelta in analyze.ts - "Top affordances" line in formatReportTable - LLM-as-judge / Affordance Taxonomy 12-label sections in README + SKILL.md - judge / improvement-candidates references in skill docs # Renamed - meta.json: hypothesizedAffordance -> designNote (37 problems) - ProblemMeta type: same # Verification - 291 tests pass (-31 from judge.test.ts retirement, as expected) - challenge:verify-solution: 37/37 PASS - typecheck / lint / format clean - core/ production: 7,370 -> 6,306 lines (-1,064)
dqn
added a commit
that referenced
this pull request
May 14, 2026
…Phase 5b) Replace the retired judge module with two operator-facing signals: 1. 5-bucket Read-target classifier (sdk-dts / sdk-package-src / sdk-docs / problem-files / other) populated per iteration into TraceMetrics. Legacy readSdkDts and readDocs become derived views of this map. 2. emitIterDiff post-processor that runs `git diff --no-index` between the first failing and first passing iteration's work tree for any problem with iterations.passRate in (0, 1), writing to <runArtifactRoot>/iter-diff/<problemId>.diff. Direct human-readable answer to "what flipped the outcome", no LLM needed. Validated against existing m05 (passedByIteration [F,F,T]) and m18 ([F,T,T]) Phase 4 trace data: - m05: passing iter consults sdk-docs 1 less time than failing iters (delta=-1.0). Consistent with the docs-misleading SDK PR #1167. - m18: passing iters consult sdk-dts +2 and problem-files +2 more than the failing iter. The discriminating bucket is not docs-vs-not-docs but type/problem-file engagement depth. Phase 5c will render this as a multi-bucket delta vector rather than gating on a single bucket. # Verification - 305 tests pass (+14 from 291) - challenge:verify-solution: 37/37 PASS - typecheck / lint / format clean - +735 / -102 lines across 7 core files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only fix surfaced by the llm-challenge benchmark: TailorDB hook examples were silently bypassing the
valueargument, leading agents that read docs to omit null fallbacks even though the type definesvalueasT | null.Why
packages/sdk/src/configure/services/tailordb/types.ts:9-20defines hook inputs asHookFn<TReturn | null, TData, TReturn>. The runtime injectsnullforvaluewhen the user omits the field on create, or on update when the field is absent from the mutation input.Docs prose mentioned this nullability on one bullet line, but every code example sidestepped
value:create: ({ user }) => user.id(ignored value)update: ({ value }) => value(passthrough, doesn't transform)create/update: ({ data }) => ...(useddata, notvalue)No example demonstrated
value ?? ""or any explicit null handling. Agents that read docs internalize the bypass pattern and writevalue.toLowerCase(), which fails at runtime (and at typecheck ifstrictNullChecksis on).Evidence
A/B experiment via
pnpm challenge:experimentonm05-db-type-hooks-create(a problem that asks for a lowercase-normalizing create hook):All three iterations on the candidate converged on the reference solution shape:
Baseline iterations wrote two
value.toLowerCase()(typecheck fail) and onevalue!.toLowerCase()(non-null assertion workaround).What changed
valuebullet in the hook input contract to explicitly noteT | nulland thevalue ?? ""idiom.valuewith the null fallback ((value ?? "").toLowerCase()).user" pattern visible while clarifying that it explicitly ignoresvalue.No source code or type changes. The types already encoded the contract; the examples now reinforce what the types say.
Notes
Validated only against
m05because that was the problem whose full-package iteration passRate diverged from types-only in the benchmark. A follow-up sweep should grep for the same anti-pattern in other SDK docs (resolver / executor argument handling) that transform nullable inputs.llm-challenge benchmark harness lives in PR #1148.