Index a TypeScript file's own documentation

[cruhl]

• A leading /** … */ block that no declaration owns is a file's own documentation, and it was being dropped rather than indexed.
• It now emits as a file chunk, keyed ts/file, which is for TypeScript and JavaScript what the package chunk is for Go.
• [reindex]: existing indexes gain these chunks on the next run.

The gap

Doc association is by adjacency: a comment group ending directly above a declaration documents that declaration. A file whose first statement is an import, whose only export is export {}, or which is documentation and nothing else, leaves its leading block with no declaration below it to own. The walk consumed the block and dropped it.

That is the @fileoverview / @module convention, and it is also where a design note above the imports lives. So a file written that way contributed only signatures to the index, and the one piece of prose saying what the file is for could not be retrieved at all. Go has not had this gap: emitPackage gives a package's doc its own chunk.

Before and after

Two files, one documenting itself above its imports and one that is documentation with a bare export {} under it:

$ sqlite3 .semantic/index.db 'select chunk_key from chunks order by chunk_key'
ts/func/put

Both blocks are absent. After:

ts/file
ts/file
ts/func/put

Which is the difference between finding a file and not:

$ semantic search "how do retries avoid a thundering herd" --limit 2
barrel.ts:1  0.74  file
    Retries are exponential and jittered, so a thundering herd spreads out
    instead of arriving together.

Before the change the same query returned an unrelated function signature at 0.32, because the sentence it was looking for was not in the index.

On a 950-file TypeScript codebase that documents its modules this way, 46 of 59 leading doc blocks were missing from the index. All 59 are present now.

Decisions worth reviewing

• The variant is file, not module. VariantModule already names a declared module or namespace in Rust, Ruby, C#, C++, and Scala, and conflating the two would break a --lang-style filter on meaning rather than on syntax.
• A shebang and a directive prologue are skipped. "use client" above the block is the common React case, and #!/usr/bin/env node is the common CLI one. A directive below the block also leaves it unowned, so both orders emit.
• A block that documents a declaration is untouched. documentsNext decides by exclusion — import, a declaration-less export, a second comment, a directive, end of file — so every existing association keeps its current behavior.
• One file chunk per file. The key is fixed, so a second leading block cannot collide with it.

Proof

• go test ./... passes; go vet ./... is clean.
• Four new tests in pkg/chunk/typescript_test.go: a block above imports emits; the declaration under it still chunks; six unowned shapes emit (export {}, nothing at all, a second block, a directive above, a directive below, a shebang, export *); a block that owns a declaration does not emit; a // line comment does not emit.