Skip to content

ISS-10889: index every tracked file in a supported language; skip-list directories judge only untracked files - #34

Merged
wongk merged 1 commit into
mainfrom
worktree-iss-10889-tracked-files-indexed
Sep 21, 2026
Merged

wongk merged 1 commit into
mainfrom
worktree-iss-10889-tracked-files-indexed

Conversation

@wongk

@wongk wongk commented Sep 21, 2026

Copy link
Copy Markdown

Implements ISS-10889.

What changes

A file git tracks, in a supported language, is now always indexed. The built-in directory-name skip list (node_modules, .venv, dist, build, data, results, …) applies only to files git does not track: untracked-but-not-ignored files, and every file in the non-git fallback walk.

Before, the skip list matched any path segment at any depth, tracked or not, so real source under a directory that happened to be called build, data or results was never indexed and never found by search, relations, code_query or code_changes.

Why

Tracking a file is the owner's statement that it belongs to the project. The name list only earns its place where git gives no signal: a repo that forgot to ignore .venv or node_modules must not flood the index with untracked dependency files.

Rules after this change

Rule Tracked Untracked / non-git
Supported language + extension gate applies applies
.lemoncrow/.ignore applies applies
Skipped-directory list applies (list unchanged)
git-ignore — (git never ignores a tracked file) applies
1 MB file-size limit applies applies
Free-tier file cap applies applies

.lemoncrow/.ignore still binds tracked files: it is an explicit, user-written opt-out, and the CLI docs already describe it as the way to keep tracked data files out of the index.

Where

  • inclusion.py: new git_tracked(root, candidates) (literal pathspecs so app/[id]/page.tsx is not a glob; submodules included; chunked at 1000 paths per call; fails open to "untracked", which applies the full rule list). exclusion_rule takes a required tracked set and skips the directory rung for tracked paths. indexable_paths (the per-edit re-index) asks git once per batch.
  • graph.py: the whole-repo scan keeps which ls-files call listed each entry and applies the skip list only to the --others half.
  • coverage.py: code_coverage_check names rules with the same tracked set, so it agrees with both entry points.
  • engine.py: repo-scope symbol search no longer drops indexed hits by directory name (the index's admission rules are the only gate). Commit-history and external-scope hits keep the existing filter.
  • docs/reference/cli.md: describes the tracked/untracked split.

Verification

  • New tests: scan (tracked vs untracked under skip-list names), coverage + per-edit re-index agreement, .lemoncrow/.ignore still binding a tracked file, repo search returning a tracked build/ symbol, and git_tracked (literal brackets, submodules, at and past the chunk size, non-git fallback). Every new behavior test fails on main; the search test also fails with only the query-time filter reverted.
  • Differential check with the real scan on two repos (old vs new iter_source_files, nothing dropped in either):
    • symphony-alpha: +8 files, exactly the tracked source under build/, data/ and results/ that was missing.
    • lemoncrow: +1 file, benchmarks/codebench/results/.../graft.json.
  • Full fast suite passes. test_index_pool_does_not_fork_live_parent_state fails identically on main on this machine (spawn vs forkserver start method), so it is unrelated.

…ped-directory list judges only untracked files
@wongk
wongk merged commit 82cc941 into main Sep 21, 2026
9 checks passed
@wongk
wongk deleted the worktree-iss-10889-tracked-files-indexed branch September 21, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant