config.yaml

# Set your GitHub org(s), user(s), or repo(s) under `connections` below, then
# `make deploy`. This template ships with nothing selected — the indexer fails
# fast ("connection selects nothing") until you uncomment and fill one in.
version: 1

# How many repos the job indexes at once. Default 4, max 8, min 1.
# This is a DISK bound, not a CPU one: each worker holds its 500 MB tarball on
# disk. That is the only on-disk artifact — the archive is streamed in memory and
# never extracted — so budget 0.5 GB per worker: 2 GB at the default 4, 4 GB at
# the ceiling of 8.
# Symbol extraction does not parallelise across THREADS (measured 0.95x on 4
# threads — the tree walk is GIL-serialized), which is why extraction now runs
# in its own shared process pool instead — see extract_processes below. Raising
# this knob buys disk-bound repo fan-out, not extraction throughput.
# With semantic indexing enabled this is clamped to 4 (issue #109 raised it from
# 2, after re-deriving the memory model and confirming empirically against the
# live dev job at N=4: peak self+children RSS landed at ~83% of the 0.7*container
# memory budget, comfortably under and with more margin than N=2's own ~90%) — a
# MEMORY bound, since embedding materialises a whole repo's chunks (structural
# ~32 KB/chunk, resident ~40.1 KB/chunk measured; see effective_workers'
# docstring and docs/perf/issue-109-measurements.md for the full derivation).
# index_concurrency: 4

# How many worker PROCESSES the job uses to extract symbols/edges (issue #108).
# Independent of index_concurrency above: this is a CPU knob (a shared, spawn-
# based process pool decoupled from the per-repo worker threads), not a disk or
# memory one. Default (unset) derives from the runtime's affinity/cgroup-aware
# CPU count, clamped to 8. Setting this to 1 restores fully serial, in-process
# extraction and spawns no process pool at all — the rollback switch if the
# pool ever misbehaves in this runtime.
# extract_processes: 4

connections:
  - type: github
    # orgs / users / repos are UNIONED, then deduplicated by canonical org/repo.
    # users:
    #   - your-github-username

    # orgs:
    #   - acme
    #   - acme-labs

    # Explicitly listed repos are indexed VERBATIM — no enumeration API call,
    # and `exclude` below does NOT apply to them. Explicit always wins.
    # repos:
    #   - otherorg/specific-repo

    # Extra branches to index PER REPO, on top of the default branch (always
    # included regardless of match). Glob patterns (fnmatchcase, exact-name
    # match, NOT a regex) against each repo's branch list. Empty (the
    # default) means default-branch-only — no behavior change for an
    # unmodified config, and the GitHub branches API is not even called.
    # Soft-capped at 20 branches per repo (default-first, then alphabetical);
    # see docs/runbooks/multi-branch.md.
    # branches:
    #   - "main"
    #   - "release/*"

    # `exclude` filters ONLY repos discovered via `orgs` / `users`.
    # exclude:
    #   forks: true       # default true — forks duplicate their upstream
    #   archived: true    # default true — re-downloaded every run, never change
    #   repos:            # fnmatch globs against canonical org/repo
    #     - "acme/*-deprecated"
    #     - "acme/test-*"
    #   size_mb: 500      # GitHub reports size in KB; this is MB. Default: no cap.

# Per-repo override of the semantic chunk-embedding ceiling — a MAP, keyed by
# repo. This is the SPOT knob: it moves the cap for the repos named here only,
# leaving the GLOBAL default alone. To move the global default for the whole job,
# use `semantic.max_chunks_per_repo` in the `semantic:` block below — NOT this map
# (the two are similarly named on purpose: same dimension, different scope). A repo
# whose real chunk count exceeds its effective cap degrades to indexing its core
# corpus WITHOUT chunks (semantic is additive, never fails the branch); list it
# here to raise (or lower) its own cap without touching everyone else's.
# Keys are canonicalised the same way as `repos:` above (URL/SSH/bare all work,
# matched case-insensitively); an override naming a repo the corpus never
# resolves logs a WARNING (typo guard) and is otherwise a no-op. A repo named here
# beats the global from any source (config.yaml or env): the job resolves the
# effective cap as `per-repo override OR global`.
#
# Mind the memory math before raising one: buffered vectors are ~32 KB/chunk
# structural (dim=1024, Python float-list storage) but ~40.1 KB/chunk RESIDENT
# (measured, issue #109 — includes pymalloc overhead/fragmentation; use this
# figure for headroom arithmetic), so 8000 ≈ 313 MiB resident for the duration of
# that repo's write. With semantic on, at most 4 workers run concurrently
# (indexer/repo_config.py's effective_workers clamp, raised from 2 by #109), so
# a large override multiplies straight into the job container's peak memory —
# e.g. two repos overridden to 20000 concurrently is ≈1.6 GB just in vectors, on
# top of the base per-worker cost. A repo that legitimately needs far more than
# that needs the temp-table staging path (follow-up), not a bigger override.
#
# The derived per-worker chunk-cap ceiling, from a full container-memory model
# (issue #109; docs/perf/issue-109-measurements.md §12 — pinned at N=2 there
# only to break a circularity in solving for C from a formula whose dominant
# term IS C, not a claim about the adopted concurrency): ≈73,300 chunks at
# N=2, ≈36,700 at the shipped N=4 (both halve/double with N). The current
# global default of 8000 uses well under a quarter of either budget, so it is
# NOT the binding constraint and was left unchanged.
# semantic_max_chunks_per_repo:
#   "acme/huge-monorepo": 20000

# Semantic-indexing behavior for THIS JOB. config.yaml is the job's only reachable
# config surface: the serverless job sets no CODE_SEARCH_* env vars (nothing in
# resources/job.yml does), so every semantic Settings default in app/config.py is
# effectively hard-coded for the job unless overridden here. Precedence is
# config.yaml > CODE_SEARCH_* env > code default — a field set here wins; a field
# left out falls through to the env value / the app/config.py default. The env vars
# remain the MCP-server / webui surface (separate deploy, separate environment);
# this block does not reach them. An absent `semantic:` block is a pure no-op.
#
# `max_chunks_per_repo` here is a single INT: the GLOBAL ceiling for every repo
# (the default is 8000). It is NOT the per-repo `semantic_max_chunks_per_repo` MAP
# above — that spot-overrides individual repos and still wins over this global. Use
# this to raise the floor everyone inherits; use the map for the outliers. The same
# ~32 KB structural / ~40.1 KB resident per-chunk memory math and 4-worker clamp
# above apply here, magnified: raising the global lifts the buffer cost for EVERY
# concurrently-indexing repo at once.
#
# `enabled: false` makes the job a true semantic no-op (no embedder built, no
# chunking, the 4-worker memory clamp not applied) even if the env says enabled —
# the fastest way to turn semantic off for the job alone.
#
# `embedding_concurrency` (#107) is in-flight embedding requests PER WORKER, sent
# via a ThreadPoolExecutor that preserves submission order — vectors always come
# back in the order their texts were sent, regardless of which request finishes
# first. Total in-flight gateway requests for the job is workers x concurrency:
# 4 x 4 = 16 at this default, 4 x 8 = 32 at the max of 8 — the latter now EXCEEDS
# the SDK's 20-connection pool (issue #109 raised workers from 2 to 4; this
# combination was not possible before). Lower embedding_concurrency if raising it
# alongside a near-ceiling index_concurrency. Set embedding_concurrency to 1 to
# restore fully serial embedding (no thread pool spawned at all) if you need to
# roll back.
# semantic:
#   enabled: true
#   max_chunks_per_repo: 8000
#   embedding_endpoint: /ai-gateway/mlflow/v1/embeddings
#   embedding_model: system.ai.gte-large-en
#   embedding_batch_size: 64
#   embedding_timeout_s: 20.0
#   embedding_concurrency: 4