Skip to content

A table.include.list entry that matches no table is raised, with the corrected entry when the mistake is a missing schema or a shell glob - #13

Merged
avison9 merged 1 commit into
mainfrom
feat/captured-table-missing
Sep 25, 2026
Merged

avison9 merged 1 commit into
mainfrom
feat/captured-table-missing

Conversation

@avison9

@avison9 avison9 commented Sep 25, 2026

Copy link
Copy Markdown
Owner

What it changes

A new rule, captured-table-missing (warning): each table.include.list entry (or the old table.whitelist spelling) is checked against the tables in the migrations, and an entry that matches none is raised on the connector file.

Two shapes get the corrected entry as the fix, because they are how people get it wrong in practice and both follow from Debezium's documented matching (each entry is a regular expression matched against the whole schema.table name, never a substring):

warning captured-table-missing include-table-no-schema/connector.json
  ipaddrs in table.include.list matches no table in the source schema
  Debezium matches each entry against the whole schema.table name
  fix: write myschema\.ipaddrs, or remove it
warning captured-table-missing include-table-glob/connector.json
  public.bg_* in table.include.list matches no table in the source schema
  Debezium reads each entry as a regular expression, where * repeats the character before it
  fix: write public\.bg_.* to capture public.bg_items and public.bg_orders, or remove it

Anything else gets the plain fix: remove it, or check the spelling against the migrations.

Why a warning, not an error

Debezium itself only logs a warning here and keeps running. debezium/dbz#872 asks it to fail; a maintainer answered that a table may be created later, which is a legitimate case a linter reading migrations cannot rule out. When a sink actually reads the table, the existing sink-table-not-captured error still fails the pull request, as all three new corpus entries show.

What was rejected

  • Checking exclude-list entries: one that matches nothing excludes nothing, the same reasoning as captured-column-missing.
  • Reading an entry that already contains .* as a glob: it was written as a regular expression, and rewriting it would be a guess.
  • Suggesting a nearest name for plain typos: an edit-distance guess could point at the wrong table; the plain fix is honest.

Verified

  • Corpus first: include-table-no-schema (reduced from Stack Overflow 74103659, 6.9k views), include-table-glob (Stack Overflow 51345636, 5.9k views), include-table-typo. The expectations were written by hand and failed before the code; they pass as written. Every existing corpus entry is unchanged (no -update).
  • gofmt -l . clean, go vet ./..., go build ./..., go test ./... pass.
  • RefuseRadar at 1986043: 0 error(s), 0 warning(s), 159 info, unchanged, so no false positive on a real pipeline.
  • A probe on a throwaway copy of RefuseRadar's connector with public.reports changed to reports: the rule raises reports ... matches no table, suggests public\.reports, and the existing error shows the consequence in the real sink.

Docs

README: the rule in the rules table (marked next until it is released), and the symptom rows for table.include.list not working and a table-list typo now name it. corpus/README: the three entries.

Next for you

Merge when happy. It ships in the next release tag.

…corrected entry when the mistake is a missing schema or a shell glob

Debezium logs a warning when a table.include.list entry matches nothing
and keeps running, so the topic is simply never produced. People ask it
to fail instead (debezium/dbz#872); a maintainer answered that a table
may be created later. cdclint raises it as a warning for the same
reason: when a sink actually reads the table, sink-table-not-captured
already fails the pull request.

The two shapes behind the most-viewed questions get a pointed fix,
because both follow from Debezium's documented matching (each entry is
a regular expression matched against the whole schema.table name):
an entry without its schema (Stack Overflow 74103659: ipaddrs for
myschema.ipaddrs) gets "write myschema\.ipaddrs", and a shell glob
(Stack Overflow 51345636: public.bg_* names no table as a regex) gets
"write public\.bg_.* to capture ..." with the tables it would capture.
An entry that already contains .* is read as the regex it is.

Exclude-list entries are not checked: one that matches nothing excludes
nothing, the same reasoning as captured-column-missing.

Three corpus entries, written before the code: include-table-no-schema,
include-table-glob, include-table-typo.
@avison9
avison9 force-pushed the feat/captured-table-missing branch from 610bec0 to cf0363a Compare September 25, 2026 22:24
@avison9
avison9 merged commit 2870854 into main Sep 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant