Exploration describes the data and asks what it cannot tell

[WaylandYang]

Closes #570. Stacked on #567 (it changes explore_mappings, which #503 already rewrote there); retarget to dev once #567 lands.

Chat without conventions answers 2 of 18 questions on the wide corpus; with six conventions written as prose it answers 14 (#520). The conventions had nowhere to live. This gives them a home and lets exploration write the half it can.

What changes

• knowledge_bases gains three columns (migration 0047): data_description and data_questions, written by exploration every run; data_conventions, written by a person through PATCH /kbs/{id} and never touched by exploration. Two writers, two paths (kbs::set_data_description vs kbs::update); mixing them in one field would let the next run overwrite the answers.
• Exploration makes one more model call before proposing: a description that states only what the schema and comments say (grain, keys, units, code meanings, time axes, look-alike columns) and a list of questions for what the schema leaves open. It is told not to guess conventions — a guessed convention in the prompt is followed as confidently as a right one. A failed description does not fail the run.
• Chat reads the description and the conventions into the system prompt every turn. Exploration reads the conventions into its own prompt, so a proposal can carry the filter and the unit conversion.
• ask.mjs / mappings.mjs --conventions write the truth file’s conventions through the PATCH.

Measured (wide corpus, fresh bases, #570 base)

What exploration wrote for dwd_ord_dtl, unprompted: grain (“one row is one line item within an order”), the keys, “money amounts … are in cents”, every status and channel code, six time axes, “amt_total is pre-discount, amt_pay is paid amount”, “price_old … no longer written after 2023 migration”. And eight questions, six of which are the truth file’s conventions verbatim in spirit: which ord_st count, which pay_st count, is is_test excluded, amt_pay or amt_total for revenue, are refunds already subtracted.

	exploration covered	chat right
nothing	0/18	2/18
generated description only	0/18	2/18
description + the six conventions in the prompt	3/18	11/18 (13 by SQL)
conventions + 18 definitions as a prose page, retrieval only (#520)	—	14/18
27 confirmed definitions in the prompt (#520)	—	17/18

Two things the table says. The description alone moves nothing — the model already gets grain and units from the schema document; what it lacked was never structure. Conventions move exploration for the first time: 0 to 3 definitions covered, and the five wrong proposals now all carry ord_st IN (2,3,4) AND is_test != 1 and differ from the truth by a column choice (amt_total for amt_pay) or a filter variant (ord_st where money uses pay_st) — the same ambiguity the prose experiment found, now visible in proposals instead of answers. Six general rules in the prompt are worth 11; the prose page that also spelled out 18 definitions was worth 14; structured definitions 17. The order of work follows.

Verified

• Unit: parse_description (fenced JSON, blank questions dropped, empty description refused).
• DB test: description and conventions do not overwrite each other across a re-run and a PATCH; a PATCH that omits conventions leaves them.
• The bench rows above, on a fresh base migrated by this branch.

Does not touch Mappings.tsx (#552). Touches chat.rs in the prompt-assembly block only (#548 rewrites the loop; the injection is twelve lines before the semantic-layer block).

🤖 Generated with Claude Code