Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
The e2e/cli-smoke-test.sh check added with #698 assumes a gibberish query must have no results:
drive9 fs grep --json "zzz_nonexistent_marker_xyz" /
Run the CLI smoke against an auto-embedding deployment whose embedding model considers that query similar enough to existing files. The scheduled reproduction was:
- run:
20260711T120000Z
- endpoint:
https://alicloud-ap-southeast-1.drive9.ai
- tenant:
d8e5b676-9f75-443a-8499-907939d69603
- branch/commit:
smk/issue-676-e2e-tidbcloud-native / 8aef6cf
- local log:
/home/ubuntu/.cache/drive9-smoke/scheduled/alicloud-ap-southeast-1/20260711T120000Z/smoke-all.log
The same batch passed this assertion on the three AWS endpoints.
2. What did you expect to see? (Required)
The smoke check expects an empty JSON array so it can validate the --json empty-result encoding fixed by #698:
The empty-result case should be deterministic across embedding providers and regions.
3. What did you see instead (Required)
Alicloud returned 16 unrelated results, so CLI smoke ended 96 passed, 1 failed even though JSON serialization and the rest of the CLI behavior worked. Representative output shape:
[
{"path":"/cli-...-batch/file-3.txt","score":0.01639344262295082},
{"path":"/cli-...-batch/file-6.txt","score":0.016129032258064516}
]Clinic correlation at 2026-07-11T12:45:32Z recorded a successful grep_ok event with query_len=26, results=16, and trace df5224af93ed7d08c1811e2cd875989e. There was no HTTP or runtime error.
Root-cause assessment
The assertion is not deterministic in semantic mode:
pkg/backend/dat9.go merges any FTS or vector rows returned by the ranking paths.pkg/datastore/search.go keeps vector results above vectorScoreThreshold = 0.3.- RRF then emits rank scores around
0.016, which are the values visible in the CLI output.
- Whether a gibberish string has vector matches depends on the embedding provider/model and current indexed content.
The CLI correctly emitted a JSON array containing the server results; the false failure is in the smoke test's method for forcing an empty result.
Impact
- Multi-region scheduled smoke is marked
FAIL_UNKNOWN on a healthy CLI JSON path.
- Provider/model variance can make the check flaky across deployments.
- The false failure can obscure unrelated regressions in the same batch.
Suggested fix direction
Keep the positive JSON-array/result checks unchanged, but make the empty-result check use a guaranteed empty path prefix or scope that the test never creates, rather than relying on a semantically irrelevant query. For example, query a unique /cli-${TS}-empty-scope/ path and assert the JSON output is [].
Semantic relevance thresholds can be evaluated separately; they are not required to make this JSON-format regression test deterministic.
Validation plan
- Run
bash -n e2e/cli-smoke-test.sh.
- Run the focused CLI smoke against both an auto-embedding deployment and an application-embedding deployment.
- Verify the positive
grep --json check still finds the expected batch file.
- Verify the guaranteed-empty-scope check returns exactly
[] in all regions.
- Run the 4-region scheduled smoke batch and confirm this assertion no longer varies by provider/model.
4. What is your drive9 version? (Required)
branch: smk/issue-676-e2e-tidbcloud-native
commit: 8aef6cf60de088c5b98a602b25e46b994e2a114d
CLI_SOURCE: build
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
The
e2e/cli-smoke-test.shcheck added with #698 assumes a gibberish query must have no results:drive9 fs grep --json "zzz_nonexistent_marker_xyz" /Run the CLI smoke against an auto-embedding deployment whose embedding model considers that query similar enough to existing files. The scheduled reproduction was:
20260711T120000Zhttps://alicloud-ap-southeast-1.drive9.aid8e5b676-9f75-443a-8499-907939d69603smk/issue-676-e2e-tidbcloud-native/8aef6cf/home/ubuntu/.cache/drive9-smoke/scheduled/alicloud-ap-southeast-1/20260711T120000Z/smoke-all.logThe same batch passed this assertion on the three AWS endpoints.
2. What did you expect to see? (Required)
The smoke check expects an empty JSON array so it can validate the
--jsonempty-result encoding fixed by #698:The empty-result case should be deterministic across embedding providers and regions.
3. What did you see instead (Required)
Alicloud returned 16 unrelated results, so CLI smoke ended
96 passed, 1 failedeven though JSON serialization and the rest of the CLI behavior worked. Representative output shape:[ {"path":"/cli-...-batch/file-3.txt","score":0.01639344262295082}, {"path":"/cli-...-batch/file-6.txt","score":0.016129032258064516} ]Clinic correlation at
2026-07-11T12:45:32Zrecorded a successfulgrep_okevent withquery_len=26,results=16, and tracedf5224af93ed7d08c1811e2cd875989e. There was no HTTP or runtime error.Root-cause assessment
The assertion is not deterministic in semantic mode:
pkg/backend/dat9.gomerges any FTS or vector rows returned by the ranking paths.pkg/datastore/search.gokeeps vector results abovevectorScoreThreshold = 0.3.0.016, which are the values visible in the CLI output.The CLI correctly emitted a JSON array containing the server results; the false failure is in the smoke test's method for forcing an empty result.
Impact
FAIL_UNKNOWNon a healthy CLI JSON path.Suggested fix direction
Keep the positive JSON-array/result checks unchanged, but make the empty-result check use a guaranteed empty path prefix or scope that the test never creates, rather than relying on a semantically irrelevant query. For example, query a unique
/cli-${TS}-empty-scope/path and assert the JSON output is[].Semantic relevance thresholds can be evaluated separately; they are not required to make this JSON-format regression test deterministic.
Validation plan
bash -n e2e/cli-smoke-test.sh.grep --jsoncheck still finds the expected batch file.[]in all regions.4. What is your drive9 version? (Required)