feat: push tests down - test more at go level#569
Conversation
Signed-off-by: Giles Cope <gilescope@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request introduces several Earthfile test fixtures to validate various valid and invalid VERSION formats, along with corresponding unit tests in ast/version_test.go. The review feedback recommends capturing the loop variables locally within the test loops before invoking parallel subtests to ensure compatibility with older Go versions and satisfy linters.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
🎉 Are we earthbuild yet?Great progress! You've reduced "earthly" occurrences by 21 (0.39%) 📈 Overall Progress
📁 Changes by file type:
Keep up the great work migrating from Earthly to Earthbuild! 🚀 💡 Tips for finding more occurrencesRun locally to see detailed breakdown: ./.github/scripts/count-earthly.shNote that the goal is not to reach 0. |
Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
|
Had to rework it following the parsing changes. |
| r.ErrorContains(err, "earthfile: unable to open file") | ||
| } | ||
|
|
||
| func TestVersionFixtures(t *testing.T) { |
There was a problem hiding this comment.
These tests duplicate what the table tests in the TestParse already do.
Extend the table tests for missing scenarios in TestParse?
There was a problem hiding this comment.
I retained the previous ast tests in internal/earthfile/testdata to avoid making big changes and keep the change scope as small as possible when implementing parsing.
In reality, the tests in this dir have been transferred into table tests in TestParse(). Effectively, these tests are run twice: as unit tests and as higher-level tests +test-ast later.
This requires a better understanding of what earthly debug ast does. My current assumption is we can remove "+test-ast" and internal/earthfile/testdata, and leave only unit tests.
There was a problem hiding this comment.
Dug into what earthly debug ast actually does: it's a thin wrapper - earthfile.ParseFile + json.Marshal + print (cmd/earthly/subcmd/debug_cmds.go). No buildkit involvement beyond the test harness itself.
So +test-ast guards two things the unit tests don't:
- The JSON wire format.
TestParsecompares Go structs, so a struct-tag change would pass unit tests while silently changingdebug astoutput for any external consumer of that JSON. Worth checking whether anything (editor/LSP tooling?) still consumes it before declaring the format free to change. - A realistic corpus - it parses the real integration Earthfiles under
tests/. Downside: brittle goldens - two.ast.jsonfiles changed in this PR just because a dind image bump touched the source.earthfiles.
Proposal: replace rather than delete. A plain Go golden test in internal/earthfile that walks the existing *.ast.json files, parses the paired .earth, marshals, and diffs normalised JSON keeps both of the above at unit-test cost. Then +test-ast, tests+ast-test-input, and the binary round-trip can go (perhaps keeping one debug ast CLI smoke, same pattern as the slimmed tests/version).
On the duplication in this file: agreed - folded. The invalid VERSION cases now live in TestParseErrors, the valid spellings in a TestVersionVariants table with inline inputs asserting Version.Args, and internal/earthfile/testdata/ is deleted. (invalid-feature-flag-override.earth was already covered inline in features/features_test.go, so that fixture went too.) Will be in the next push.
These tests are currently earthbuild integration tests, this pushes them down to be go tests. Faster tests, less to go wrong.