Skip to content

feat: push tests down - test more at go level#569

Open
gilescope wants to merge 6 commits into
mainfrom
giles-ast-version-tests
Open

feat: push tests down - test more at go level#569
gilescope wants to merge 6 commits into
mainfrom
giles-ast-version-tests

Conversation

@gilescope

@gilescope gilescope commented Jun 14, 2026

Copy link
Copy Markdown

These tests are currently earthbuild integration tests, this pushes them down to be go tests. Faster tests, less to go wrong.

Signed-off-by: Giles Cope <gilescope@gmail.com>
@gilescope gilescope requested a review from a team as a code owner June 14, 2026 12:09
@gilescope gilescope requested review from kmannislands and removed request for a team June 14, 2026 12:09

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several Earthfile test fixtures to validate various valid and invalid VERSION formats, along with corresponding unit tests in ast/version_test.go. The review feedback recommends capturing the loop variables locally within the test loops before invoking parallel subtests to ensure compatibility with older Go versions and satisfy linters.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread ast/version_test.go Outdated
Comment thread ast/version_test.go Outdated
@github-actions

github-actions Bot commented Jun 14, 2026

Copy link
Copy Markdown

🎉 Are we earthbuild yet?

Great progress! You've reduced "earthly" occurrences by 21 (0.39%)

📈 Overall Progress

Branch Total Count
main 5408
This PR 5387
Difference -21 (0.39%)

📁 Changes by file type:

File Type Change
Go files (.go) ➖ No change
Documentation (.md) ➖ No change
Earthfiles ✅ -21

Keep up the great work migrating from Earthly to Earthbuild! 🚀

💡 Tips for finding more occurrences

Run locally to see detailed breakdown:

./.github/scripts/count-earthly.sh

Note that the goal is not to reach 0.
There is anticipated to be at least some occurences of earthly in the source code due to backwards compatibility with config files and language constructs.

Signed-off-by: Giles Cope <gilescope@gmail.com>
@gilescope gilescope mentioned this pull request Jun 14, 2026
10 tasks
Comment thread ast/testdata/version/comment-and-whitespace-before-version.earth Outdated
Signed-off-by: Giles Cope <gilescope@gmail.com>
@gilescope gilescope requested a review from janishorsts July 1, 2026 07:34
Signed-off-by: Giles Cope <gilescope@gmail.com>
@gilescope

Copy link
Copy Markdown
Author

Had to rework it following the parsing changes.

r.ErrorContains(err, "earthfile: unable to open file")
}

func TestVersionFixtures(t *testing.T) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests duplicate what the table tests in the TestParse already do.

Extend the table tests for missing scenarios in TestParse?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I retained the previous ast tests in internal/earthfile/testdata to avoid making big changes and keep the change scope as small as possible when implementing parsing.

In reality, the tests in this dir have been transferred into table tests in TestParse(). Effectively, these tests are run twice: as unit tests and as higher-level tests +test-ast later.

This requires a better understanding of what earthly debug ast does. My current assumption is we can remove "+test-ast" and internal/earthfile/testdata, and leave only unit tests.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dug into what earthly debug ast actually does: it's a thin wrapper - earthfile.ParseFile + json.Marshal + print (cmd/earthly/subcmd/debug_cmds.go). No buildkit involvement beyond the test harness itself.

So +test-ast guards two things the unit tests don't:

  1. The JSON wire format. TestParse compares Go structs, so a struct-tag change would pass unit tests while silently changing debug ast output for any external consumer of that JSON. Worth checking whether anything (editor/LSP tooling?) still consumes it before declaring the format free to change.
  2. A realistic corpus - it parses the real integration Earthfiles under tests/. Downside: brittle goldens - two .ast.json files changed in this PR just because a dind image bump touched the source .earth files.

Proposal: replace rather than delete. A plain Go golden test in internal/earthfile that walks the existing *.ast.json files, parses the paired .earth, marshals, and diffs normalised JSON keeps both of the above at unit-test cost. Then +test-ast, tests+ast-test-input, and the binary round-trip can go (perhaps keeping one debug ast CLI smoke, same pattern as the slimmed tests/version).

On the duplication in this file: agreed - folded. The invalid VERSION cases now live in TestParseErrors, the valid spellings in a TestVersionVariants table with inline inputs asserting Version.Args, and internal/earthfile/testdata/ is deleted. (invalid-feature-flag-override.earth was already covered inline in features/features_test.go, so that fixture went too.) Will be in the next push.

@gilescope gilescope enabled auto-merge (squash) July 4, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants