feat: make test auto-discovery recursive like pytest#47
Merged
Conversation
- _discover_modules_in now uses rglob() instead of glob(), so test files in subdirectories are found automatically (matching pytest's default discovery behaviour). - MODULE values are now relative file paths with .py extension (e.g. 'subdir/test_foo.py') so they can be passed as positional path arguments to pytest for precise per-file targeting. - MODULE keys flatten subdirectory separators with underscores (e.g. MODULE/subdir_test_foo) to produce valid, flat identifiers. - task.yaml templates updated: "$MODULE" is now a positional pytest path argument instead of -k "$MODULE". - Removed dead _discover_test_modules function (never called). - Updated docs (AGENTS.md, docs/ISD283.md) to reflect new behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Explicit MODULE values must now be file paths (e.g. test_charm.py) since $MODULE is passed as a positional pytest argument. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Makes the key/value symmetric: MODULE/test_charm.py: test_charm.py. Explicit entries in auto-discover: false suites now clearly mirror what auto-discovery generates, with no surprise extension stripping. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…9_]) The spread varname regex ^[a-zA-Z_][a-zA-Z0-9_]*(?:/[a-zA-Z0-9_]+... only allows alphanumerics and underscores in the variant part of an environment key. Dots (from .py) are rejected as invalid variable names. Key uses stem (e.g. MODULE/test_charm), value retains the file path (e.g. test_charm.py) passed as a positional pytest argument. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hyphens and dots in directory or file names (e.g. k8s-charm/test_deploy.py) would produce invalid spread varname keys. Replace any char outside [a-zA-Z0-9] with _ using re.sub, not just forward slashes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When two discovered test paths sanitize to the same key (e.g. a-b/test_x.py and a_b/test_x.py both → MODULE/a_b_test_x), the second would silently overwrite the first, dropping a test from the spread matrix entirely. Now raises ConfigurationError with a clear message identifying both conflicting paths and the colliding key. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pytest runs from SPREAD_PATH/OPCLI_CWD (project root by default). MODULE values were relative to the suite directory (e.g. test_charm.py), so pytest would look for <root>/test_charm.py which doesn't exist. Fix: prepend posixpath.relpath(suite, cwd) to each discovered module path so values are resolvable from OPCLI_CWD: suite tests/integration/, cwd ./ → tests/integration/test_charm.py suite haproxy/tests/integration/, cwd haproxy/ → tests/integration/test_charm.py Update docs and test assertions accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MODULE values must be relative to OPCLI_CWD. For machine-charm with cwd: machine-charm/, test_charm.py → tests/integration/test_charm.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pytest {posargs} tests/integration/ -v appends the suite dir after
MODULE path, running all tests. Use {posargs:tests/integration/} so
MODULE replaces the default rather than being added alongside it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test auto-discovery was non-recursive (
globonly searched the immediate suite directory). This aligns it with how pytest works by default.Changes
Core (
src/opcli/core/spread.py)_discover_modules_in: switched fromglob()torglob()for recursive discovery_module_key(new): derives a flat spread variant key from a relative path by replacing/with_and stripping.pysubdir/test_foo.py) passed as positional pytest path arguments — more precise than-k_(e.g.MODULE/subdir_test_foo)task.yamltemplates:"$MODULE"is now a positional arg, not-k "$MODULE"_discover_test_modulesfunction (was never called)Tests (
tests/unit/test_spread.py)test_integration_suites_auto_discovers_modulesto assert.py-suffixed valuestest_integration_suites_auto_discovers_nested_modulesfor the subdir caseDocs (
AGENTS.md,docs/ISD283.md)MODULE format
test_foo.pyMODULE/test_footest_foo.pysubdir/test_foo.pyMODULE/subdir_test_foosubdir/test_foo.pyCompatibility
auto-discover: falsesuites with explicitMODULE/…entries are unaffected. Auto-discovered MODULE values change from stems (test_charm) to paths (test_charm.py).