refactor: overhaul unit tests and achieve 100% coverage on core modules#347
Merged
Conversation
- Add missing docstrings to core models and services. - Fix critical pylint errors (missing arguments, not-callable false positives). - Move imports to top level where possible. - Address broad exception catching with selective pylint-disable. - Fix signatures of abstract methods for consistency. - Refactor sandbox and pre-flight services for better adherence to standards. - Improve logging with lazy formatting. - Resolve various minor warnings (unused imports, reimports, etc.).
…pdating AI test mocks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The Autograder project required a significant overhaul of its unit and integration testing suite to ensure high-fidelity verification of core domain logic, recursive data structures, and failure recovery mechanisms. Previous tests relied too heavily on superficial mocking, which bypassed critical algorithmic paths such as score balancing and state transition validations.
Solution
This PR refactors and expands the testing suite to achieve near 100% coverage on core modules while enforcing deep, meaningful assertions:
MagicMockusage intests/unit/services/grader/test_submission_grader.pywith realSubjectNodeandTestNodeobjects. This allows for actual verification of the recursive weight-balancing algorithm and score calculation logic.tests/unit/models/test_criteria_tree.pyto achieve 100% coverage on recursive tree traversal methods (get_all_tests).tests/unit/services/test_sandbox_service.pywith comprehensive coverage for success paths and critical failure recovery (e.g., ensuring sandboxes are released back to the pool if workdir preparation fails).tests/unit/pipeline/test_pipeline_execution_accessors.pyto cover all state transition branches and strict data requirement validations.SandboxStep, including branches for skipping execution when no sandbox is required by templates.Further clarifications
pytest --cov=autograder --cov-branchto ensure maximal path coverage.Related issues
Addresses gaps identified in testing coverage audit.
Checklist