Merged
Conversation
Without the --capture=no, it seemed like capturing wasn't reverted after pytest.main(). OTOH, I am unable to reproduce in a simple-test-case to report as a pytest bug. :-/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've convinced myself this is a good idea. If you're not as convinced, no worries, challenge me to defend it!
1. Why?
Because this is important for eventually testing compiled-builds. Pyinstaller has a reasonably good discovery process for finding what should be included when compiling, but it's not perfect. Missing items (if any) need to be identified manually and added to pyinstaller config file. Still need to test a compiled-build before blessing it for release, otherwise there's an unacceptable risk of releasing an "incomplete" LabGym. That risk needs to be mitigated, and performing a self-test is useful to develop confidence in the compiled-build candidate-for-release.
2. This seems useful only for compiled-builds, so why introduce it into main branch now? Why not wait and keep it in a separate dev branch for now?
That position is valid. But I'm thinking that as we write unit tests now (irrelevant of timeline for compiled-builds), we want those unit tests to work inside the LabGym context as well, so this lets us easily demonstrate that we are writing unit tests that are ultimately runnable from inside LabGym. As you can see from this PR, I had to rewrite some existing unit tests slightly to get them runnable from inside LabGym. I'm thinking better to not let unit test coverage grow with incompatibilities, that's the advantage of merging it now instead of later.
3. Is there any normal-user -facing change?
Yes there's a new menubar item "Misc" --> "Selftest" that pops up a modal dialog.
For now, instead of actually running the tests, it only gives instructions how to get LabGym to run the selftest.
I wanted to be able to run selftest from the menubar, but so far, I’m not seeing a good way to do it. That’s because wx.App is supposed to be a singleton, and, if you run unit tests that make changes to that wx.App instance, then you risk leaving behind a polluted wx.App.
4. How do you actually run the selftests?
Start LabGym with selftest action specified.
Either by command line option (--selftest) or by configfile entry, “selftest = true” (case sensitive!).
LabGym will execute selftests (same as pytest LabGym/tests/test*.py to run all test modules in LabGym/tests folder), then exit without proceeding to normal operation. Try it like
LabGym --debug --selftest