Skip to content

Conversation

@israelmcmc
Copy link
Collaborator

I realized, by eye and by chance, that one of the cells of the TS map tutorial was failing. Interestingly, the notebook continued running until the end, and the program reported a success.

While the error was only in a file checksum (fixed by this PR), and the code and tutorial are fine, this prompted me to look why this was not reported as an error.

As it happens, "cell magics" (the ones with "%%" at the beginning of the cell) are run in sub-process, and therefore the errors are caught inside and supressed. In particular, the TS map notebook is using the "%%capture" magic. Other notebooks use the "%%time" magic as well.

Prompted by this I changed a few things:

  • Fixed the file checksum
  • run_tutorials.py now looks for cell magics and comments them out.
  • I added a test tutorial to check for this: test_must_fail_magic.ipynb
  • Such tutorials with "must_fail" in the name now "fail succesfully". They show as FAILED but still in green.
  • The program now catches the exceptions from failed tutorial, to allow the exceuted version to still be saved (up to the cell with the error)

…fore failing tutorials were showing as passing.

Signed-off-by: Israel Martinez <imc@umd.edu>
Signed-off-by: Israel Martinez <imc@umd.edu>
Signed-off-by: Israel Martinez <imc@umd.edu>
Signed-off-by: Israel Martinez <imc@umd.edu>
Signed-off-by: Israel Martinez <imc@umd.edu>
Signed-off-by: Israel Martinez <imc@umd.edu>
…ean log file handler

Signed-off-by: Israel Martinez <imc@umd.edu>
@israelmcmc israelmcmc added documentation Improvements or additions to documentation maintenance tests labels Apr 15, 2025
@israelmcmc israelmcmc requested a review from ckarwin April 15, 2025 01:46
@codecov
Copy link

codecov bot commented Apr 15, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.46%. Comparing base (79d2f92) to head (122d4ad).
⚠️ Report is 133 commits behind head on develop.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation maintenance pull-request-needs-reviewer No reviewer assigned tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant