Skip to content

Unit test for CLI mode #122

Open
miguelglezb wants to merge 110 commits into
danieljprice:mainfrom
miguelglezb:main
Open

Unit test for CLI mode #122
miguelglezb wants to merge 110 commits into
danieljprice:mainfrom
miguelglezb:main

Conversation

@miguelglezb

Copy link
Copy Markdown

This MR contains the base for unit testing in SPLASH. We have included a yml that runs after the build test. For now, the test is only run in a Debian 12.11 VM but could be expanded to include macOS. The CI installs the SPLASH version to be tested with the latest version of giza, then downloads a Phantom binary stellar simulation snapshot and generates a series of render images using some of the basic CLI options for SPLASH, namely:

  • Simple density render
  • Simple internal energy render
  • Density render + vector field
  • Options for --sink=0,1,2

These images are compared using perceptual hashing, which allows some flexibility in the numerical rigor of the testing, as oposed a methods such as checksum hashing. The control images were generated using a Docker image, and the corresponding Dockerfile can be found in scripts along the script that runs SPLASH for the images generation. Some further tweaking of the Docker may be needed in the future, but this method should prevent issues with small differences in fonts or window sizes once a standard is set.

The test is written in Python, using pytest as base. We also added a IntEnum class to define the tolerance in image differences:

class Threshold(IntEnum):
    HIGH     =  4  # Nearly identical, tight tolerance
    MEDIUM   =  7  # Minor platform differences expected
    LOW      = 10  # Moderate differences acceptable
    RELAXED  = 15  # Known sensitivity to platform/version differences

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a testing framework for rendering, adding configuration files, a Dockerfile, a script to generate test renders, and a Python test suite that uses image hashing to verify the outputs. The reviewer feedback is highly constructive and should be addressed: it identifies a potential TypeError in the test suite when the SPLASH_DIR environment variable is missing, suggests refactoring the repetitive test cases using pytest parameterization, points out an unused import in the rendering script, and recommends optimizing the Dockerfile build times by using shallow git clones.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread tests/test_figure_hash.py Outdated
Comment thread tests/test_figure_hash.py
Comment thread scripts/generate_renders_testing.py Outdated
Comment thread scripts/Dockerfile
miguelglezb and others added 3 commits June 14, 2026 22:06
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@miguelglezb

Copy link
Copy Markdown
Author

After checking gemini feedback, I approved them. Moreover, I forgot to mention in the original MR that each test checks three conditions (assert statements).

  • The first is a sanity check that verifies the image (using the SPLASH tested version) is generated in the first place.
  • Second is that the image is similar, or similar enough depending on the Threshold value, to the equivalent control image.
  • Third one is a sanity check that verifies that the generated image is not similar to an obvious completely different figure (bad_plot.png)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant