Tests are failing but the application works

[Julien00859]

The two tests test_get_snippet and test_missing_snippet are both failing even though the application is actually working.

The test_get_snippet test case if failing because it attempts to access a snippet named iexist but no such snippet actually exist inside of the storage.

The test_missing_snippet test case if failing because it attempts to access a snippet named idontexist that shouldn't exist inside of the storage and it verifies that the web application return a 404 Page not Found error... but the snippet actually exists inside of the storage so it downloads it wth a 200 - Ok http response.

The tests should not run using the actual, production, file-system storage!

[Julien00859]

There are multiple options possible to isolate the tests from the production storage.

Solution 1: Mocking

We could mock the Snippet class in order to patch get_by_name and make it return values/raise exceptions controlled by the test. The real method wouldn't be called, so it wouldn't lookup the storage.

Faking a return value:

with unittest.mock.patch('simplebin.Snippet') as mock_snippet:
    mock_snippet.get_by_name.return_value = Snippet(...)

Faking an exception with a side effect:

with unittest.mock.patch('simplebin.Snippet') as mock_snippet:
    mock_snippet.get_by_name.side_effect = ValueError(...)

We would still need to verify that the method was called by the correct argument, I suggest we use one of the many assert_called_x methods

Solution 2: Test storage

We could replace the storage path when running the tests, e.g. with a temporary directory that would be emptied between tests. Each test would then be able to manually save/delete snippets inside of the temporary directory and directory used by the web application without mocks.

We can use tempfile.mkdtemp and shutil.rmtree, to create/delete the temporary directory. Make sure to wrap the temporary directory inside of a pathlib.Path object (its needed by the application).

I think we can use setUpClass/tearDownClass/setUp/tearDown for that. Creating the temporary directory and replacing the simplebin.STORAGE_PATH inside of the setUpClass method, restoring the original path and removing the temporary directory in a tearDownClass method, emptying the temporary directory in between tests in a setUp method.

The following code can be used to remove the files inside of a directory: (replace the path variable with the actual path of the temporary directory)

path = ...
for file in path.iterdir():
    file.unlink()

Each test can then save/delete snippets, e.g:

simplebin.STORAGE_PATH.joinpath('iexist').write_text("Hello world")

simplebin.STORAGE_PATH.joinpath('idontexist').unlink(missing_ok=True)