Software Testing for Research - Python: Key Points

Introduction

This course will teach you how to write effective tests and ensure the quality and reliability of your research software
No prior testing experience is required
You can catch up on practicals by copying the corresponding folder from the files directory of this course’s materials

Automated testing helps to catch hard to spot errors in code & find the root cause of complex issues.
Tests reduce the time spent manually verifying (and re-verifying!) that code works.
Tests help to ensure that code works as expected when changes are made.
Tests are especially useful when working in a team, as they help to ensure that everyone can trust the code.

The assert keyword is used to check if a statement is true and is a shorthand for writing if statements in tests.
Pytest is invoked by running the command pytest ./ in the terminal.
pytest will run all the tests in the current directory, found by looking for files that start with test_.
The output of a test is displayed in the terminal, with green text indicating a successful test and red text indicating a failed test.
It’s best practice to write tests in a separate file from the code they are testing. Eg: scripts.py and test_scripts.py.

You can run multiple tests at once by running pytest in the terminal.
Pytest searches for tests in files that start or end with ‘test’ in the current directory and subdirectories.
The output of pytest tells you which tests have passed and which have failed and precisely why they failed.
Flags such as -v, -q, -k, and -x can be used to get more detailed output, less detailed output, run specific tests, and stop running tests after the first failure, respectively.

Complex functions can be broken down into smaller, testable units.
Testing each unit separately is called unit testing.
The AAA pattern is a good way to structure your tests.
Test driven development can help you to write clean, maintainable code.
Randomness in tests can be made deterministic using random seeds.
Adding tests to an existing project can be done incrementally, starting with regression tests.

You can test equality of lists and dictionaries using the == operator.
Numpy arrays cannot be compared using the == operator. Instead, use numpy.testing.assert_array_equal and numpy.testing.assert_allclose.
Data structures that contain numpy arrays should be compared using numpy.testing.assert_equal.
Pandas DataFrames and Series should be compared using pandas.testing.assert_frame_equal and pandas.testing.assert_series_equal.

Fixtures are useful way to store data, objects and automations to re-use them in many different tests.
Fixtures are defined using the @pytest.fixture decorator.
Tests can use fixtures by passing them as arguments.
Fixtures can be placed in a separate file or in the same file as the tests.

Parametrization is a way to run the same test with different parameters in a concise and more readable way, especially when there is a lot of repetition in the setup for each of the different test cases.
Use the @pytest.mark.parametrize decorator to define a parametrized test.

Regression testing ensures that the output of a function remains consistent between changes and are a great first step in adding tests to an existing project.
pytest-regtest provides a simple way to do regression testing.
pytest-mpl provides a simple way to test plots by comparing the output of a test function to a reference image.

Continuous Integration (CI) is the practice of automating the merging of code changes into a project.
GitHub Actions is a feature of GitHub that allows you to automate the testing of your code.
GitHub Actions are defined in yaml files and are stored in the .github/workflows directory in your repository.
You can use GitHub Actions to only allow code to be merged into the main branch if the tests pass.