Functions and Classes in Python
Testing basics¶
Introduction¶
Bugs are inevitable when writing any nontrivial piece of software. (Some estimates suggest that there are around 15-50 errors per 1000 lines of delivered code!) As developers, we want the users of our code to have confidence in its output. How can we be sure that our code is reproducible and does what it says on the tin?
Testing is the process by which software is validated and allows us a to catch and correct bugs before they become a problem. In this course we aim to teach you some of the basic principles for writing good tests. We will use the powerful pytest framework for writing and running our tests, and will show you how to automate the testing process using Travis.
In this course we will be working within Jupyter notebooks, as well as running commands in a terminal. We recommend that you open a terminal now (go the Jupyter home tab for this session, click on New, then select Terminal from the dropdown. We also suggest that you split your screen with the terminal and notebook tabs. This will make it easy to switch between them.
Assertions¶
Think of a test as an experiment. We perform an experiment by running a piece of code, then validating the output against a known value.
The assert statement is the usual means of testing that a condition is True.
assert condition
An AssertionError will be raised whenever the condition is False.
For example:
assert True
assert False
In Python 3 True and False are keywords, and are equal to 1 and 0 respectively. Hence, we can do:
assert True == 1
assert False == 0
Let's use assert to test the built in addition operator:
assert 1 + 2 == 3
Here the condition is 1 + 2 == 3. In our experiment we are adding the values of 1 and 2 together, then comparing the result to the known answer of 3.
Since the addition operator should be commutative, a better test would be:
assert 1 + 2 == 2 + 1 == 3
We've provided a package called mypkg containing a single module with some functions to test. Let's load it into the current namespace:
from mypkg import *
The module has its own addition function called add. Let's test that it works as expected:
help(add)
assert add(2, 2) == 4
Wow, look at that, it passed! How about another test.
assert add(3, 3) == 6
Fantastic! We must be brilliant programmers. Hang on, how about testing two different numbers...
assert add(3, 4) == add(4, 3) == 7
Damn, not so smart after all!
Lesson: One test is never enough. Make sure to test a function using a range of different inputs.
Now let's test the greaterThan function. First let's see how it works:
help(greaterThan)
assert greaterThan(2, 1)
assert greaterThan(100, 57)
assert greaterThan(33, 15)
assert greaterThan(999, 256)
assert greaterThan(-67, -354)
That's a load of tests. It must be right, surely?
Hang on a minute...
def alwaysTrue(x, y):
return True
assert alwaysTrue(2, 1)
assert alwaysTrue(100, 57)
assert alwaysTrue(33, 15)
assert alwaysTrue(999, 256)
assert alwaysTrue(-67, -354)
Lesson: Make sure that your test fails when you expect it to.
assert greaterThan(1, 2)
Phew!
It's good to also write tests for a condtions not being met.
assert not greaterThan(1, 2)
Approximate assertions¶
So far we have been testing conditions using integer values. What happens if we were to use floating point numbers?
assert 0.1 + 0.2 == 0.3
Comparisons between floating point numbers are not exact due to the limited precision by which they are stored, and changes in the order of operations can change results. As such, we need to test that the condition is approximately correct.
In the next section we will fully introduce the pytest framework. This comes with a useful approx function for comparing floating point numbers.
from pytest import approx
assert 0.1 + 0.2 == approx(0.3)
It also works for sequences of numbers. (And for almost all other useful data structures.)
assert (0.1 + 0.2, 0.2 + 0.4) == approx((0.3, 0.6))
By default, the approx function compares the result using a relative tolerance of 1e-6. This can be changed by passing a keyword argument to the function.
assert 2 + 3 == approx(7, rel=2)
Unit tests¶
In their simplest form, tests are usually just functions that encapsulate some assertions to validate a unit of code, such as a function or method. Tests like these are often referred to as unit tests, although the exact definiton of a unit is a little fuzzy. Regardless, unit tests should aim to be short and clearly defined. Aim to test one thing, and test it well.
Exercise¶
Below are two example test unit tests for the add and greaterThan functions.
Find out how the mul and sub functions from mymodule work and fill in the stub functions, test_mul and test_sub, below with some assert statements that test for the expected behaviour. Run the unit tests to make sure that they pass.
Add some assertions to test the functions using floating point numbers.
def test_add():
assert add(1, 2) == add(2, 1) == 3
assert add(3, 4) == add(4, 3) == 7
def test_greaterThan():
assert greaterThan(2, 1)
assert greaterThan(100, 57)
assert greaterThan(33, 15)
assert greaterThan(999, 256)
assert greaterThan(-67, -354)
def test_mul():
# Example assertions.
assert mul(3, 7) == mul(7, 3) == 21
assert mul(6.2, 4.7) == approx(mul(4.7, 6.2)) == approx(29.14)
assert mul(-5.3, 28.9) == approx(mul(28.9, -5.3)) == approx(-153.17)
def test_sub():
# Example assertions.
assert sub(8, 5) == -sub(5, 8) == 3
assert sub(11.18, 32.71) == -sub(32.71, 11.18) == -21.53
# Run the unit tests for the "add" function.
test_add()
# Run the unit tests for the "greaterThan" function.
test_greaterThan()
test_mul()
test_sub()