Testing

Build Status codecov

Testing is an important part Software Development, and Janis is no exception. Within the core janis project, there is a suite of code-level tests. High quality tests allow developers to be more confident in their code, and will decrease the amount of bugs found in production code.

For every issue, it’s worth looking at is there a tests than could have caught the error we’re facing, that way we can stop someone reintroducing the bug (regression).

Where to place tests

For each new logical section, you should create a file with a new set of tests. All of the tests are placed in: janis/tests/test_*.py.

How to run tests

In the terminal you can run:

nosetests -w janis

Alternatively if you use Pycharm, you can right-click the janis/tests/ folder, and click Run 'Unittests in tests'.

Run unit tests

Code Coverage

codecov

Code coverage isn’t a great metric for evaluating how good our tests are, though it does give an indication on whether more tests are needed.

Code coverage is automatically generated by the build server and uploaded to codecov (that’s where the badge comes from).

You can generate codecov locally by running:

nosetests -w janis --with-coverage --cover-package=janis

Code Quality

A style guide hasn’t been produced and hence a list of practices are not enforced, however it’s best to follow the PEP guidelines. You can use mypy to run some basic tests, we like to keep these suggestions minimised:

mypy janis
# OR
mypy --ignore-missing-imports .

Ignore missing imports stops a lot of extraneous errors from third-party modules from showing up

Type annotations

The project uses type annotations to improve the clarity of code. Using tools such as mypy allows us to catch typing errors that might not have been easier caught by other static analysers.

For this reason the project is pegged to at least Python 3.6.

Why might a type system be good?

Although not strictly related to Python, JS has some super fun issues with types:

1 + 1 		// 2
'1' + 1		// '11'
1 + '1'		// 2
let a = '1'
a -= 1		// 2

Some of these examples come from this discussion of a postmortem from airbnb.