Automate testing with tox.
This WIP merge request addresses point 1 of issue #5 (closed): Run unit tests.
It provides a basic skeleton for automating code checks. It's based on the cookiecutter template that Platform uses to bootstrap data pipelines.
Gitlab's CI Pipeline status can be found at https://gitlab.wikimedia.org/repos/research/knowledge-gaps/-/pipelines
Note: Marking as draft because I want to start a conversation to validate direction. This change is not prescriptive, and feedback is very much welcome.
I adopted the same type of code checks we use in https://gitlab.wikimedia.org/repos/generated-data-platform/datapipelines, but I relaxed the strictness a bit.
A CI Pipeline is triggered
after each push
and merge request and will run the following checks:
- Unit tests (stored under
tests/
). - Linting, with flake8, runs against the
knowledge_gaps
module (rules are defined in the tox.ini file) - Type checking, with mypy, runs against the
knowledge_gaps
module (rules are defined in tox.ini file)
Only unit tests are required to pass to mark a pipeline as successful, flake8/mypy are allowed to fail (but will raise a warning).
Changes
- add the capability to automate pytest / mypy / flake8 via tox.
- added a Gitlab CI pipeline config (
.gitlab-ci.yml
). - unit tests have been moved into a toplevel tests/ dir.
Known issues
- Tests are failing. But I'd rather address a fix in a dedicate merge request.
- some code under
interactive/
is not valid Python. There's some notebook magic (%%load_ext) that will raise a syntax error when parsed by mypy/flake8. For know I'd be keen in leavinginteractive
code alone, under the assumption the it will be refactored. @aikochou @bmansurov does this track with you? How do you plan to structure this code base?