🐥 Running pytest as a Github Action

I recently rediscovered my love for Python by adding testing to an existing project. The project needed to be migrated from Python 2 to 3 which made testing a crucial part of the exercise.

I settled on using pytest for implementing the tests.

To be productive, one of the first steps I did was to automate the tests with a Github action so that as soon as I push code to the repository, the tests would run automatically.

Let's first start with the setup of the setup of the project.

There are two very important files which should be checked in into the repository to make this work: requirements.txt and packages.list. The packages.list file contains the list of packages which need to be installed via apt (the action will be based on an Ubuntu Linux container). The contents is something like:

packages.list

mupdf-tools
poppler-utils

The second one (requirements.txt) is the list of Python packages we need for the project:

requirements.txt

pytest
pytest-cov
pytest-github-actions-annotate-failures

You might have noticed that there are 2 different pytest packages we install:

• pytest: needed to run pytest
• pytest-cov: adds the option for code coverage to pytest
• pytest-github-actions-annotate-failures: turns pytest failures into action annotations

Speaking of code coverage, you might also want to add a config file for that as well, which should be called .coveragerc. The contents in my case is:

.coveragerc

[run]
omit =
    *test.py

When you want to run the tests manually, you can execute:

pytest --exitfirst --verbose --failed-first --cov=. --cov-report html

Now, let's combine this into an action. We start with creating a file called .github/workflows/tests.yaml in the root of the repository. This is the file which describes how the action should work. It's contents is as follows:

.github/workflows/tests.yaml

name: Tests
on: [push]

jobs:
  build:
    name: Run Python Tests
    runs-on: ubuntu-latest

    steps:
    
    - uses: actions/checkout@v2

    - name: Setup timezone
      uses: zcong1993/setup-timezone@master
      with:
        timezone: UTC

    - name: Set up Python 3.8
      uses: actions/setup-python@v2
      with:
        python-version: 3.8
    
    - name: Install Python dependencies
      run: |
        sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
        python3 -m pip install --upgrade pip
        pip3 install -r requirements.txt

    - name: Test with pytest
      run: |
        pytest --exitfirst --verbose --failed-first \
        --cov=. --cov-report html

The action is configured to run each time you push to the repository. The workflow is called Test. You can also see that it runs on the latest version of Ubuntu as specified by runs-on: ubuntu-latest.

The first step is to checkout the source code by using the actions/checkout@v2 action.

The next step is to configure the timezone. This is important if you happen to do time-related stuff and you want to ensure you are in a known timezone. You can use an action zcong1993/setup-timezone@master for doing this.

After that, we install the correct version of Python using the actions/setup-python@v2 action. We are using version 3.8 for this example.

Then, we install all the Linux and Python dependencies by running a couple of terminal commands:

sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt

The last step is to run the tests themselves by running the pytest command.

A sample of the output of the GitHub action can be seen in this example. When you click on a failed run, you'll see the failures in the list of annotations.

A full repository showing this setup can be found here.