At Hori Systems, we maintain an open-source Python package, a piece of code we found useful when developing applications for our clients, which we thought might be helpful for others. The crux of the matter is that the code requires maintenance: a bug suggested by a developer in the community or a quick update to support a new API version. So naturally, there’s hesitation because of the multiple steps involved in ensuring updates are appropriately done, tested, documented, versioned and released to the community. Occasionally, we forget to upload a release to PyPI; it is the Python Package Index, a software repository for the Python programming language. It is an annoying blocker we overcame recently by utilising Github Actions.
We got acquainted with Github Actions in 2019, a service Github released in 2018 that provides automatic CI/CD tasks on the code in your repositories. It’s free for public repositories, but they offer a limited amount of minutes for private repositories; it starts at 2,000 minutes per month on the free plan. Github’s documentation includes an example demonstrating how to release new packages to PyPI when creating a Github Release.
Now, let’s get to the fun part! First, add the YAML code to this file (.github/workflows/pypi.yml) in your repository.
name: PyPI Release
on:
push:
branches:
- main
jobs:
scheduled:
runs-on: ${ { matrix.os } }
strategy:
matrix:
os: [macos-latest]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Setup pip
run: |
python -m pip install --upgrade pip
python -m pip install twine
python -m pip install wheel
- name: Build wheel
run: python setup.py bdist_wheel
env:
CIBW_BUILD: cp27-* cp36-* cp37-* cp38-*
- name: Publish distribution 📦 to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${ { secrets.PYPI_API_TOKEN } }
run: |
python setup.py sdist bdist_wheel
twine upload dist/*Github Actions runs all the jobs each time any of the events in the on clause occurs. That said, you might configure your YAML file to have multiple jobs run and even enable strict dependencies between the jobs, so specific jobs run if and only other jobs pass certain conditions. In addition, it allows you to configure workflows, for instance, where the release job won’t run unless an earlier test job pass. The code or recipe in our YAML code is simple logic; even though it states anytime we create a Github Release, the steps in the release job run. It sets up Python, installs Python dependencies, and runs setup.py to build our distribution before finally using the twine tool and uploading it to PyPI using the API Token (TWINE_PASSWORD) we use to authenticate when uploading packages to PyPI.
To generate an API Token for your repository, follow the instructions on this page. Note: don’t forget to limit the token’s scope only to allow Github Actions to upload packages for your repo, copy the token because PyPI won’t reveal it again and paste it into your Github repo secrets with the name PYPI_API_TOKEN. There’s no need for us to keep the token in shared storage or expose it in case of unauthorized access to our private repository or feels the project is compromised – we can easily deactivate the old token and generate a new one.
Finally, we have a workflow for updating our open-source project by:
- updating our code as usual and modifying or specifying a new version number (twine will return a 400 error code if we don’t update the version number ‘cause of PyPI strict rules of not allowing the community to overwrite existing packages with the same version – the rule is the rule).
- Creating a new release by simply pushing changes to the repository.
We eliminated our blocker by efficiently releasing new versions using Github Actions to automate the entire process. Removing this blocker makes us feel we have more control over the process without breaking a sweat. 😎 🚀
Resources
-
PyPI published a note with instructions on creating a PyPI release using their custom Github Action if you prefer it.
-
The possibility is enormous; you could even automate the release of a new version by creating tags or uploading your release to PyPI when you push code to specific branches.
