Deployment and CI/CD

This document explains how the pyjiit package is built, versioned, and published to the Python Package Index (PyPI). It covers the automated CI/CD workflows using GitHub Actions and the Poetry-based build system.

Scope: This page focuses on package deployment to PyPI. For documentation deployment to GitHub Pages, see Documentation Deployment. For local development setup and dependency management, see Build System and Dependencies.

Overview

The pyjiit project uses a streamlined deployment strategy consisting of:

1. Poetry as the build backend and dependency manager
2. GitHub Actions for automated publishing to PyPI
3. GitHub Releases as the primary trigger for deployments
4. Manual workflow dispatch as a fallback trigger

The deployment process is fully automated—creating a GitHub release automatically builds and publishes the package to PyPI without manual intervention.

Build System Configuration

The package build system is defined in pyproject.toml using modern Python packaging standards (PEP 621). The configuration specifies the build backend as Poetry, which handles dependency resolution, packaging, and publishing.

Package Metadata

The package metadata is defined in pyproject.toml1-17

Dependencies

The project separates runtime dependencies from development dependencies:

Runtime Dependencies (shipped with the package):

• requests (>=2.32.3,<3.0.0) - HTTP client for API calls
• pycryptodome (>=3.22.0,<4.0.0) - AES encryption implementation

Documentation Dependencies (not shipped with the package):

• sphinx (>=7.4.7) - Documentation generator
• furo (^2024.8.6) - Sphinx theme

Runtime dependencies are defined in pyproject.toml9-12 Documentation dependencies are in a separate docs group at pyproject.toml19-21 which keeps the production installation lean.

Build Backend

The build system configuration specifies poetry-core as the build backend:

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

This allows Poetry to build source distributions (.tar.gz) and wheel distributions (.whl) that are uploaded to PyPI.

PyPI Publishing Workflow

The automated publishing workflow is defined in .github/workflows/python-publish.yml. This workflow handles the complete build and publish process when triggered.

Workflow Triggers

The workflow can be triggered by two mechanisms:

Primary Trigger: GitHub release publication .github/workflows/python-publish.yml3-5

When a release is published on GitHub, the workflow automatically executes. This is the intended production deployment method.

Secondary Trigger: Manual workflow dispatch .github/workflows/python-publish.yml7-8

The workflow can be manually triggered from the GitHub Actions UI, useful for hotfixes or testing the deployment process.

Workflow Permissions

The workflow is granted contents: write permission .github/workflows/python-publish.yml9-10 which allows it to:

• Read repository contents during checkout
• Write release artifacts if needed

Workflow Execution Steps

The publishing job runs on ubuntu-latest and executes the following steps:

Detailed Step Breakdown:

Authentication and Secrets

The workflow authenticates to PyPI using an API token stored in GitHub Secrets:

• Secret Name: PYPI_API_KEY
• Configuration Command: poetry config pypi-token.pypi ${{ secrets.PYPI_API_KEY }}
• Usage: .github/workflows/python-publish.yml27

This token must be generated from PyPI's account settings and added to the repository's GitHub Secrets. Poetry reads this configured token when executing poetry publish.

Build and Upload Process

The poetry publish --build command performs two operations:

1. Build: Creates distribution packages

   • Source distribution: pyjiit-{version}.tar.gz
   • Wheel distribution: pyjiit-{version}-py3-none-any.whl

2. Publish: Uploads both distributions to PyPI using the configured token

The --build flag ensures that the latest code is packaged before upload, preventing accidental publication of stale builds.

Version Management

Package versioning follows semantic versioning with alpha release indicators:

• Current Version: 0.1.0a8 pyproject.toml3
• Format: MAJOR.MINOR.PATCH[aN]
  • 0.1.0 - Major, minor, and patch numbers
  • a8 - Alpha release number 8

The version is manually updated in pyproject.toml before creating a GitHub release. Once the package reaches stability, the aN suffix will be removed for the first stable release.

Complete CI/CD Pipeline

The following diagram shows the complete deployment pipeline from code changes to PyPI publication:

Pipeline Characteristics:

• Trigger Point: GitHub release creation .github/workflows/python-publish.yml4-5
• Execution Time: ~2-3 minutes for checkout, setup, build, and upload
• Artifacts: Source and wheel distributions uploaded to PyPI
• Distribution: Available globally via pip install pyjiit immediately after upload

Release Process

To deploy a new version of pyjiit to PyPI, follow this process:

Step 1: Update Version

Edit the version field in pyproject.toml:

version = "0.1.0a9"  # Increment as appropriate

Location: pyproject.toml3

Step 2: Commit and Push

git add pyproject.toml
git commit -m "Bump version to 0.1.0a9"
git push origin main

Step 3: Create GitHub Release

1. Navigate to the repository's Releases page
2. Click "Draft a new release"
3. Create a new tag matching the version (e.g., v0.1.0a9)
4. Set the release title and description
5. Click "Publish release"

Step 4: Automated Deployment

The python-publish.yml workflow executes automatically:

• Builds the package using Poetry
• Publishes to PyPI using the configured token
• Package becomes available via pip install pyjiit within minutes

Manual Deployment Alternative

If needed, the workflow can be manually triggered:

1. Navigate to Actions tab in GitHub
2. Select "Build and publish python package" workflow
3. Click "Run workflow"
4. Select the branch/tag to deploy
5. Click "Run workflow" to start

The manual trigger uses .github/workflows/python-publish.yml7-8 for execution.

Workflow Comparison

The pyjiit project has two primary GitHub Actions workflows with different purposes:

Both workflows use Poetry for dependency management but serve distinct purposes in the project's deployment pipeline.

Dependency Lock File

While poetry.lock is not included in the provided files, it plays a critical role in the deployment process:

• Purpose: Locks exact versions of all dependencies and sub-dependencies
• Generation: Created/updated by poetry install and poetry update
• Usage: Ensures reproducible builds across different environments
• CI/CD Impact: The workflow's poetry install command reads poetry.lock to install exact versions

The lock file ensures that the package built in CI/CD uses the same dependency versions as local development, preventing "works on my machine" issues.

Security Considerations

Token Security

• PyPI API token stored as GitHub Secret PYPI_API_KEY
• Token never appears in logs or workflow outputs
• Token scope should be limited to the pyjiit package on PyPI

Permission Model

• Workflow has contents: write permission .github/workflows/python-publish.yml10
• Only repository maintainers can create releases that trigger deployment
• Manual workflow dispatch requires repository write access

Package Integrity

• Published packages are immutable on PyPI (cannot be replaced)
• Each version can only be published once
• Source code is tagged at release time, providing traceability