Skip to content

Publishing

Stéphane Brunner edited this page Oct 6, 2026 · 15 revisions

Introduction

C2C CI utils is built to be able to publish a project with the following structure:

Have stabilization branches named by default <major>.<minor>. Have tag for the release named by default <major>.<minor>.<patch>.

With C2C CI utils, you can publish a python package and a Docker image from the same repository.

The default publishing is:

  • Push on the <major>.<minor> branch will publish Docker images.
  • Create the tag <major>.<minor>.<patch> will publish the Docker images, and the Python package.
  • Push on a pull request will publish the Docker images with the pull request ID ad tag.
  • Close a pull request will delete the related Docker images.
  • Push on the master branch will publish the Docker images with the master tag (Publishing a python package is also possible).
  • The version at the last line of the SECURITY.md of the master branch will also be published using the latest tag, this will respect the tags present in the configuration.
  • In the SECURITY.md file of the master branch, we can also add a column Alternate Tag to publish the Docker images with another tag, this will respect the tags present in the configuration (only for Docker).

The Docker images are published on Docker Hub and GitHub Container Registry.

See also the publishing part of the Configuration.

Publishing with tag-publish

The publishing is now done with tag-publish, it can publish Python packages (PyPI), Docker images, Helm charts and npm packages from a tag, a stabilization branch, a feature branch or a pull request, without secrets (GitHub OIDC and GITHUB_TOKEN).

Note

The c2cciutils-publish flow described below is the legacy way, the new projects should use tag-publish.

Set the permissions, install the package with only the extras you need (this reduces the attack surface of the publishing jobs), and run the publishing:

permissions:
  # To publish Docker images on GHCR, npm packages and Helm charts on OCI registry
  packages: write
  # To publish Python packages using OIDC and for cosign keyless signing
  id-token: write
  # To publish Helm charts (chart-releaser) and send repository dispatch notifications
  contents: write
      - name: Install tag-publish
        run: pip install 'tag-publish[docker,helm]'

      - name: Publish
        run: tag-publish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Extras

A plain pip install tag-publish installs only the core dependencies (PyYAML, security-md, githubkit). If a publishing is configured in .github/publish.yaml but the corresponding extra is missing, the command fails with an explicit error before any side effect: ::error::Missing optional dependencies: ..., install tag-publish with: pip install tag-publish[<extra>].

Extra Kind of publishing Dependencies
pypi Python packages to PyPI requests, twine, id (OIDC trusted publishing)
docker Docker images (dpkg audit) debian-inspector
helm Helm charts (GitHub/OCI) applications-download (chart-releaser, cosign)
new tag-publish-new command multi-repo-automation, ruamel-yaml
validation .github/publish.yaml validation jsonschema-validator-new

Notes:

  • The Node (npm) publishing and the repository dispatch need no extra, they only use the npm CLI and the core githubkit.
  • Without the validation extra the schema validation of .github/publish.yaml is skipped with a warning.
  • Install several extras at once: pip install 'tag-publish[pypi,docker,helm]'.

The configuration file is .github/publish.yaml, see the tag-publish README.

To create a new stabilization branch, use tag-publish-new --version=<version> (requires the new extra), see the tag-publish README.

CI integration

Add your project to the Geo GHCI GitHub application to manage the security:

  • Trigger rebuild, then a dpkg version is updated.
  • Update the dependency against the CVE.

Note

Requires to be GitHub organization owner ot application owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.

There are different ways to be able to publish an artifact:

  • Directly with the GitHub Token, possible for anything that's on GitHub (Release, Docker images, ...)
  • Using OpenID connect (OIDC), possible to publish on PyPI.
  • Using other credentials available on gopass.

GitHub token

To be able to publish on GitHub Packages using the GitHub Token, you need the following permission in your workflow job:

permissions:
  package: write

OpenID connect (OIDC)

From version 1.7 you can publish on PyPI without any additional credentials, for that, you need the following permission in your workflow job:

permissions:
  id-token: write

Credentials from gopass

Add your project in the global secrets named CI_GPG_PRIVATE_KEYand GOPASS_CI_GITHUB_TOKEN.

Note

Requires to be GitHub organization owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.

Step used to initialize gopass:

      # Get the secrets
      - uses: camptocamp/initialise-gopass-summon-action@v2
        with:
          ci-gpg-private-key: ${{secrets.CI_GPG_PRIVATE_KEY}}
          github-gopass-ci-token: ${{secrets.GOPASS_CI_GITHUB_TOKEN}}
          patterns: pypi docker
        if: env.HAS_SECRETS == 'HAS_SECRETS'

Pull request automation workflow

Get the workflow pull-request-automation.yaml to review automatically the CVE, dpkg and Renovate updates.

Docker image

You should create a ci/dpkg-versions.yaml file with {}:

echo {} > ci/dpkg-versions.yaml
git add ci/dpkg-versions.yaml

Add your Docker images in ci/config.yaml like:

publish:
  docker:
    images:
      - name: camptocamp/project

Publishing

c2cciutils should be added in ci/requirements.txt.

You can use this workflow to integrate it in the CI

env:
  HAS_SECRETS: ${{ secrets.HAS_SECRETS }}

jobs:
  main:
    ...

    permissions:
      ...

    steps:
      ...

      # Install c2cciutils
      - run: echo "${HOME}/.local/bin" >> ${GITHUB_PATH}
      - run: python3 -m pip install --user --requirement=ci/requirements.txt

      ...

      # Publish
      - name: Publish
        run: c2cciutils-publish
        if: env.HAS_SECRETS == 'HAS_SECRETS'

Note

The ci/dpkg-versions.yaml file is managed automatically, the only manual task to do is to remove old distributions wher we uses a new one.

Run locally

You can run the publishing locally in dry-run mode:

GITHUB_REF=... c2cciutils-publish --dry-run ...

Create version branch

When we create a new version branch, we should:

  • Add a backport <version> label.
  • Update the Renovate base branch.
  • Update the SECURITY.md.
  • Create the branch :-P

To create all of that, you can also run: c2cciutils-version --version=<version> [--force]

Note that didn't create a release

Create a release

Just create a tag:

git tag <release>
git pus origin <tag>

Clone this wiki locally