Repository navigation
Publishing
C2C CI utils is built to be able to publish a project with the following structure:
Have stabilization branches named by default <major>.<minor>.
Have tag for the release named by default <major>.<minor>.<patch>.
With C2C CI utils, you can publish a python package and a Docker image from the same repository.
The default publishing is:
- Push on the
<major>.<minor>branch will publish Docker images. - Create the tag
<major>.<minor>.<patch>will publish the Docker images, and the Python package. - Push on a pull request will publish the Docker images with the pull request ID ad tag.
- Close a pull request will delete the related Docker images.
- Push on the
masterbranch will publish the Docker images with the master tag (Publishing a python package is also possible). - The version at the last line of the
SECURITY.mdof themasterbranch will also be published using thelatesttag, this will respect thetagspresent in the configuration. - In the
SECURITY.mdfile of themasterbranch, we can also add a columnAlternate Tagto publish the Docker images with another tag, this will respect thetagspresent in the configuration (only for Docker).
The Docker images are published on Docker Hub and GitHub Container Registry.
See also the publishing part of the Configuration.
The publishing is now done with tag-publish, it can publish
Python packages (PyPI), Docker images, Helm charts and npm packages from a tag, a stabilization branch,
a feature branch or a pull request, without secrets (GitHub OIDC and GITHUB_TOKEN).
Note
The c2cciutils-publish flow described below is the legacy way, the new projects should use tag-publish.
Set the permissions, install the package with only the extras you need (this reduces the attack surface of the publishing jobs), and run the publishing:
permissions:
# To publish Docker images on GHCR, npm packages and Helm charts on OCI registry
packages: write
# To publish Python packages using OIDC and for cosign keyless signing
id-token: write
# To publish Helm charts (chart-releaser) and send repository dispatch notifications
contents: write - name: Install tag-publish
run: pip install 'tag-publish[docker,helm]'
- name: Publish
run: tag-publish
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}A plain pip install tag-publish installs only the core dependencies (PyYAML, security-md, githubkit).
If a publishing is configured in .github/publish.yaml but the corresponding extra is missing, the command
fails with an explicit error before any side effect:
::error::Missing optional dependencies: ..., install tag-publish with: pip install tag-publish[<extra>].
| Extra | Kind of publishing | Dependencies |
|---|---|---|
pypi |
Python packages to PyPI |
requests, twine, id (OIDC trusted publishing) |
docker |
Docker images (dpkg audit) | debian-inspector |
helm |
Helm charts (GitHub/OCI) |
applications-download (chart-releaser, cosign) |
new |
tag-publish-new command |
multi-repo-automation, ruamel-yaml
|
validation |
.github/publish.yaml validation |
jsonschema-validator-new |
Notes:
- The Node (npm) publishing and the repository dispatch need no extra, they only use the
npmCLI and the coregithubkit. - Without the
validationextra the schema validation of.github/publish.yamlis skipped with a warning. - Install several extras at once:
pip install 'tag-publish[pypi,docker,helm]'.
The configuration file is .github/publish.yaml, see the
tag-publish README.
To create a new stabilization branch, use tag-publish-new --version=<version> (requires the new extra),
see the tag-publish README.
Add your project to the Geo GHCI GitHub application to manage the security:
- Trigger rebuild, then a dpkg version is updated.
- Update the dependency against the CVE.
Note
Requires to be GitHub organization owner ot application owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.
There are different ways to be able to publish an artifact:
- Directly with the GitHub Token, possible for anything that's on GitHub (Release, Docker images, ...)
- Using OpenID connect (OIDC), possible to publish on PyPI.
- Using other credentials available on gopass.
To be able to publish on GitHub Packages using the GitHub Token, you need the following permission in your workflow job:
permissions:
package: write
From version 1.7 you can publish on PyPI without any additional credentials, for that, you need the following permission in your workflow job:
permissions:
id-token: write
Add your project in the global secrets named CI_GPG_PRIVATE_KEYand GOPASS_CI_GITHUB_TOKEN.
Note
Requires to be GitHub organization owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.
Step used to initialize gopass:
# Get the secrets
- uses: camptocamp/initialise-gopass-summon-action@v2
with:
ci-gpg-private-key: ${{secrets.CI_GPG_PRIVATE_KEY}}
github-gopass-ci-token: ${{secrets.GOPASS_CI_GITHUB_TOKEN}}
patterns: pypi docker
if: env.HAS_SECRETS == 'HAS_SECRETS'Get the workflow pull-request-automation.yaml to review automatically the CVE, dpkg and Renovate updates.
You should create a ci/dpkg-versions.yaml file with {}:
echo {} > ci/dpkg-versions.yaml
git add ci/dpkg-versions.yamlAdd your Docker images in ci/config.yaml like:
publish:
docker:
images:
- name: camptocamp/projectc2cciutils should be added in ci/requirements.txt.
You can use this workflow to integrate it in the CI
env:
HAS_SECRETS: ${{ secrets.HAS_SECRETS }}
jobs:
main:
...
permissions:
...
steps:
...
# Install c2cciutils
- run: echo "${HOME}/.local/bin" >> ${GITHUB_PATH}
- run: python3 -m pip install --user --requirement=ci/requirements.txt
...
# Publish
- name: Publish
run: c2cciutils-publish
if: env.HAS_SECRETS == 'HAS_SECRETS'Note
The ci/dpkg-versions.yaml file is managed automatically, the only manual task to do is to remove old distributions wher we uses a new one.
You can run the publishing locally in dry-run mode:
GITHUB_REF=... c2cciutils-publish --dry-run ...When we create a new version branch, we should:
- Add a
backport <version>label. - Update the Renovate base branch.
- Update the
SECURITY.md. - Create the branch :-P
To create all of that, you can also run: c2cciutils-version --version=<version> [--force]
Note that didn't create a release
Just create a tag:
git tag <release>
git pus origin <tag>