Skip to content

Build tdigest for Ubuntu 20.04 (focal) — PG11–16 pipeline - #1239

Open
Buket (aybuket) wants to merge 3 commits into
developfrom
tdigest-focal
Open

Buket (aybuket) wants to merge 3 commits into
developfrom
tdigest-focal

Conversation

@aybuket

Copy link
Copy Markdown

What&Why

PGDG dropped Ubuntu 20.04 wholesale: focal-pgdg now 404s on its Release file, and the archived mirror that replaced it freezes at tdigest 1.4.3. Nothing newer is available to pin against, so keeping focal current means rebuilding tdigest from upstream source.

This targets tdigest 1.4.6, the latest release — it carries the memory-safety and correctness work from recent versions (stricter NULL/empty validation, result clamping around rounding errors, interruptible long-running loops, plus 32-bit/arm64 build fixes).

What's here

File Role
Dockerfile focal builder image; pulls build tooling from the PGDG archive
build_tdigest_focal the entrypoint: upstream tarball + frozen focal debian/dpkg-buildpackage
smoke_test_focal_tdigest_debs installs the set in a clean focal container and proves the extension actually loads
build-tdigest-focal.yml orchestrates build → sign → verify → smoke test

The signing hand-off (debsigner mount layout) and the .ddeb.deb rename follow the conventions already established by the focal core pipeline, so nothing new is introduced on the publishing side.

How the build is shaped

One source build covers every major — there's no per-major matrix and no assemble step. tdigest is a straightforward PGXS extension, and its Debian packaging is multi-version by construction: debian/pgversions plus the pgxs_loop debhelper add-on hand the major list to pg_buildext, which emits postgresql-<major>-tdigest for each one in a single pass. The script narrows debian/pgversions to the requested majors and regenerates debian/control from control.in (pg_buildext updatecontrol), so only the intended packages are declared and built.

There are no packaging gymnastics: each runtime .deb carries its own tdigest.control and versioned SQL, with no split -scripts package and no update-alternatives bookkeeping.

PostgreSQL range

The default is PG11–16. Upstream's own make installcheck CI runs the full [18 17 16 15 14 13 12 11] matrix with no exclusions, so 11 and 12 compile and pass regression there. (The "PostgreSQL": "13.0.0" entry in META.json is stale PGXN prereq metadata — it contradicts that CI matrix and is never read by PGXS or pg_buildext.) On the archive side, postgresql-server-dev-11 and postgresql-11-tdigest are both reachable, so 11 is included in the enabled components.

Integrity

The upstream tarball is sha256-pinned — GitHub publishes no detached signature, so the pin is the only integrity gate and must not silently degrade:

TDIGEST_SHA256=5158d3a57e96883262a299d559ffb18da80616825b13f29f1c8b3113dd467a95

The frozen debian/ comes from apt-get source tdigest=1.4.3-1.pgdg20.04+1, which validates it against the archive's signed Release.

What CI checks

  • Signed — every .deb must contain an embedded _gpgmaint.
  • Self-contained — each runtime ships extension/tdigest.control and tdigest--<version>.sql.
  • Runtime ↔ dbgsym — the build aborts if a dbgsym's exact-version dependency ever drifts from its runtime; that mismatch is invisible at the package layer and only bites at apt --fix-broken install.
  • Smoke test on stock ubuntu:20.04, both paths that matter:
    • clean installCREATE EXTENSION tdigest, assert extversion, run a real tdigest_percentile aggregate on each major;
    • upgrade over PGDG 1.4.3 → install the old package, apply the new set, ALTER EXTENSION tdigest UPDATE, re-verify.

Locally I've confirmed the static pieces — bash -n on both scripts, YAML parse of the workflow, the sha256 pin against the real tarball, and PG11 support against upstream CI. The end-to-end build + smoke run happens in CI; the push: trigger on tdigest-focal exercises it before it merges to develop.

Notes for reviewers

  • Everything is overridable via workflow_dispatch: tdigest_version (1.4.6), pg_versions (11 12 13 14 15 16), tdigest_sha256 (blank → pinned default), run_tests (0).
  • RUN_TESTS=1 wires up the upstream regression suite but drags in postgresql-all; worth a single dispatch to confirm before depending on it.
  • DEB_REVISION defaults to 1.citus20.04+1, giving 1.4.6-1.citus20.04+1, which sorts cleanly above the frozen 1.4.3-1.pgdg20.04+1.

Rebuild tdigest .deb packages for Ubuntu 20.04 (focal) from upstream source, mirroring the focal PostGIS pipeline. PGDG dropped focal (focal-pgdg 404s, frozen archive stops at tdigest 1.4.3), so newer tdigest (default 1.4.5, memory-safety fixes) is rebuilt from upstream source + the frozen focal-era debian/ packaging. One multi-version source build emits postgresql-<major>-tdigest for PG12-16, signed via the debsigner image and smoke-tested in a stock ubuntu:20.04.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Some advertised dispatch inputs fail because of a source-directory collision and unsupported PostgreSQL majors.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Builds, signs, and smoke-tests tdigest 1.4.6 packages for PostgreSQL 11–16 on Ubuntu 20.04.

Changes:

  • Adds a focal builder using archived PGDG packaging.
  • Adds package signing and validation workflow.
  • Tests clean installation and upgrades from tdigest 1.4.3.
File summaries
File Description
scripts/build_tdigest_focal Builds multi-version tdigest packages.
scripts/smoke_test_focal_tdigest_debs Tests installation and upgrades.
dockerfiles/focal-tdigest-builder/Dockerfile Defines the focal build environment.
.github/workflows/build-tdigest-focal.yml Orchestrates build, signing, and testing.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/build-tdigest-focal.yml
Comment on lines +78 to +88
PKG_DIR="$(find "${WORK}" -maxdepth 1 -type d -name 'tdigest-*' -exec test -d '{}/debian' \; -print | head -1)"
[ -n "${PKG_DIR}" ] || { echo "ERROR: could not locate unpacked packaging source" >&2; exit 1; }

echo "==> [2/6] Fetch and verify upstream tdigest-${TDIGEST_VERSION}.tar.gz"
curl -4 -fsSL -o "tdigest-${TDIGEST_VERSION}.tar.gz" "${UPSTREAM_URL}"
echo "${TDIGEST_SHA256} tdigest-${TDIGEST_VERSION}.tar.gz" | sha256sum -c -

SRCDIR="${WORK}/tdigest-${TDIGEST_VERSION}"
rm -rf "${SRCDIR}"
tar xzf "tdigest-${TDIGEST_VERSION}.tar.gz"
cp -a "${PKG_DIR}/debian" "${SRCDIR}/debian"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants