Skip to content

Commit 3d6a8b1

Browse files
ihalatciihalatci-msftCopilot
authored
Add PostgreSQL 19 support, drop PostgreSQL 16 (#423)
* Add PostgreSQL 19 support, drop PostgreSQL 16 Retarget the Docker image automation to the PG 17/18/19 window for Citus 15.0.0, and bump the hll/topn extension pins to 2.21.citus-1 / 2.7.1.citus-1 across the docker templates and the live test images. - update_docker.py / publish_docker.py: postgres16 -> postgres19 - templates/docker: add postgres-19, delete postgres-14/15/16 - latest.tmpl.dockerfile: bump stale hll 2.17 / topn 2.5.0 pins, which would otherwise break the `latest` image now that it tracks PG19 - test-images: bump hll/topn on the 8 non-EOL distros - package-tests.yml: 14.1.0 -> 15.0.0 Also fixes two latent bugs that block the PG19 rollout: 1. update_docker.py used `git add --update .`, which stages only tracked files, so a newly generated image directory was never committed. This is why postgres-18/ and the pg18 Docker Hub tag do not exist today despite the script generating them. Now uses `git add -A .`. 2. citus_package.py built the nightly PG list by filtering `release_versions` instead of `nightly_versions`, so any nightly exclusion would silently emit release versions. Dormant only because the nightly exclusion map is currently empty; packaging is about to populate it for el/8 on PG19. The ol/7 unit test asserted the buggy output and has been corrected. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ed7b58a-0ba9-4f47-a1de-3ee665bae7f2 * Sync latest/alpine docker templates with citusdata/docker The latest and alpine templates had drifted behind the docker repo, so every release run silently reverted manual fixes made there: - alpine: clang/llvm -> clang21/llvm21 (c9917b8, re-applied in 75bc0c2). Alpine has no unversioned llvm package, so reverting this would break the PG19 alpine build. - latest: back-port the hadolint hardening from b47008b (SHELL pipefail, pinned =* apt specs, --no-install-recommends, quoted package specs, sed merged into the chmod RUN). Both templates now render byte-identical to the committed docker files. The latest template gained one line, so its line-index assertions in test_update_docker.py were adjusted accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ed7b58a-0ba9-4f47-a1de-3ee665bae7f2 * Fix CI: alpine test fixture and package-tests fallback version Two CI regressions from this branch: 1. dbe5abd synced the alpine docker template with citusdata/docker, bumping clang/llvm to clang21/llvm21. test_process_template_file compares the rendered template byte-for-byte against expected_alpine_10.0.3.txt, so the fixture needs the same change. 2. The package-tests.yml fallback was bumped from 14.1.0 to 15.0.0, but Citus 15.0.0 is not released yet, so every test_execution job failed on "No match for argument: citus150_<pg>-15.0.0.citus-1". Reverted to 14.1.0. The bump is not needed for PostgreSQL 19 support: 15.0.0 currently resolves to the same [16, 17, 18] matrix as 14.1.0, because packaging's postgres-matrix.yml has no 15.0 entry yet. Both should be updated together when Citus 15.0 ships. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ed7b58a-0ba9-4f47-a1de-3ee665bae7f2 --------- Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ed7b58a-0ba9-4f47-a1de-3ee665bae7f2
1 parent 62a4613 commit 3d6a8b1

22 files changed

Lines changed: 93 additions & 187 deletions

File tree

‎packaging_automation/citus_package.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,7 +329,7 @@ def get_postgres_versions(
329329
if exclude_dict_nightly and platform_key_nightly in exclude_dict_nightly:
330330
nightly_versions = [
331331
v
332-
for v in release_versions
332+
for v in nightly_versions
333333
if v not in exclude_dict_nightly[platform_key_nightly]
334334
]
335335

‎packaging_automation/publish_docker.py‎

Lines changed: 8 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ class DockerImageType(Enum):
2828
latest = 1
2929
alpine = 2
3030
nightly = 3
31-
postgres_14 = 4
32-
postgres_15 = 5
33-
postgres_16 = 6
34-
postgres_17 = 7
35-
postgres_18 = 8
31+
postgres_17 = 4
32+
postgres_18 = 5
33+
postgres_19 = 6
3634

3735

3836
class ManualTriggerType(Enum):
@@ -58,21 +56,6 @@ class ScheduleType(Enum):
5856
"docker-tag": "alpine",
5957
"schedule-type": ScheduleType.regular,
6058
},
61-
DockerImageType.postgres_14: {
62-
"file-name": "postgres-14/Dockerfile",
63-
"docker-tag": "pg14",
64-
"schedule-type": ScheduleType.regular,
65-
},
66-
DockerImageType.postgres_15: {
67-
"file-name": "postgres-15/Dockerfile",
68-
"docker-tag": "pg15",
69-
"schedule-type": ScheduleType.regular,
70-
},
71-
DockerImageType.postgres_16: {
72-
"file-name": "postgres-16/Dockerfile",
73-
"docker-tag": "pg16",
74-
"schedule-type": ScheduleType.regular,
75-
},
7659
DockerImageType.postgres_17: {
7760
"file-name": "postgres-17/Dockerfile",
7861
"docker-tag": "pg17",
@@ -83,6 +66,11 @@ class ScheduleType(Enum):
8366
"docker-tag": "pg18",
8467
"schedule-type": ScheduleType.regular,
8568
},
69+
DockerImageType.postgres_19: {
70+
"file-name": "postgres-19/Dockerfile",
71+
"docker-tag": "pg19",
72+
"schedule-type": ScheduleType.regular,
73+
},
8674
DockerImageType.nightly: {
8775
"file-name": "nightly/Dockerfile",
8876
"docker-tag": "nightly",

‎packaging_automation/templates/docker/alpine/alpine.tmpl.dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ RUN apk add --no-cache \
2020
curl-dev \
2121
openssl-dev \
2222
ca-certificates \
23-
clang \
24-
llvm \
23+
clang21 \
24+
llvm21 \
2525
lz4-dev \
2626
zstd-dev \
2727
libxslt-dev \

‎packaging_automation/templates/docker/latest/latest.tmpl.dockerfile‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,15 @@ LABEL maintainer="Citus Data https://citusdata.com" \
1414
ENV CITUS_VERSION ${VERSION}.citus-1
1515

1616
# install Citus
17+
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
1718
RUN apt-get update \
1819
&& apt-get install -y --no-install-recommends \
19-
ca-certificates \
20-
curl \
20+
ca-certificates=* \
21+
curl=* \
2122
&& curl -s https://install.citusdata.com/community/deb.sh | bash \
22-
&& apt-get install -y postgresql-$PG_MAJOR-citus-{{project_minor_version}}=$CITUS_VERSION \
23-
postgresql-$PG_MAJOR-hll=2.17.citus-1 \
24-
postgresql-$PG_MAJOR-topn=2.5.0.citus-1 \
23+
&& apt-get install -y --no-install-recommends "postgresql-$PG_MAJOR-citus-{{project_minor_version}}=$CITUS_VERSION" \
24+
"postgresql-$PG_MAJOR-hll=2.21.citus-1" \
25+
"postgresql-$PG_MAJOR-topn=2.7.1.citus-1" \
2526
&& apt-get purge -y --auto-remove curl \
2627
&& rm -rf /var/lib/apt/lists/*
2728

@@ -33,10 +34,10 @@ COPY 001-create-citus-extension.sql /docker-entrypoint-initdb.d/
3334

3435
# add health check script
3536
COPY pg_healthcheck wait-for-manager.sh /
36-
RUN chmod +x /wait-for-manager.sh
37+
RUN chmod +x /wait-for-manager.sh \
38+
&& sed "/unset PGPASSWORD/d" -i /usr/local/bin/docker-entrypoint.sh
3739

3840
# entry point unsets PGPASSWORD, but we need it to connect to workers
3941
# https://github.com/docker-library/postgres/blob/33bccfcaddd0679f55ee1028c012d26cd196537d/12/docker-entrypoint.sh#L303
40-
RUN sed "/unset PGPASSWORD/d" -i /usr/local/bin/docker-entrypoint.sh
4142

4243
HEALTHCHECK --interval=4s --start-period=6s CMD ./pg_healthcheck

‎packaging_automation/templates/docker/postgres-14/postgres-14.tmpl.dockerfile‎

Lines changed: 0 additions & 42 deletions
This file was deleted.

‎packaging_automation/templates/docker/postgres-15/postgres-15.tmpl.dockerfile‎

Lines changed: 0 additions & 42 deletions
This file was deleted.

‎packaging_automation/templates/docker/postgres-17/postgres-17.tmpl.dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ RUN apt-get update \
2020
curl \
2121
&& curl -s https://install.citusdata.com/community/deb.sh | bash \
2222
&& apt-get install -y postgresql-$PG_MAJOR-citus-{{project_minor_version}}=$CITUS_VERSION \
23-
postgresql-$PG_MAJOR-hll=2.18.citus-1 \
24-
postgresql-$PG_MAJOR-topn=2.7.0.citus-1 \
23+
postgresql-$PG_MAJOR-hll=2.21.citus-1 \
24+
postgresql-$PG_MAJOR-topn=2.7.1.citus-1 \
2525
&& apt-get purge -y --auto-remove curl \
2626
&& rm -rf /var/lib/apt/lists/*
2727

‎packaging_automation/templates/docker/postgres-18/postgres-18.tmpl.dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ RUN apt-get update \
2020
curl \
2121
&& curl -s https://install.citusdata.com/community/deb.sh | bash \
2222
&& apt-get install -y postgresql-$PG_MAJOR-citus-{{project_minor_version}}=$CITUS_VERSION \
23-
postgresql-$PG_MAJOR-hll=2.19.citus-1 \
24-
postgresql-$PG_MAJOR-topn=2.7.0.citus-1 \
23+
postgresql-$PG_MAJOR-hll=2.21.citus-1 \
24+
postgresql-$PG_MAJOR-topn=2.7.1.citus-1 \
2525
&& apt-get purge -y --auto-remove curl \
2626
&& rm -rf /var/lib/apt/lists/*
2727

packaging_automation/templates/docker/postgres-16/postgres-16.tmpl.dockerfile renamed to packaging_automation/templates/docker/postgres-19/postgres-19.tmpl.dockerfile

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# This file is auto generated from it's template,
2-
# see citusdata/tools/packaging_automation/templates/docker/postgres-16/postgres-16.tmpl.dockerfile.
2+
# see citusdata/tools/packaging_automation/templates/docker/postgres-19/postgres-19.tmpl.dockerfile.
33
FROM postgres:{{postgres_version}}
44
ARG VERSION={{project_version}}
55
LABEL maintainer="Citus Data https://citusdata.com" \
@@ -20,8 +20,8 @@ RUN apt-get update \
2020
curl \
2121
&& curl -s https://install.citusdata.com/community/deb.sh | bash \
2222
&& apt-get install -y postgresql-$PG_MAJOR-citus-{{project_minor_version}}=$CITUS_VERSION \
23-
postgresql-$PG_MAJOR-hll=2.18.citus-1 \
24-
postgresql-$PG_MAJOR-topn=2.6.0.citus-1 \
23+
postgresql-$PG_MAJOR-hll=2.21.citus-1 \
24+
postgresql-$PG_MAJOR-topn=2.7.1.citus-1 \
2525
&& apt-get purge -y --auto-remove curl \
2626
&& rm -rf /var/lib/apt/lists/*
2727

‎packaging_automation/tests/files/verify/expected_alpine_10.0.3.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ RUN apk add --no-cache \
2020
curl-dev \
2121
openssl-dev \
2222
ca-certificates \
23-
clang \
24-
llvm \
23+
clang21 \
24+
llvm21 \
2525
lz4-dev \
2626
zstd-dev \
2727
libxslt-dev \

0 commit comments

Comments
 (0)