Skip to content

PMM-15238: Expose the built-in PostgreSQL to SEP - #5700

Open
ademidoff wants to merge 1 commit into
mainfrom
PMM-15238-expose-pg-to-sep
Open

PMM-15238: Expose the built-in PostgreSQL to SEP#5700
ademidoff wants to merge 1 commit into
mainfrom
PMM-15238-expose-pg-to-sep

Conversation

@ademidoff

@ademidoff ademidoff commented Jul 26, 2026

Copy link
Copy Markdown
Member

Ticket number: PMM-15238

Feature build: Percona-Lab/pmm-submodules#4495

Lets SEP, running in a side container on a shared Docker bridge network, use PMM's embedded PostgreSQL for its persistence layer. Entirely opt-in — with PMM_ENABLE_SEP unset, nothing about PMM changes.

What it does

When PMM_ENABLE_SEP=1 and PMM_SEP_POSTGRES_PASSWORD are set, the entrypoint runs a new helper (build/ansible/roles/postgres/files/postgres-sep) before supervisord starts. It:

  • appends a marker-delimited block to postgresql.conf setting listen_addresses = '*' — this only covers the container's own network namespace, nothing is published on the host;
  • appends a marker-delimited block to pg_hba.conf with one scram-sha-256 rule per attached Docker subnet (derived from ip -o -4 route show scope link), scoped to the sep database and sep user only;
  • provisions a non-superuser sep role (NOSUPERUSER NOCREATEROLE NOCREATEDB) owning a dedicated sep database.

The postgres, pmm-managed and grafana accounts stay unreachable over the network — Postgres rejects them with no pg_hba.conf entry.

Both config blocks are delimited by # BEGIN PMM SEP / # END PMM SEP and rewritten on every start, so the operation is idempotent. Unsetting PMM_ENABLE_SEP strips them on the next start and returns Postgres to loopback-only; the sep role and database are deliberately left in place, since a config flag should not destroy data.

Drive-by: PG version literals

/srv/postgres14, /srv/.postgres_password and /usr/pgsql-14/bin were duplicated across the entrypoint and postgres-migration. They are now declared once in build/docker/server/entrypoint.sh and passed to both helper scripts via a subshell-scoped export, so they do not leak into the environment supervisord and its children inherit (verified across every /proc/*/environ). Each script aborts with a named message if one is missing.

psql now resolves through $POSTGRES_BIN_DIR rather than /usr/bin/psql, which is an alternatives symlink — fine today, but it resolves to whichever version alternatives prefers, exactly the wrong property during a 14 → 18 migration with both installed.

Still pinned to 14 and out of scope here: build/ansible/roles/supervisord/files/pmm.ini, managed/services/supervisord/pmm_config.go, and the build/ansible/roles/postgres/tasks/*.yml playbooks.

Testing

Verified against percona/pmm-server:3.8.1 with the scripts bind-mounted, using a psql client container on a shared bridge network.

Scenario Result
Fresh volume, SEP enabled sep@sep connects, DDL/DML works, pg_stat_statements created
seppmm-managed / postgres superuser / grafana over the network all rejected, no pg_hba.conf entry
Wrong password authentication failed
Role attributes rolsuper, rolcreatedb, rolcreaterole all false
Restart, password rotated one marker block per file, old password rejected, new accepted
Restart with PMM_ENABLE_SEP unset blocks removed, loopback-only, connection refused, sep role and database survive
Missing PMM_SEP_POSTGRES_PASSWORD container exits 1 with an actionable message
With PMM_DISABLE_BUILTIN_POSTGRES=1 / PMM_HA_ENABLE=1 warns and starts normally
Two bridge networks attached one host rule per subnet, connects from both
Existing volume SEP-off, and brand-new volume never SEP-enabled start clean, readyz OK, health=healthy
POSTGRES_* in supervisord and all children absent

shellcheck clean on all three scripts; go test ./managed/utils/envvars/... passes.

Notes

  • Docker only — the AMI and OVF distributions do not run this entrypoint.
  • Traffic on the bridge network is not TLS-encrypted; acceptable container-to-container on a dedicated network, would need revisiting if SEP ever connects from another host.
  • PG 18 migration: SEP data now lives in PMM's cluster. A future 14 → 18 migration must carry over all databases, not just pmm-managed and grafana, or the sep database will be silently dropped — nothing in PMM's code references it, so the omission would not be caught.

@ademidoff
ademidoff requested a review from a team as a code owner July 26, 2026 23:24
@ademidoff
ademidoff requested review from 4nte and JiriCtvrtka and removed request for a team July 26, 2026 23:24
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.52%. Comparing base (31318c7) to head (45f2a37).
⚠️ Report is 108 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5700      +/-   ##
==========================================
+ Coverage   43.59%   45.52%   +1.92%     
==========================================
  Files         415      418       +3     
  Lines       43134    43409     +275     
==========================================
+ Hits        18804    19761     +957     
+ Misses      22454    21694     -760     
- Partials     1876     1954      +78     
Flag Coverage Δ
admin 34.93% <ø> (+0.15%) ⬆️
agent 51.43% <ø> (+2.39%) ⬆️
managed 45.00% <100.00%> (+2.02%) ⬆️
vmproxy 72.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add an opt-in integration that lets SEP, running in a side container on a
shared Docker bridge network, use PMM's embedded PostgreSQL for its
persistence layer.

When PMM_ENABLE_SEP is set, the entrypoint appends marker-delimited blocks to
postgresql.conf (listen_addresses) and pg_hba.conf (one scram-sha-256 rule per
attached Docker subnet, scoped to the sep database and role), then provisions a
non-superuser sep role owning a dedicated sep database. Nothing is published on
the host, and the postgres, pmm-managed and grafana accounts remain unreachable
over the network. Unsetting the variable reverts the configuration on the next
start and leaves the role and database intact.

The postgres data directory, password file and binary directory are now
declared once in the entrypoint and passed to the helper scripts via a
subshell-scoped export, replacing the /usr/pgsql-14 literals that were
duplicated across them.
@yyyyyyyan
yyyyyyyan force-pushed the PMM-15238-expose-pg-to-sep branch from 42cf667 to 45f2a37 Compare August 10, 2026 16:16
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The change adds optional SEP access to the built-in PostgreSQL server. It introduces conditional role and database provisioning, subnet-based authentication rules, configurable PostgreSQL binary paths, and entrypoint handling for SEP environment variables.

Changes

SEP PostgreSQL access

Layer / File(s) Summary
Runtime configuration and PostgreSQL paths
build/ansible/roles/postgres/files/postgres-migration, build/docker/server/entrypoint.sh, docker-compose.yml, managed/utils/envvars/*, .env.example
PostgreSQL scripts use exported data, password, and binary directory variables. The entrypoint passes these variables in a scoped subshell. Compose and example configuration define SEP variables. The environment parser ignores SEP variables and tests cover this behavior.
SEP provisioning and PostgreSQL configuration
build/ansible/roles/postgres/files/postgres-sep
The new script enables or disables SEP configuration blocks, validates attached container subnets, writes SCRAM authentication rules, and provisions the sep role and database.

Sequence Diagram(s)

sequenceDiagram
  participant Entrypoint
  participant postgres-sep
  participant PostgreSQL
  Entrypoint->>postgres-sep: Export SEP and PostgreSQL settings
  postgres-sep->>PostgreSQL: Apply network and authentication configuration
  postgres-sep->>PostgreSQL: Create or update sep role and database
  PostgreSQL-->>postgres-sep: Return command results
Loading

Possibly related PRs

  • percona/pmm#5295: Both changes modify PostgreSQL migration and entrypoint behavior, including shared path and environment handling.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the primary change: exposing embedded PostgreSQL to SEP.
Description check ✅ Passed The description includes the required ticket and feature build fields and provides detailed implementation and testing information.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@managed/utils/envvars/parser.go`:
- Around line 123-125: Update the raw environment-entry trace logging in the
parser, before the switch handling “PMM_ENABLE_SEP” and
“PMM_SEP_POSTGRES_PASSWORD”, to avoid emitting the password value; log only the
environment key or redact its value while preserving existing filtering
behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5f3afa25-7950-41c5-8756-d7229db5279d

📥 Commits

Reviewing files that changed from the base of the PR and between dd8cfdc and 45f2a37.

📒 Files selected for processing (7)
  • .env.example
  • build/ansible/roles/postgres/files/postgres-migration
  • build/ansible/roles/postgres/files/postgres-sep
  • build/docker/server/entrypoint.sh
  • docker-compose.yml
  • managed/utils/envvars/parser.go
  • managed/utils/envvars/parser_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • percona/pmm-qa (manual)
  • percona/pmm (manual)

Comment on lines +123 to +125
case "PMM_ENABLE_SEP", "PMM_SEP_POSTGRES_PASSWORD":
// skip env variables consumed by the entrypoint to expose postgres to SEP
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Prevent PMM_SEP_POSTGRES_PASSWORD from reaching trace logs.

Line 85 logs every raw environment entry before this case executes. When trace logging is enabled, it records the SEP database password. Log only the key, or redact the value before logging. Make it so.

Proposed fix
-		logrus.Tracef("ParseEnvVars: %#q: k=%#q v=%#q", env, k, v)
+		logrus.Tracef("ParseEnvVars: k=%#q", k)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@managed/utils/envvars/parser.go` around lines 123 - 125, Update the raw
environment-entry trace logging in the parser, before the switch handling
“PMM_ENABLE_SEP” and “PMM_SEP_POSTGRES_PASSWORD”, to avoid emitting the password
value; log only the environment key or redact its value while preserving
existing filtering behavior.

Source: Coding guidelines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants