Conversation
Compose is never given a terminal to draw on: on Unix it runs under a pseudo-terminal so that its messages can be captured, and stdout is discarded unless in debug mode. Recent Compose versions decide to render interactive progress from stderr being a terminal and then require stdout to be one too, so every `docker compose build` fails with "failed to get console: provided file is not a console". Observed with Docker Compose v5.5.1: `elastic-package stack up` cannot start, from a terminal or not, unless ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT happens to be set. Plain progress is also what ends up in logs and in the error messages reported back to the user, so it is the right default rather than a CI setting. The environment variable keeps its other effects (--ansi never on Compose < 2.19.0, --quiet-pull).
There was a problem hiding this comment.
🟡 Changes recommended
A few updated docs/comments are still slightly misleading about the env var semantics and TTY behavior, and should be aligned with the new default --progress plain behavior before merge.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates elastic-package’s Docker Compose invocation to always use --progress plain for Docker Compose v2 >= 2.19.0, addressing failures seen with newer Compose versions when stdout is not a TTY (e.g., “failed to get console: provided file is not a console”). It also adds targeted unit coverage for the version boundary and updates documentation to reflect the new default behavior.
Changes:
- Set
--progress plainby default for Compose v2>= 2.19.0via a newcomposeProgressOutput(version)helper. - Add a unit test covering the
--progressflag boundary across several Compose versions. - Update README and README template documentation for
ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT.
File summaries
| File | Description |
|---|---|
internal/compose/compose.go |
Always selects an appropriate Compose --progress mode based on Compose version and adds rationale in comments. |
internal/compose/compose_test.go |
Adds unit coverage for progress output selection across Compose versions. |
README.md |
Documents the new default --progress plain behavior and clarifies env var effects. |
tools/readme/readme.md.tmpl |
Mirrors the README env var documentation updates in the generated template. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
🔵 Needs a closer look
A few wording and maintainability issues in the changed documentation and Compose initialization block should be addressed to avoid misleading users and reduce coupling to side effects.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
internal/compose/compose.go:216
- NewProject relies on dockerComposeVersion() mutating c.composeVersion as a side effect, even though the parsed version is already available as
ver. Usingverdirectly makes this easier to follow and avoids coupling this logic to that side effect; while touching this block, the comment should also avoid stating that stdout is always discarded (some commands capture stdout to a buffer, and debug mode may wire stdout differently).
README.md:937
- The README entry says this env var “disables the progress output”, but the behavior is actually about avoiding interactive/verbose rendering (and
--progress plainis now always set for Compose >= 2.19.0). Updating the wording will prevent users from expecting no progress output at all.
- `ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT`: If set to `true`, it disables the progress output from `docker compose`/`docker-compose` commands.
- For versions v2 `< 2.19.0`, it sets `--ansi never` flag.
- For all versions, it sets `--quiet-pull` for `up` sub-command.
- `--progress plain` is always set for versions v2 `>= 2.19.0`, regardless of this variable: Compose is run without a terminal on its standard output, and recent versions fail to render interactive progress there.
tools/readme/readme.md.tmpl:325
- This env var description says it “disables the progress output”, but the behavior is actually about avoiding interactive/verbose rendering (and
--progress plainis now always set for Compose >= 2.19.0). Updating the wording will prevent users from expecting no progress output at all.
- `ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT`: If set to `true`, it disables the progress output from `docker compose`/`docker-compose` commands.
- For versions v2 `< 2.19.0`, it sets `--ansi never` flag.
- For all versions, it sets `--quiet-pull` for `up` sub-command.
- `--progress plain` is always set for versions v2 `>= 2.19.0`, regardless of this variable: Compose is run without a terminal on its standard output, and recent versions fail to render interactive progress there.
- Files reviewed: 4/4 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
The updated docs still scope behavior to “versions v2”, which can be misleading given the described affected Compose version (e.g., 5.5.1) and the implementation’s semver-based gating.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Lite
There was a problem hiding this comment.
🟢 Approval recommended
The change is narrowly scoped, version-gated, covered by a targeted unit test, and the user-facing documentation is updated to match the new behavior.
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 0 new
- Review effort level: Lite
💛 Build succeeded, but was flaky
Failed CI Steps
History |
TL;DRAll five failing Buildkite jobs are failing before any package tests run due to transient Go module download failures from Remediation
Investigation detailsRoot CauseEach failed step exits during dependency installation ( This appears across multiple unrelated integration jobs with different modules, which is strong evidence of infra-level flakiness instead of a deterministic code/test failure introduced by this PR. Evidence
Verification
Follow-upNo open-code-path failure was identified to fix in this PR. If needed, we can add CI-level retry/cache hardening for module fetches in a separate infra-focused change. What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
What
Always pass
--progress plainto Docker Compose (≥ 2.19.0, where the flag exists), instead of only whenELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUTis set.Note: drafted with 🤖 Cursor/Fable 5.1, under my supervision.
Why
With Docker Compose v5.5.1 (current Docker Desktop),
elastic-package stack upfails on macOS and Linux before starting anything:It fails the same way from an interactive terminal and from a script. The only thing that makes it work today is setting
ELASTIC_PACKAGE_COMPOSE_DISABLE_VERBOSE_OUTPUT=true, which CI does and users do not.The cause is how compose is run.
runDockerComposeCmdon Unix starts it under a pseudo-terminal (creack/pty) so that its messages can be captured, whilecmd.Stdoutisio.Discard(oros.Stdoutin debug mode). So compose sees a terminal on stderr and a pipe on stdout. Recent compose decides to render interactive progress because stderr is a terminal, then hands stdout to BuildKit's console renderer, which refuses a non-terminal. Isolated with a one-service compose file:docker compose buildfailed to get consolefailed to get console--progress plainUpstream this is a regression in Compose v5.5.1 (v5.5.0 is fine, bisected with the release binaries), reported as docker/compose#14182 and fixed today by docker/compose#14194, which is not in a release yet. Their diagnosis is the same: the progress mode is resolved against stderr, the display is written to stdout. Docker Desktop 4.90.0 ships v5.5.1, so every macOS and Linux user who updated Docker Desktop in the last few days is affected as soon as compose has something to build —
stack upalways does, for the package registry image — until the next Desktop release picks up the fix. Windows is not affected:runDockerComposeCmddoes not use a pseudo-terminal there, so compose sees pipes everywhere and picks plain progress on its own. Onlybuildis affected;pull,up,psanddownwork in the same layout.Whatever compose does about it, elastic-package's wiring is the failing row by construction, so the fix belongs here too, and it covers every compose invocation: they all go through
compose.Project(stack, serverless local services, the compose/custom-agent/terraform service deployers, the agent deployer).internal/dockerrunsdockerdirectly with pipes and is not affected.Plain progress is also the right default on its own terms: compose's output is only ever seen through the captured stderr — in debug output and in the error message reported when a command fails — where the interactive renderer's escape sequences are noise. This is what the CI setting has been producing all along.
Changes
NewProjectsetsprogressOutputfor every compose ≥ 2.19.0, via a smallcomposeProgressOutput(version)so it can be unit-tested. The environment variable keeps its other effects:--ansi neveron compose < 2.19.0, and--quiet-pullonup.TestComposeProgressOutputcovers the version boundary (2.18.1, 2.19.0, 2.40.0, 5.5.1).Verified
A build of this branch, no environment variable set, from a shell without a terminal:
The released v0.125.1 fails at the build step on the same machine.
make lint,make gomodandgo test ./internal/compose/are clean.