Skip to content

[Backport] Allow multipart artifact uploads above 5 GiB - #4317

Open
buildkate wants to merge 2 commits into
v3from
backport-4292-multipart-artifacts
Open

[Backport] Allow multipart artifact uploads above 5 GiB#4317
buildkate wants to merge 2 commits into
v3from
backport-4292-multipart-artifacts

Handle missing multipart upload instructions

e5b931f
Select commit
Loading
Failed to load commit list.
buildsworth-bk-app / Buildsworth review succeeded Sep 2, 2026 in 6m 18s

Review submitted — approved

Review summary: #4317

Change reviewed

This PR backports #4292 to the v3 branch. It addresses A-1839, where the agent rejected Buildkite-managed artifacts above 5 GiB even after the artifact API had accepted the artifact and supplied multipart upload instructions under a larger organization quota.

The implementation narrows the 5 GiB client-side guard in internal/artifact/bk_uploader.go: an oversized artifact is rejected only when its upload instructions are absent or contain no multipart actions. When actions are present, the existing multipart path sorts them by part number, divides the file into ranges, and creates one bkMultipartUpload work unit per action. Oversized single-form uploads still fail before opening or transmitting artifact data, and the error now explains that multipart instructions are required.

internal/artifact/bk_uploader_test.go expands the oversized-form test to cover both nil instructions and an empty action list. It also adds a boundary-focused multipart test using a logical size of 5 GiB plus one byte, without allocating a large fixture, and verifies the resulting workers' ordering, offsets, and sizes.

Evidence and verification

I read the full PR diff and both changed files, then traced the surrounding upload flow through:

  • api/artifacts.go, including the template and per-artifact instruction contracts.
  • internal/artifact/batch_creator.go, where API responses are attached to artifacts and per-artifact multipart instructions override the template.
  • internal/artifact/uploader.go, where work units are created, executed, and reported as multipart state with part ETags.
  • The CLI and job-runner configuration paths that advertise or disable multipart support.

I also reviewed the original issue A-1839, the merged source PR #4292, its review discussion, and its final commits. After excluding diff metadata such as blob IDs, the backport diff is identical to the merged source PR; no v3 translation changes were introduced. The original review's nil-instruction concern is covered by the second backport commit and its regression case.

Checks performed in this workspace:

  • git diff --check origin/v3...HEAD passed.
  • I could not rerun Go tests locally because this review environment does not have the Go executable installed.
  • The PR author reports go test ./internal/artifact passed and go tool gofumpt -extra produced no changes. The description records unrelated environment failures for the attempted full suite.
  • The original merged PR's Buildkite status passed.
  • At review submission time, the current PR's label and Socket checks had passed; Buildkite agent build #14231 was still running, so its result was not yet available.

Findings and verdict

No actionable correctness, security, compatibility, or backport-specific findings were identified. The review was submitted as APPROVE with no inline comments.

Trigger source: automatic.