Skip to content

Add configurable chunked uploads - #342

Open
XYenon wants to merge 4 commits into
zhaofengli:mainfrom
XYenon:feat/chunked-upload
Open

XYenon wants to merge 4 commits into
zhaofengli:mainfrom
XYenon:feat/chunked-upload

Conversation

@XYenon

@XYenon XYenon commented Jun 1, 2026

Copy link
Copy Markdown

Add chunked transport upload sessions for pushing large NARs. Clients can upload a NAR as sequential transport parts, and the server stores those parts in temporary upload sessions before ingesting the reconstructed NAR.

Expose chunked upload capability through cache-config when the server configures upload.max-chunk-size. Clients use non-chunked uploads when the capability is absent, use the advertised max chunk size by default when present, and reject explicit --chunk-size values that exceed the server limit.

Finalize upload sessions asynchronously and let clients poll for completion. The server waits until all expected parts are present before starting finalization, then validates part ordering and size, ingests the reconstructed NAR, records the result, and cleans up temporary session parts.

Add upload session database entities, migrations, API routes, and garbage collection for stale session data.

Closes #287.

@XYenon
XYenon marked this pull request as ready for review June 1, 2026 11:30
@XYenon
XYenon force-pushed the feat/chunked-upload branch from b30f058 to 74a031a Compare June 15, 2026 07:49
@XYenon
XYenon force-pushed the feat/chunked-upload branch 4 times, most recently from b997c10 to 97b4135 Compare June 28, 2026 09:04
XYenon and others added 3 commits July 17, 2026 00:27
Add chunked transport upload sessions for pushing large NARs. Clients can upload a NAR as sequential transport parts, and the server stores those parts in temporary upload sessions before ingesting the reconstructed NAR.

Expose chunked upload capability through cache-config when the server configures upload.max-chunk-size. Clients use non-chunked uploads when the capability is absent, use the advertised max chunk size by default when present, and reject explicit --chunk-size values that exceed the server limit.

Finalize upload sessions asynchronously and let clients poll for completion. The server waits until all expected parts are present before starting finalization, then validates part ordering and size, ingests the reconstructed NAR, records the result, and cleans up temporary session parts.

Add upload session database entities, migrations, API routes, and garbage collection for stale session data.

Amp-Thread-ID: https://ampcode.com/threads/T-019e8181-6abd-750d-a9c6-42d48108a529
Co-authored-by: Amp <amp@ampcode.com>
@XYenon
XYenon force-pushed the feat/chunked-upload branch from 97b4135 to fe8e54d Compare July 16, 2026 16:33
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support pushing data in chunk

1 participant