Skip to content

[Core] Add OCI artifact reuse support for ome-agent#665

Open
op109lvb wants to merge 1 commit into
mainfrom
ome-agent-replica-reuse
Open

[Core] Add OCI artifact reuse support for ome-agent#665
op109lvb wants to merge 1 commit into
mainfrom
ome-agent-replica-reuse

Conversation

@op109lvb

Copy link
Copy Markdown
Collaborator

What this PR does

Adds OCI Object Storage artifact reuse support to ome-agent replica flow.

The replica agent now:

  • Writes a completion marker after a successful upload.
  • Reuses an existing completed target artifact when reuse is explicitly allowed.
  • Creates an active upload lock before writing to a reusable target prefix.
  • Makes concurrent jobs for the same target prefix wait instead of uploading at the same time.
  • Handles stale active upload locks with a configurable timeout.
  • Cleans up the active upload lock after success or normal failure.
  • Adds Object Storage helpers for conditional upload/delete and richer object listing metadata.

Why we need it

Multiple replication jobs can target the same OCI Object Storage artifact prefix. Without coordination, they can upload concurrently and expose partial artifacts to later consumers. The completion marker and active upload lock make reuse safe while preserving legacy behavior unless reuse is explicitly enabled by the replication job.

Fixes #

How to test

Run:

go test ./internal/ome-agent/replica ./pkg/ociobjectstore ./pkg/constants

E2E:

  • Create two replication jobs targeting the same reusable OCI Object Storage prefix.
  • Verify only one job uploads the artifact.
  • Verify the second job waits and then reuses the completed artifact.
  • Verify stale active upload lock behavior by using a short lock timeout.

Checklist

  • Tests added/updated (if applicable)
  • Docs updated (if applicable)
  • make test passes locally

@github-actions github-actions Bot added ome-agent OME agent changes storage Storage provider changes tests Test changes config Configuration changes labels Jul 13, 2026
EnableSizeLimitCheck: true,
HFDownloadTimeout: 72 * time.Hour,
HFDownloadStaleProgressTimeout: 30 * time.Minute,
ArtifactUploadLockTimeout: 120 * time.Hour,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why 120 hrs? (why so long)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

120h is conservative because the existing HF download timeout is 72h and the replication job active deadline is 96h. The lock timeout should be longer than a legitimate long-running job, otherwise a second job could treat an active upload as stale and write to the same OCI Object Storage prefix. It is configurable, so we can lower it from the ConfigMap to unblock if needed.

@op109lvb
op109lvb requested a review from EdHasNoLife July 15, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config Configuration changes ome-agent OME agent changes storage Storage provider changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants