Skip to content

fix(sbom): fetch last 10 releases for Dakota history #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
castrojo merged 1 commit into from
May 14, 2026
+30 −19
Merged

fix(sbom): fetch last 10 releases for Dakota history #838

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 30 additions & 19 deletions scripts/fetch-github-sbom.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,27 +338,37 @@ const STREAM_SPECS = [
* @param {object|null} existing Existing SBOM cache for incremental updates
*/
async function processLatestTagStream(spec, existing) {
const imageRef = `ghcr.io/${spec.org}/${spec.package}:latest`;
console.log(` ${spec.id}: processing :latest tag (${imageRef})`);

const dateStr = await getImageCreatedDate(imageRef);
const cacheKey = dateStr ? `latest-${dateStr}` : "latest-unknown";
console.log(` ${spec.id}: cache key = ${cacheKey}`);

const existingEntry = existing?.streams?.[spec.id]?.releases?.[cacheKey];
const hasVersions = existingEntry?.packageVersions != null;
const hasAllPackages =
existingEntry?.packageVersions?.allPackages != null &&
Object.keys(existingEntry.packageVersions.allPackages).length > 0;
const isVerified = existingEntry?.attestation?.verified === true;
const isCacheHit =
!FORCE_REFRESH && hasVersions && hasAllPackages && isVerified;

// Seed releases from existing cache so history accumulates across nightly runs.
// Without this, every run discards all prior entries — leaving only today's key.
// Seed from existing cache — accumulates history across nightly runs.
const existingReleases = existing?.streams?.[spec.id]?.releases || {};
const releases = { ...existingReleases };

// Build the list of image refs to process: :latest plus the 10 most recent
// commit-SHA tags (each is a distinct tagged build pushed to GHCR).
const allTags = await fetchGhcrTags(spec.org, spec.package);
const commitTags = allTags
.filter((t) => /^[0-9a-f]{40}$/.test(t))
.slice(-10); // last 10 = most recently pushed
const imageRefs = [
`ghcr.io/${spec.org}/${spec.package}:latest`,
...commitTags.map((t) => `ghcr.io/${spec.org}/${spec.package}:${t}`),
];

Comment on lines +341 to +355
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Latest-stream backfill logic is not reachable with current stream specs.

processLatestTagStream() is only called when spec.usesLatestTag is truthy (Line 454), but Dakota specs in this file are configured via streamPrefix: "latest" and don’t set usesLatestTag. That makes the new multi-ref logic effectively dead for Dakota.

💡 Proposed fix 
@@
   {
     id: "dakota-latest",
@@
     streamPrefix: "latest",
+    usesLatestTag: true,
     keyRepo: "projectbluefin/dakota",
     keyless: true,
   },
   {
     id: "dakota-nvidia-latest",
@@
     streamPrefix: "latest",
+    usesLatestTag: true,
     keyRepo: "projectbluefin/dakota",
     keyless: true,
   },

Also applies to: 454-456

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/fetch-github-sbom.js` around lines 341 - 355, The new multi-ref
imageRefs logic is never exercised because processLatestTagStream() is only
invoked when spec.usesLatestTag is true but Dakota streams use streamPrefix:
"latest" instead; update the condition that triggers processLatestTagStream to
also accept specs with streamPrefix === "latest" (or set spec.usesLatestTag =
true when streamPrefix === 'latest') so the code that builds imageRefs (the
commitTags/latest GHCR refs) runs for those streams as well; ensure you modify
the call site that checks spec.usesLatestTag (and any related branching) rather
than the imageRefs construction itself so existing behavior is preserved for
other streams.

for (const imageRef of imageRefs) {
const dateStr = await getImageCreatedDate(imageRef);
const cacheKey = dateStr ? `latest-${dateStr}` : null;
if (!cacheKey) continue;
Comment on lines +357 to +359
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Date-only cache keys can overwrite multiple same-day Dakota builds.

Using latest-YYYYMMDD as the sole key means different refs pushed on the same day collapse into one entry (releases[cacheKey] = ...), so you can lose history even when processing 10 commit-SHA tags.

💡 Proposed fix 
-    const dateStr = await getImageCreatedDate(imageRef);
-    const cacheKey = dateStr ? `latest-${dateStr}` : null;
-    if (!cacheKey) continue;
+    const dateStr = await getImageCreatedDate(imageRef);
+    const refTag = imageRef.split(":").pop() || "unknown";
+    let cacheKey = dateStr
+      ? `latest-${dateStr}`
+      : `latest-unknown-${refTag.slice(0, 12)}`;
+    if (releases[cacheKey] && releases[cacheKey].imageRef !== imageRef) {
+      cacheKey = `${cacheKey}-${refTag.slice(0, 12)}`;
+    }

Also applies to: 417-419

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/fetch-github-sbom.js` around lines 357 - 359, The current cache key
uses only dateStr (from getImageCreatedDate) producing `latest-YYYYMMDD` which
lets multiple same-day image refs overwrite each other; change the cacheKey
generation to include a unique identifier (e.g., append a time component from
dateStr, the imageRef, or the commit SHA) so each imageRef produces a distinct
key (update the cacheKey calculation where dateStr and cacheKey are computed and
the other occurrence around lines 417-419 that assigns releases[cacheKey]).
Ensure the key remains deterministic and safe for map lookups but is unique per
image (use a short imageRef/sha or ISO datetime).


const existingEntry = releases[cacheKey];
const hasVersions = existingEntry?.packageVersions != null;
const hasAllPackages =
existingEntry?.packageVersions?.allPackages != null &&
Object.keys(existingEntry.packageVersions.allPackages).length > 0;
const isVerified = existingEntry?.attestation?.verified === true;
const isCacheHit =
!FORCE_REFRESH && hasVersions && hasAllPackages && isVerified;

console.log(` ${spec.id}: ${cacheKey}${isCacheHit ? " (cache hit)" : ""}`);

if (isCacheHit) {
releases[cacheKey] = existingEntry;
} else {
Expand Down Expand Up @@ -405,14 +415,15 @@ async function processLatestTagStream(spec, existing) {
}

releases[cacheKey] = {
tag: "latest",
tag: imageRef,
imageRef,
digest: null,
attestation,
packageVersions,
checkedAt: new Date().toISOString(),
};
}
} // end for imageRefs

return {
id: spec.id,
Expand Down
Loading