Repository navigation
Add apm (Agent Package Manager) ecosystem support #16382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Diogo-Costa
wants to merge
36
commits into
dependabot:main
Choose a base branch
from
Diogo-Costa:feature/apm-ecosystem
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
4f94bab
Add apm (Agent Package Manager) ecosystem support
Diogo-Costa 939ea6d
Address PR review for apm ecosystem
Diogo-Costa 69f917a
Address second apm PR review round
Diogo-Costa 1c2c43c
Fix apm file_parser spec under isolated CI test runner
Diogo-Costa e9781cf
Fix same-line duplicate ref bumps and nested non-GitHub repos
Diogo-Costa d277b50
Enforce SemVer semantics, deep GitLab metadata, case-insensitive host
Diogo-Costa 194cc29
Make APM git tag resolution strict SemVer aware
Diogo-Costa 185f191
Update apm lockfile ref on bump and parse apm CLI version permissively
Diogo-Costa 4f68767
Bump apm file_fetcher to `# typed: strong`
Diogo-Costa d7efcab
Align apm lockfile updater with the real apm.lock.yaml schema
Diogo-Costa a1c7ba3
Register apm in production updater and match lockfile identity by hos…
Diogo-Costa 5b724d7
Do not rewrite apm.lock.yaml; leave it for APM to re-resolve on drift
Diogo-Costa 11fb024
Match prerelease requirement bounds with SemVer ordering
Diogo-Costa 18a8b90
Harden apm manifest parsing against valid-but-unsupported inputs
Diogo-Costa aa09394
Fix apm dependency identity for case-sensitive hosts and non-APM images
Diogo-Costa 2c7fec5
Skip apm scalars whose decoding differs from their source span
Diogo-Costa adc33ea
Document unsupported apm scalar spellings in README scope
Diogo-Costa dc08d5a
Strip the manifest default host from apm dependency names
Diogo-Costa 1e2aeca
Recognise GitHub Enterprise Cloud (*.ghe.com) as GitHub family
Diogo-Costa 862b117
Skip Azure DevOps hosts and preserve trailing whitespace on update
Diogo-Costa 2b5bfb0
Drop non-https ports from explicit clone URLs
Diogo-Costa f77435b
Classify hosts by hostname and skip http/git clone URLs
Diogo-Costa 7f72b1a
Don't downgrade higher refs when rewriting merged requirements
Diogo-Costa 1fea8bb
Handle uppercase URI schemes and pessimistic version constraints
Diogo-Costa b4667b9
Address round-22 review: identity, tags, and production classification
Diogo-Costa a201507
Recognise package-scoped tags in the parser and update checker
Diogo-Costa 6b6f3c3
Preserve custom-port authority in APM metadata source lookup
Diogo-Costa 5569a22
Fix APM identity ports, per-requirement tag families, and beta error
Diogo-Costa 7d4a315
Make BETA_DISABLED_MESSAGE a plain frozen literal for Sorbet strict
Diogo-Costa c2ec1c7
Resolve tags per-requirement across families and register apm in dry-run
Diogo-Costa f104e65
Detect vulnerability per requirement ref, not just the merged version
Diogo-Costa 10329c4
Report merged version from lowest post-update pin, not the max tag
Diogo-Costa d3be314
Allow apm in bin/dry-run.rb package-manager allowlist
Diogo-Costa 9f6cdc2
Fix apm security-fix recognition and per-declaration cooldown for mer…
Diogo-Costa 9ae6e65
Dedupe lockfile version parsing and fix manifest file matching
v-abhishekbhaskar c9e82e8
APM: drop unreachable security-update logic
Diogo-Costa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -93,3 +93,6 @@ | |
|
|
||
| "L: sbt": | ||
| - '(sbt)' | ||
|
|
||
| "L: apm": | ||
| - '(apm)' | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,9 @@ | ||
| [ | ||
| { | ||
| "core": "apm", | ||
| "test": "apm", | ||
| "ecosystem": "apm" | ||
| }, | ||
| { | ||
| "core": "bazel", | ||
| "test": "bazel", | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| BUNDLE_GEMFILE: "../dependabot-updater/Gemfile" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| /.bundle/* | ||
| !.bundle/config | ||
| /tmp | ||
| /dependabot-*.gem |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| inherit_from: ../.rubocop.yml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # syntax=docker.io/docker/dockerfile:1.20 | ||
| FROM ghcr.io/dependabot/dependabot-updater-core | ||
|
|
||
| USER dependabot | ||
|
|
||
| COPY --chown=dependabot:dependabot --parents apm common $DEPENDABOT_HOME/ | ||
| COPY --chown=dependabot:dependabot updater $DEPENDABOT_HOME/dependabot-updater |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| ## `dependabot-apm` | ||
|
|
||
| [APM (Agent Package Manager)][apm-repo] support for [`dependabot-core`][core-repo]. | ||
|
|
||
| APM is a git-based package manager for AI agent context — skills, prompts, | ||
| chat modes, instructions and other agent primitives — declared in an `apm.yml` | ||
| manifest. Because every dependency resolves to a git ref, Dependabot bumps APM | ||
| dependencies the same way it bumps other git-sourced ecosystems (GitHub Actions, | ||
| git submodules): by resolving the newest semver tag on the remote and rewriting | ||
| the manifest ref. | ||
|
|
||
| ### What Dependabot updates | ||
|
|
||
| Dependabot reads `apm.yml` and proposes updates for **string-shorthand git | ||
| dependencies that are pinned to a semver tag**, for example: | ||
|
|
||
| ```yaml | ||
| dependencies: | ||
| apm: | ||
| - microsoft/edge-ai#v1.0.0 # GitHub shorthand pinned to a tag | ||
| - gitlab.com/acme/prompts#v2.1.0 # FQDN shorthand for any git host | ||
| - octo-org/octo-skills/skills/review#v1.4.0 # virtual sub-path within a repo | ||
| - acme.ghe.com/org/repo/skills/review#v1.0.0 # virtual sub-path on a GHE Cloud host | ||
| ``` | ||
|
|
||
| Virtual sub-paths (`repo/skills/review`) are resolved on GitHub-family hosts — | ||
| `github.com` and GitHub Enterprise Cloud data-residency hosts (`*.ghe.com`), | ||
| which APM also treats as GitHub. On other hosts the whole path is treated as the | ||
| repository; virtual packages on self-hosted GHES (reachable only via an | ||
| arbitrary configured `GITHUB_HOST`) are a follow-up. | ||
|
|
||
| For each such entry Dependabot: | ||
|
|
||
| 1. Resolves the git remote (`https://<host>/<owner>/<repo>`). | ||
| 2. Finds the highest semver tag that satisfies the update/cooldown/ignore rules, | ||
| reusing `Dependabot::GitCommitChecker` (the same tag resolution used by the | ||
| GitHub Actions ecosystem). | ||
| 3. Rewrites only the ref in `apm.yml` (e.g. `#v1.0.0` → `#v1.4.0`), preserving | ||
| the rest of the declaration byte-for-byte. | ||
|
|
||
| Both plain `v1.4.0` / `1.4.0` tags and APM's package-scoped tags — | ||
| `review-v1.4.0`, `review--v1.4.0` and `review_v1.4.0`, where the prefix is the | ||
| package's own name (the repository name, or the final virtual-path component) — | ||
| are recognised, so a monorepo that tags each package independently is updated | ||
| correctly. Build metadata (`+build.5`) is preserved and, per SemVer, ignored for | ||
| precedence; equal-precedence tags break ties on the full tag string, so tag | ||
| resolution stays deterministic regardless of the order the remote advertises | ||
| them. | ||
|
|
||
| `apm.lock.yaml` is fetched as a read-only support file (used only to report the | ||
| APM CLI version) and is **never modified** — see the lockfile note below. | ||
|
|
||
| `devDependencies.apm` entries are updated too and are flagged as non-production | ||
| via the `development` dependency group. | ||
|
|
||
| ### Scope of this version | ||
|
|
||
| To keep the first iteration small and reviewable, the following are intentionally | ||
| **out of scope** and are ignored (never modified, never erroring): | ||
|
|
||
| - **Object-form entries** (`git:`, `registry:`, `id:`, `path:` maps) and `mcp:` | ||
| entries — only the string shorthand is parsed. | ||
| - **Branch- and SHA-pinned entries** — these are resolved by APM's own lockfile, | ||
| which Dependabot does not regenerate, so their manifest ref is left untouched. | ||
| - **Local path entries** (`./pkg`, `../pkg`, `/pkg`) — not backed by a remote git | ||
| host, so there is nothing to bump. | ||
| - **Azure DevOps hosts** (`dev.azure.com`, `ssh.dev.azure.com` and legacy | ||
| `*.visualstudio.com`) — APM resolves these to `org/project/_git/repo` clone | ||
| URLs, a structure this version's generic `host/owner/repo` builder cannot | ||
| construct, so ADO entries are skipped rather than resolved to a wrong remote. | ||
| Native `_git` clone-URL support is a follow-up. | ||
| - **`http://` and `git://` clone URLs** — Dependabot enumerates tags over HTTPS, | ||
| so an `https://` or `ssh://`/SCP explicit URL is resolved (SSH over HTTPS on the | ||
| same host, keeping any `https://` port). A plain `http://` or `git://` URL names | ||
| a different endpoint (a distinct port, and for `http` an unencrypted service), | ||
| so it is skipped rather than silently rewritten to `https://`. | ||
| - **Block-scalar and escaped string entries** — a shorthand written as a YAML | ||
| block scalar (folded `>` / literal `|`) or as a quoted scalar that relies on | ||
| escape sequences (e.g. `"owner/repo\x23v1.0.0"`) decodes to text that is not a | ||
| contiguous slice of the manifest source, so its ref cannot be rewritten in | ||
| place. These uncommon spellings are skipped rather than producing a failing | ||
| update; write the shorthand as a plain or simply-quoted scalar | ||
| (`owner/repo#v1.0.0`) to have it updated. | ||
| - **Lockfile regeneration** — `apm.lock.yaml` is intentionally **left | ||
| unchanged**. Rewriting only its `resolved_ref:` to the new tag would be | ||
| actively harmful: APM's install path compares the manifest ref to | ||
| `resolved_ref` via `detect_ref_change`, and once they match it stops | ||
| detecting drift and replays the *stale* `resolved_commit` — so `apm install` | ||
| would keep installing the old commit. A full regeneration is also not | ||
| possible offline because `content_hash:` (the hash of the materialised | ||
| package tree) cannot be recomputed without downloading and building the | ||
| package. Leaving the entry untouched means the bumped manifest ref no longer | ||
| matches `resolved_ref`, so APM detects the drift and re-resolves the tag — | ||
| regenerating `resolved_commit` and `content_hash` correctly — on the next | ||
| `apm install`. | ||
|
|
||
| These are natural follow-ups and can be layered on without changing the manifest | ||
| parsing model established here. | ||
|
|
||
| ### Running locally | ||
|
|
||
| 1. Start a development shell | ||
|
|
||
| ``` | ||
| $ bin/docker-dev-shell apm | ||
| ``` | ||
|
|
||
| 2. Run tests | ||
| ``` | ||
| [dependabot-core-dev] ~ $ cd apm && rspec | ||
| ``` | ||
|
|
||
| [core-repo]: https://github.com/dependabot/dependabot-core | ||
| [apm-repo]: https://github.com/microsoft/apm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| Gem::Specification.new do |spec| | ||
| common_gemspec = | ||
| Bundler.load_gemspec_uncached("../common/dependabot-common.gemspec") | ||
|
|
||
| spec.name = "dependabot-apm" | ||
| spec.summary = "Provides Dependabot support for APM (Agent Package Manager)" | ||
| spec.description = "Dependabot-Apm provides support for bumping APM (Agent Package Manager) " \ | ||
| "dependencies via Dependabot. " \ | ||
| "If you want support for multiple package managers, you probably want the meta-gem " \ | ||
| "dependabot-omnibus." | ||
|
|
||
| spec.author = common_gemspec.author | ||
| spec.email = common_gemspec.email | ||
| spec.homepage = common_gemspec.homepage | ||
| spec.license = common_gemspec.license | ||
|
|
||
| spec.metadata = { | ||
| "bug_tracker_uri" => common_gemspec.metadata["bug_tracker_uri"], | ||
| "changelog_uri" => common_gemspec.metadata["changelog_uri"] | ||
| } | ||
|
|
||
| spec.version = common_gemspec.version | ||
| spec.required_ruby_version = common_gemspec.required_ruby_version | ||
| spec.required_rubygems_version = common_gemspec.required_ruby_version | ||
|
|
||
| spec.require_path = "lib" | ||
| spec.files = Dir["lib/**/*"] | ||
|
|
||
| spec.add_dependency "dependabot-common", Dependabot::VERSION | ||
|
|
||
| common_gemspec.development_dependencies.each do |dep| | ||
| spec.add_development_dependency dep.name, *dep.requirement.as_list | ||
| end | ||
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # typed: strict | ||
| # frozen_string_literal: true | ||
|
|
||
| # These all need to be required so the various classes can be registered in a | ||
| # lookup table of package manager names to concrete classes. | ||
| require "dependabot/apm/file_fetcher" | ||
| require "dependabot/apm/file_parser" | ||
| require "dependabot/apm/update_checker" | ||
| require "dependabot/apm/file_updater" | ||
| require "dependabot/apm/metadata_finder" | ||
| require "dependabot/apm/requirement" | ||
| require "dependabot/apm/version" | ||
|
|
||
| require "dependabot/pull_request_creator/labeler" | ||
| Dependabot::PullRequestCreator::Labeler | ||
| .register_label_details("apm", name: "apm", colour: "0e8a16") | ||
|
|
||
| require "dependabot/dependency" | ||
| Dependabot::Dependency | ||
| .register_production_check( | ||
| "apm", | ||
| # Production when the explicit "dependencies" marker is present (or when a | ||
| # dependency carries no group information at all). Presence-based rather than | ||
| # "not development", so a package merged from both `dependencies.apm` and | ||
| # `devDependencies.apm` -- whose flattened groups include both markers -- | ||
| # stays production, while a `devDependencies`-only entry does not. | ||
| ->(groups) { groups.empty? || groups.include?("dependencies") } | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # typed: strict | ||
| # frozen_string_literal: true | ||
|
|
||
| require "sorbet-runtime" | ||
|
|
||
| require "dependabot/errors" | ||
| require "dependabot/file_fetchers" | ||
| require "dependabot/file_fetchers/base" | ||
| require "dependabot/apm/package_manager" | ||
|
|
||
| module Dependabot | ||
| module Apm | ||
| class FileFetcher < Dependabot::FileFetchers::Base | ||
| extend T::Sig | ||
| extend T::Helpers | ||
|
|
||
| MANIFEST_FILENAME = "apm.yml" | ||
| LOCKFILE_FILENAME = "apm.lock.yaml" | ||
|
|
||
| # Shown when file fetching runs with beta ecosystems disabled. apm is a | ||
| # beta ecosystem, so it only operates when a repo opts in; surfacing the | ||
| # remediation here avoids the misleading "apm.yml not found" message when | ||
| # the manifest actually exists and the real blocker is the disabled flag. | ||
| BETA_DISABLED_MESSAGE = | ||
| "apm is a beta ecosystem. Set `enable-beta-ecosystems: true` in your " \ | ||
| "dependabot.yml so Dependabot fetches and updates apm.yml." | ||
|
|
||
| sig { override.params(filenames: T::Array[String]).returns(T::Boolean) } | ||
| def self.required_files_in?(filenames) | ||
| filenames.include?(MANIFEST_FILENAME) | ||
| end | ||
|
|
||
| sig { override.returns(String) } | ||
| def self.required_files_message | ||
| "Repo must contain an #{MANIFEST_FILENAME} file." | ||
| end | ||
|
|
||
| sig { override.returns(T::Array[DependencyFile]) } | ||
| def fetch_files | ||
| # apm is a beta ecosystem, so file fetching is hidden behind the | ||
| # beta-ecosystems feature flag (see NEW_ECOSYSTEMS.md). When the flag is | ||
| # off, tell the user to enable it rather than claim the manifest is | ||
| # missing, which would hide the real remediation when apm.yml exists. | ||
| unless allow_beta_ecosystems? | ||
| raise Dependabot::DependencyFileNotFound.new( | ||
| File.join(directory, MANIFEST_FILENAME), | ||
| BETA_DISABLED_MESSAGE | ||
| ) | ||
| end | ||
|
|
||
| fetched_files = T.let([manifest_file], T::Array[DependencyFile]) | ||
| fetched_files << T.must(lockfile) if lockfile | ||
| fetched_files | ||
| end | ||
|
|
||
| sig { override.returns(T.nilable(T::Hash[Symbol, T.anything])) } | ||
| def ecosystem_versions | ||
| return unless lockfile | ||
|
|
||
| version = parsed_lockfile_apm_version | ||
| return unless version | ||
|
|
||
| { package_managers: { "apm" => version } } | ||
| end | ||
|
|
||
| private | ||
|
|
||
| sig { returns(Dependabot::DependencyFile) } | ||
| def manifest_file | ||
| @manifest_file ||= T.let( | ||
| fetch_file_from_host(MANIFEST_FILENAME), | ||
| T.nilable(Dependabot::DependencyFile) | ||
| ) | ||
| end | ||
|
|
||
| sig { returns(T.nilable(Dependabot::DependencyFile)) } | ||
| def lockfile | ||
| return @lockfile if defined?(@lockfile) | ||
|
|
||
| @lockfile = T.let( | ||
| fetch_file_if_present(LOCKFILE_FILENAME)&.tap { |f| f.support_file = true }, | ||
| T.nilable(Dependabot::DependencyFile) | ||
| ) | ||
| end | ||
|
|
||
| sig { returns(T.nilable(String)) } | ||
| def parsed_lockfile_apm_version | ||
| PackageManager.version_from_lockfile(lockfile&.content) | ||
| end | ||
| end | ||
| end | ||
| end | ||
|
|
||
| Dependabot::FileFetchers | ||
| .register("apm", Dependabot::Apm::FileFetcher) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.