Skip to content

fix: resolve the gateway target for A/B test invocation URLs (#1854) - #1874

Merged
jariy17 merged 2 commits into
mainfrom
fix/1854-ab-test-gateway-url
Jul 30, 2026
Merged

fix: resolve the gateway target for A/B test invocation URLs (#1854)#1874
jariy17 merged 2 commits into
mainfrom
fix/1854-ab-test-gateway-url

Conversation

@jariy17

@jariy17 jariy17 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Problem

Closes #1854.

agentcore run ab-test in config-bundle mode printed an invocation URL built from the runtime name, but a gateway invocation path segment must be a gateway target name. Whenever the two differed, every request to the printed URL failed:

{"success":false,"error":"No Target found for Target name: CustomerSupportAB"}

The same wrong URL was stored in .invocationUrl for agentcore view ab-test <id> --json, so scripts inherited it too.

The cause is one line in getInvocationUrl:

return record.agent ? `${baseUrl}/${record.agent}/invocations` : undefined;

record.agent comes from opts.runtime ?? opts.name (handler.ts), i.e. the runtime. It only happened to work when a gateway target was named identically to its runtime.

Why not just look the target up

A config-bundle test has no target of its own to substitute:

  • Its variants carry configurationBundle ARNs, and the schema forbids target on them (target: z.never().optional()).
  • It attaches to the whole gateway — the service splits traffic with a gateway rule keyed on the bundles, which is why the execution role is granted CreateGatewayRule/UpdateGatewayRule.
  • Confirmed against the service: every config-bundle test in a live account stores only configurationBundle per variant, and e2e-tests/ab-test-config-bundle.test.ts creates one on a gateway with no targets at all — it reaches RUNNING fine.

So the path segment is whichever gateway target the caller chooses to invoke. The CLI cannot know it, and a gateway may have several targets fronting one runtime (a canary beside prod), where each is equally valid.

Change

Report the gateway base URL and name what to append, instead of guessing a path that 404s:

Gateway URL: https://<gatewayId>.gateway.bedrock-agentcore.<region>.amazonaws.com
  → append /<gateway-target>/invocations (see `agentcore status --json`)

Target-based tests are unchanged — their variants are targets, so variants[0].targetName is a real path and they keep a complete Invocation URL.

getInvocationUrl stays synchronous and pure, so all three call sites (view --json, printABTestDetail, the TUI detail view) pick this up with no plumbing changes.

Also corrects two comments that documented the bug as intended behaviour:

  • docs/ab-tests.md — "config-bundle uses the agent name"
  • RunABTestFlow.tsx — claimed targets deploy as ${project}-${target}; the L3 CDK deploys each target under its spec name verbatim (Gateway.ts), which is what makes the spec name the URL path segment.

--json shape change

For config-bundle tests only, invocationUrl is replaced by gatewayUrl + invocationUrlHint:

// before — a URL that 404s
{ "invocationUrl": "https://<gw>.../CustomerSupportAB/invocations" }

// after
{ "gatewayUrl": "https://<gw>...",
  "invocationUrlHint": "append /<gateway-target>/invocations (see `agentcore status --json`)" }

This is deliberate: a script reading .invocationUrl now gets nothing rather than a URL that silently fails. Target-based output is untouched. Both docs pages are updated.

Testing

  • 419 test files / 6005 tests pass; tsc clean; eslint/prettier/secretlint clean.
  • 8 tests on the changed code, including a regression asserting the URL never contains the runtime name, and that the config-bundle detail view prints no Invocation URL: line.
  • Verified end-to-end through a bundled + globally installed build (npm run bundlenpm i -g) for both modes, in --json and in the TUI (direct view ab-test <id> and via the interactive list). Config-bundle detail view:
Gateway: arn:aws:bedrock-agentcore:us-west-2:...:gateway/my-gateway-secure-abc123
Gateway filter: none
Gateway URL: https://my-gateway-secure-abc123.gateway.bedrock-agentcore.us-west-2.amazonaws.com
  → append /<gateway-target>/invocations (see `agentcore status --json`)

Notes for reviewers

Two things I found while investigating but did not change, happy to file separately:

  • docs/ab-tests.md documents --max-duration-days and --traffic-header, but neither flag is registered in run/command.tsx and CreateABTestOptions has no field for either.
  • --gateway-filter scopes a config-bundle test to a target path (live tests do carry {"targetPaths":["/orders/*"]}). A future refinement could use a literal filter path as the printed URL's segment, but glob patterns make that ambiguous so I left it out.

`getInvocationUrl` built config-bundle URLs from `record.agent` — the RUNTIME
name — but a gateway invocation path segment must be a gateway TARGET name.
Whenever the two differed, every request to the printed URL failed:

  {"success":false,"error":"No Target found for Target name: CustomerSupportAB"}

A config-bundle test has no target of its own to substitute. Its variants are
configuration bundles, it attaches to the whole gateway, and the service splits
traffic with a gateway rule — so the path segment is whichever gateway target
the caller invokes, which the CLI cannot know. (Confirmed against the service:
config-bundle tests store only `configurationBundle` per variant, and the
config-bundle e2e creates one on a gateway with no targets at all.)

So report the gateway base URL and name what to append, rather than guessing a
path that 404s. Target-based tests are unchanged — their variants *are* targets,
so `variants[0].targetName` is a real path.

  Gateway URL: https://<gatewayId>.gateway.bedrock-agentcore.<region>.amazonaws.com
    → append /<gateway-target>/invocations (see `agentcore status --json`)

`view ab-test --json` reports `gatewayUrl` + `invocationUrlHint` for
config-bundle tests instead of `invocationUrl`, so scripts reading
`.invocationUrl` get nothing rather than a URL that fails. Target-based keeps
`invocationUrl` unchanged.

Also corrects two comments that documented the bug as intended behaviour:
`docs/ab-tests.md` ("config-bundle uses the agent name") and a claim in
RunABTestFlow that targets deploy as `${project}-${target}` — the L3 CDK
deploys each target under its spec name verbatim.
@jariy17
jariy17 requested a review from a team July 30, 2026 15:31
@github-actions github-actions Bot added the size/m PR size: M label Jul 30, 2026
@github-actions github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jul 30, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Jul 30, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Package Tarball

aws-agentcore-0.25.0.tgz

How to install

gh release download pr-1874-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.25.0.tgz

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 40.46% 15123 / 37373
🔵 Statements 39.72% 16123 / 40582
🔵 Functions 34.67% 2591 / 7472
🔵 Branches 33.83% 10079 / 29786
Generated in workflow #4279 for commit a71f153 by the Vitest Coverage Report Action

return targetName ? `${baseUrl}/${targetName}/invocations` : undefined;
}
return record.agent ? `${baseUrl}/${record.agent}/invocations` : undefined;
return baseUrl;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This always returns only the gateway base URL. This breaks working same-name configurations and scripts consuming .invocationUrl, while leaving issue #1854’s workflow manual. Creation already loads gateway targets and runtime mappings, so it should emit a complete URL when one target uniquely matches, and expose candidates or a clear ambiguity when several match.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of this but I wanted to establish to the user that config bundles are applied to all gateway targets. However, I could go either with this.

notgitika
notgitika previously approved these changes Jul 30, 2026

@notgitika notgitika left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a comment otherwise LGTM

Config-bundle tests printed an invocation URL built from the runtime name, but a
gateway path segment must be a gateway TARGET name, so requests 404'd with
"No Target found for Target name: <runtime>".

Resolve the gateway target(s) fronting the runtime at create time — the link
lives in `agentCoreGateways[].targets[].httpRuntime.runtime`, which create()
already loads — and persist the result on the record:

  - exactly one match  -> `targetName`       -> a complete `invocationUrl`
  - several matches     -> `targetCandidates` -> `invocationUrlCandidates` (pick one)
  - none                -> neither            -> `gatewayUrl` + `invocationUrlHint`

This fixes both same-name and different-name configurations (the previous
gateway-base-only output regressed the same-name case and left the workflow
manual), and keeps `.invocationUrl` populated whenever a single target is known.
The runtime name is never used as the path. Target-based tests are unchanged.

`getInvocationUrl` stays synchronous — resolution happens once at create and is
read from the record — so `view --json`, `printABTestDetail`, and the TUI detail
view need no plumbing changes.

Also corrects two comments that documented the bug as intended behaviour.
@github-actions github-actions Bot removed the size/m PR size: M label Jul 30, 2026
@jariy17 jariy17 changed the title fix: stop putting the runtime name in A/B test invocation URLs (#1854) fix: resolve the gateway target for A/B test invocation URLs (#1854) Jul 30, 2026
@github-actions github-actions Bot added the size/m PR size: M label Jul 30, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels Jul 30, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 30, 2026
@jariy17
jariy17 merged commit badd642 into main Jul 30, 2026
35 checks passed
@jariy17
jariy17 deleted the fix/1854-ab-test-gateway-url branch July 30, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agentcore run ab-test outputs incorrect invocation URL (uses runtime name instead of gateway target name)

2 participants