Skip to content

fix(project): tie evaluator model ARN validation to each resource type's account format - #2134

Merged
notgitika merged 1 commit into
aws:refactorfrom
notgitika:fix/evaluator-model-arn-validation
Aug 27, 2026
Merged

fix(project): tie evaluator model ARN validation to each resource type's account format#2134
notgitika merged 1 commit into
aws:refactorfrom
notgitika:fix/evaluator-model-arn-validation

Conversation

@notgitika

Copy link
Copy Markdown
Contributor

What

Follow-up to #2124 addressing @nborges-aws's review comment on the evaluator model-ID ARN validator.

isValidBedrockModelId's ARN pattern made the account segment optional for both resource types:

/^arn:aws[a-z-]*:bedrock:[a-z0-9-]+:(\d{12})?:(inference-profile|foundation-model)\/.+$/

That fixed the original foundation-model false-negative, but also accepts two impossible shapes: an account-scoped foundation-model and an accountless inference-profile. This ties each resource type to its documented account format instead:

/^arn:aws[a-z-]*:bedrock:[a-z0-9-]+:(?:\d{12}:(?:application-)?inference-profile|:foundation-model)\/.+$/
  • foundation-model → no account: arn:aws:bedrock:us-east-1::foundation-model/…
  • (application-)inference-profile → account required: arn:aws:bedrock:us-east-1:123456789012:inference-profile/…

Also accepts application-inference-profile ARNs, which the previous pattern rejected (a real valid ARN type).

Testing

Extended the existing helper test to assert the two valid ARN shapes + application-inference-profile pass, and the two wrong-account-format combinations are rejected. bun test, lint, format, and typecheck pass.

…e's account format

The prior validator made the account segment optional for both resource
types, which also accepted impossible combinations (account-scoped
foundation-model, accountless inference-profile). Pin each type to its
documented shape: foundation-model ARNs omit the account, while
(application-)inference-profile ARNs carry it. Also accept
application-inference-profile ARNs, which the prior pattern rejected.

Follow-up to aws#2124 (nborges review).
@github-actions github-actions Bot added the size/s PR size: S label Aug 27, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Aug 27, 2026
@notgitika
notgitika marked this pull request as ready for review August 27, 2026 21:16
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Aug 27, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Nice tightening of BEDROCK_ARN_PATTERN in src/projectSchemas/evaluator.ts. I verified the regex behavior manually against the documented ARN shapes:

  • arn:aws:bedrock:<region>::foundation-model/...
  • arn:aws:bedrock:<region>:<12-digit>:inference-profile/...
  • arn:aws:bedrock:<region>:<12-digit>:application-inference-profile/... ✅ (newly supported)
  • Mismatched account formats (e.g. account on foundation-model, or empty account on inference-profile) are now rejected ✅
  • Partitions aws-us-gov and aws-cn still match ✅

Test coverage in evaluator.test.ts exercises both the new positive case (application-inference-profile) and the two rejection cases, which is exactly what this change needs. No telemetry surface is affected, and there's no mocking involved. LGTM.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Aug 27, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.26%. Comparing base (7bf13af) to head (77f2074).
⚠️ Report is 1 commits behind head on refactor.

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2134   +/-   ##
=========================================
  Coverage     97.26%   97.26%           
=========================================
  Files           469      469           
  Lines         28569    28569           
=========================================
  Hits          27787    27787           
  Misses          782      782           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@notgitika
notgitika merged commit fadad5a into aws:refactor Aug 27, 2026
22 of 25 checks passed
@notgitika
notgitika deleted the fix/evaluator-model-arn-validation branch August 27, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants