feat(templates): wire up memory into templates - #4
Closed
Hweinstock wants to merge 17 commits into
Closed
Conversation
github-actions
Bot
added
agentcore-harness-reviewing
Hweinstock
force-pushed
the
memory-in-templates
branch
2 times, most recently
from
August 26, 2026 22:25
f07f985 to
9088f13
Compare
Hweinstock
force-pushed
the
refactor-templates-pt2
branch
from
August 27, 2026 01:33
3be3fda to
81da1d7
Compare
Hweinstock
force-pushed
the
memory-in-templates
branch
from
August 27, 2026 01:48
9088f13 to
2f80ed2
Compare
…solution (aws#2122) @aws-cdk/toolkit-lib's yaml-cfn.js does require("yaml/types") at import time, a subpath that only exists in yaml v1. When an install topology resolves toolkit-lib's yaml to the hoisted yaml v2 (whose exports map blocks ./types), the module fails to load with ERR_PACKAGE_PATH_NOT_EXPORTED ("Cannot find module 'yaml/types'"). This bites bun compile, which inlines toolkit-lib and embeds whatever the build machine resolves. Add a nested override forcing toolkit-lib's yaml to ^1 so it always resolves the v1 nested copy that ships yaml/types. Co-authored-by: gitikavj <gitikavj@amazon.com>
* feat: persist minimal deploy state to top-level deployed-state.json Move the deploy-state file from agentcore/.cli/deployed-state.json to a committed agentcore/deployed-state.json, and stop storing a full snapshot of every resource. State is now keyed per target and holds only the deployed CloudFormation stack ARN (captured from the toolkit deploy result) plus the imperatively-created credential ARNs the synth step needs. Everything else is read live from CloudFormation, so the file never goes stale. Adds a DeployedState schema with readDeployedState/updateTargetState (merge-not-clobber, preserving sibling targets and unowned keys), surfaces stackArn from the CDK toolkit runner, and points the vended CDK app at the new path. * fix: harden deployed-state integrity (review feedback) Address review of the deployed-state work: - Validate any existing state before deploy, so a malformed file fails before AWS is mutated rather than after (leaving the new stack ARN unrecorded). The vended app likewise only treats a missing file as absent and surfaces a malformed one. - Require a stack ARN on a deploy result; a successful CDK deploy always has one, so its absence is malformed -- fail instead of silently skipping persistence. - Write the state file atomically (temp + rename) so an interruption can't leave unparseable JSON that blocks later deploys. - Passthrough the credential-entry schema so a stack-ARN-only rewrite doesn't strip fields a newer CLI records. - Qualify the merge guarantee: safe for sequential deploys, not concurrent. * refactor: keep deployed-state under agentcore/.cli/ Move the state file back under agentcore/.cli/ to match the released CLI's location, so a project created by an older CLI keeps reading the same path after upgrading (the vended app isn't re-vended on deploy). The scaffolded .gitignore ignores the rest of .cli/ but re-includes deployed-state.json, so the stack binding + credential ARNs stay committed and shared. --------- Co-authored-by: gitikavj <gitikavj@amazon.com>
* feat: read project stack state live from CloudFormation Adds a reader that describes a project's CloudFormation stack and classifies its lifecycle into not-deployed / in-progress / failed / ready, returning the stack outputs (resource ARNs/IDs) only when settled and successful. This is the source-of-truth side of the deploy-state refactor: resource details come from CloudFormation on demand rather than a local snapshot that can go stale. Generalizes the existing bootstrap not-found helper to isStackNotFound and reuses it. No command is wired to this yet; project status consumes it in a follow-up. * refactor: trim reader to the raw DescribeStacks API Per review, drop the stack-status classification (not-deployed / in-progress / failed / ready) and the StackState shape — that's a project status interface decision and belongs with whoever builds it, not baked in ahead of the consumer. Keep just describeStack: a DescribeStacks call that returns the stack or undefined when it doesn't exist. The CloudFormation call is injectable at the function seam (lazy-loaded like environment.ts), so it's unit-tested without a real client; wiring it through CoreClient/the project manager is left to the consumer. * fix: throw on an empty successful DescribeStacks response A missing stack is reported by a thrown ValidationError, so that stays the only not-found (undefined) signal. A successful response with no stack is malformed, not not-found; return undefined there would misreport a service problem as 'not deployed'. Throw MalformedServiceResponseError instead, matching the bootstrap reader. --------- Co-authored-by: gitikavj <gitikavj@amazon.com>
* feat: support imperative eval recommendation command * chore: leverage shared utils + delete stale tests * fix: create recommendation input interface + update required flag handling
Hweinstock
force-pushed
the
refactor-templates-pt2
branch
from
August 27, 2026 16:43
812bee4 to
917cc91
Compare
…es/sessions (aws#2098) * feat(eval): batch simulate — --ingestion-wait-ms flag + per-example failures/sessions Follow-ups from the batch-evaluation simulate review: - Add --ingestion-wait-ms (default 180000, 0 skips); thread via InvokeDatasetInput.waitIngestionMs. Removes the SIMULATE_INGESTION_WAIT_MS env var — tests pass the value through the input. - runExamples now returns per-item failures (item + error), not a bare count + firstError; invokeDataset surfaces failures: [{ exampleId, error }] so a partial failure names which examples dropped and why. - batch simulate output renders sessions[] (exampleId <-> sessionId join key for a later get) and failures[] (omitted when empty). * chore: drop explanatory comments from batch simulate follow-up * fix: always render failures[] in batch simulate output * test(eval): simulate fixture golden via id seam + stream-aware recorder - inject newSessionId into EvalClient (default randomUUID) so replay fixtures + goldens are deterministic - teach makeRecordingSend to freeze/revive a streaming SDK response (InvokeAgentRuntime), which stringify couldn't serialize - add simulate fixture-golden case; move handler edges to batch-evaluation.test.tsx - split invokeDataset.test.ts into run.test.ts (pool) + load.test.ts (parse + GT-shape); delete it and simulate.test.tsx * test(eval): drop run.test.ts + load.test.ts * chore: drop explanatory comments from the simulate test refactor * test(eval): simulate fixture asserts via matchGolden only (drop redundant expects) --------- Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>
* feat(templates): add support for dynamic templates * docs(assets): update readme reccomended commands * feat(runtime): wire up runtime version * fix(templates): use key function to key templates * docs(templates): fix comment with missing the * docs(templates): fix incorrect wording on template types * fix(manager): render in correct directory * test(template): add a snapshot test for the new template * fix(template): avoid double nesting runtimes * fix(templates): remove dead parameters * refactor(harness): adapt harness to leverage template resolver * refactor(manager): rename project template to project tree to avoid confusion * fix(renderer): add missing handlebar helpers * fix(templates): add harnesses to merge entries * fix(handlebars): add missing helpers from upstream * fix(harness): address hardcoded path * fix(hello-world): reject non-HTTP protocol on template * docs(strands-python): remove non-existent command from readme * feat(template): install runtime dependencies in scaffolding * fix(python): normalize names before being sent to python templates * fix(templates): validate dockerfile for harness/ * fix(handlers): add strands as handler input * fix(spec): avoid writing windows paths
Hweinstock
force-pushed
the
memory-in-templates
branch
from
August 27, 2026 17:37
2f80ed2 to
311ee32
Compare
Hweinstock
force-pushed
the
memory-in-templates
branch
from
August 27, 2026 18:31
311ee32 to
13f6e95
Compare
…ge generation (aws#2121) * feat: project add policy-engine * refactor: simplify add policy-engine slice * feat: attach policy engine to gateways from add policy-engine * refactor: simplify policy-engine attach tests * feat: project add policy with source-aware statement * refactor: simplify add policy slice * feat: project remove policy-engine and policy * refactor: simplify remove slice, validate gateway policy engine references * feat: generate Cedar policies from natural language in project add policy * refactor: handler-owned gateway resolution and shared resource-name rules for --generate * refactor: derive gateway resource name through the shared rule * fix: surface generation findings when no Cedar statement is produced * fix: accept Dogwood policy definition members from generation assets * refactor: final simplify pass across the policy branch * fix: address harness review findings on the policy commands * test: cover the multiple-gateway generate rejection * refactor: extract --generate and PolicyClient to a follow-up PR per review * refactor: colocate policyEngineResourceName with its handler per review * refactor: colocate gatewayResourceName with its handler per review
…ine-insight, batch-insight, gateway-connector) (aws#2108) Co-authored-by: jariy17 <tjariy+jariy17@users.noreply.github.com>
* feat(project): add `project add evaluator llm-as-a-judge` Adds a CLI command to attach a custom LLM-as-a-Judge evaluator to a project. The judge is another LLM prompted with scoring instructions and a rating scale, written into spec.evaluators (deployed as an AWS::BedrockAgentCore::Evaluator by the existing CDK constructs). - New `evaluator` subrouter under `project add` with an `llm-as-a-judge` subcommand. - Flags: --name, --level (SESSION|TRACE|TOOL_CALL), --model (Bedrock id/ARN), --instructions (inline/file:///stdin), --rating-scale, --description, --kms-key-arn, --tags. - --rating-scale accepts either a named preset (1-5-quality, 1-3-simple, pass-fail, good-neutral-bad) or an inline JSON rating scale; presets live beside the subcommand and expand into the schema's numerical/categorical shapes. - --tags is parsed via parseJsonFlagWithSchema against TagsSchema. - Instruction placeholder validation is left to the CreateEvaluator service so the CLI never rejects placeholders the service later adds. - Wire the new `evaluator` resource type through AddResourceInput and FsProjectManager.addResource / toProjectSpecKey. Verified end-to-end: preset and inline-JSON rating scales both deploy a real evaluator (CREATE_COMPLETE) and write deployed-state.json. * fix(project): accept foundation-model ARNs without an account segment Bedrock foundation-model ARNs omit the account id (arn:aws:bedrock:<region>::foundation-model/<id>) while inference-profile ARNs include it. Make the account segment optional in the evaluator model-id ARN validator so a valid foundation-model ARN is not rejected. --------- Co-authored-by: gitikavj <gitikavj@amazon.com>
* feat(dev): wire the Agent Inspector into project dev (C3) Make the Inspector reachable from the CLI. project dev now runs UI-by-default: resolve a UI port, start the Inspector HTTP server, watch agentcore.json to reload the supervised runtime set live, and open the browser when interactive and not --json. --no-ui keeps the plain single-runtime log stream. Add the two IO leaves the handler needs: openBrowser (best-effort detached launch) and watchFile (debounced single-file watch, closes on abort). Expose the collector's TraceStore to the Inspector by renaming OtelCollector.store to traces so the store is handed over without the Inspector knowing the collector. The Inspector server rides the one AbortController with the collector, supervisor, and watcher, so Ctrl-C tears everything down through one cancellation domain; the collector closes only after runners return so final spans persist. * refactor(dev): apply /simplify cleanup to the Inspector wiring - Extract findFreePort in core/dev/port.ts; resolveDevPort delegates to it and the dev handler's UI port resolution reuses it, deleting the duplicated resolveUiPort helper and its UI_PORT_ATTEMPTS copy of MAX_PORT_ATTEMPTS. - Drop the dead resolvePort ternary: the --port guard already rejects an explicit port with more than one runtime, so flags.port applies directly. - Rewrite the config-watch closure as a linear async function. - Add projectSpecPath/PROJECT_SPEC_RELATIVE_PATH in core/project/fsUtils.ts and route the manager and the watch target through it, so the watched file and the read file resolve from one source. * feat(dev): --no-ui requires an explicit --agent Without the UI there is no lazy per-agent start, so a multi-runtime project must name which one streams to the terminal. * refactor(inspector): name the A2A event extractor for its protocol Rename extractSseEventText to extractA2aEventText; it only handles A2A artifact/status/task event kinds, so the generic SSE name misled. Addresses review feedback on aws#2085. * feat(dev): replace --ui/--no-ui with a --mode enum browser (default, Agent Inspector), headless (one agent in the terminal), and tui as a reserved value for the planned terminal UI. Clearer than a boolean as more modes arrive. Addresses review feedback on aws#2086. * refactor(dev): inject the project manager instead of a reload closure The dev handler took a bespoke reloadRuntimes closure; inject the project manager (narrowed to resolve) like the sibling handlers do, and re-resolve on config change. Addresses review feedback on aws#2086. * fix(dev): hold live-agent edits until restart and await pumps on shutdown setRuntimes no longer overwrites a running or starting agent's definition, so the Inspector never proxies it with metadata that no longer matches the child; the edit is applied on the agent's next start. events() now waits for every live child's pump before ending, so an agent's final spans reach the collector before shutdown closes it. Addresses review feedback on aws#2086.
…e's account format (aws#2134) The prior validator made the account segment optional for both resource types, which also accepted impossible combinations (account-scoped foundation-model, accountless inference-profile). Pin each type to its documented shape: foundation-model ARNs omit the account, while (application-)inference-profile ARNs carry it. Also accept application-inference-profile ARNs, which the prior pattern rejected. Follow-up to aws#2124 (nborges review). Co-authored-by: gitikavj <gitikavj@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
none