Skip to content

feat(tools): add WebSearchClient for invoking Amazon Web Search - #658

Merged
sundargthb merged 3 commits into
mainfrom
feat/web-search-client
Sep 8, 2026
Merged

feat(tools): add WebSearchClient for invoking Amazon Web Search#658
sundargthb merged 3 commits into
mainfrom
feat/web-search-client

Conversation

@sundargthb

@sundargthb sundargthb commented Sep 3, 2026

Copy link
Copy Markdown
Member

Description

The SDK can already create a web search connector target on a gateway (GatewayClient.create_web_search_target, #656), but there is no way to call the resulting tool from Python. Today anything that is not already an MCP client has to hand-roll the SigV4 signed MCP handshake against the gateway endpoint to use web search.

This adds WebSearchClient:

from bedrock_agentcore.tools import WebSearchClient

client = WebSearchClient(region="us-east-1", gateway_id="my-gateway-abc123")
for result in client.search("what is amazon bedrock agentcore", max_results=5):
    print(result.title, result.url)

Design notes

No new dependency. The transport sits behind a WebSearchBackend seam. GatewayMcpBackend speaks only the slice of MCP streamable HTTP that one tool call needs (initialize, the notifications/initialized ack, optionally tools/list, then tools/call) using urllib3 and botocore.auth.SigV4Auth, both already core dependencies. mcp-proxy-for-aws would do the signing for us, but it is currently dev/test only here and it is async, so it would change the SDK's dependency surface and the calling style for one capability. If reviewers would rather take that dependency as a published extra, the backend is one file to swap and the public search() signature does not move.

Two response framings. A gateway may answer a POST with application/json or with text/event-stream. Both are parsed, since which one comes back for a single tools/call is not something the client can assume.

Tool name resolution. Gateway prefixes every tool with the name of the target it came from, delimited by three underscores, so the tool the agent sees is <targetName>___WebSearch, not WebSearch. Passing target_name derives the name directly and skips a round trip. Otherwise tools/list is walked (following nextCursor) and the WebSearch tool is picked, with a clear error when more than one target exposes one.

Validation before the call. query is required and capped at 200 characters, max_results at 1 to 25, per the documented tool schema. The domain and published-date filters require connector version 1.2.0 or later on the target, which the docstring says.

Citations are preserved. WebSearchResult keeps url, title and published_date alongside text, because the acceptable use terms require source citations and links to be retained and displayed in anything surfaced to an end user. Dropping them in the response type would make compliant use harder than it needs to be.

Endpoint safety. get_gateway_mcp_endpoint validates the gateway identifier as a DNS label before interpolating it into the hostname, matching the region validation already in _utils/endpoints.py, so a crafted identifier cannot redirect the request off AWS. The connection header is excluded from signing, which otherwise produces a signature mismatch server-side.

Region handling. The connector is offered in us-east-1, eu-west-1 and ap-northeast-1. Calling from another region logs a warning rather than raising, so a stale constant in the SDK never blocks a call to a region that was added after the release.

Testing

uv run pytest tests -q passes: 3501 passed, 10 skipped, 4 xpassed. 97 new unit tests (81 for the client, 16 for the gateway identifier validation) cover the SigV4 header set, the request sequence, session reuse, both response framings, tool-name resolution including pagination and ambiguity, the isError and JSON-RPC error paths, ARN parsing, and the gateway identifier validation. ruff check and ruff format --check are clean. Statement coverage of the new module is 100%.

Not verified against a live service. The web search connector is enabled per account and this repo's integration test account is not entitled to it, so the invoke path has been exercised only against mocked HTTP responses. The request shapes come from the public tool schema and response format documentation, and the MCP sequence from the streamable HTTP transport spec, but nothing here has completed a real search. tests_integ/tools/test_web_search_client.py is included and will exercise the real path once run against an entitled account with WEB_SEARCH_GATEWAY_ID set; it skips rather than fails without one. Reviewers with an entitled account running that file would be the useful confirmation.

Confirmed with the service team

The API surface this is built on has since been confirmed by the Web Search Tool team, and matches what is implemented here: SigV4 (AWS_IAM) with no search-specific API key or credential; availability in us-east-1, eu-west-1 and ap-northeast-1; access scoped through IAM policies on the gateway ARN rather than any key; and both domainFilter (include and exclude, 100 domains per list) and publishedDateFilter (from/to, ISO-8601 UTC, inclusive) at target and request level from connector version 1.2.0.

Two clarifications from that exchange are now in the docstrings: the caller's credentials need bedrock-agentcore:InvokeGateway on the gateway ARN while the gateway service role needs bedrock-agentcore:InvokeWebSearch on the connector, and request-level include lists intersect with the target-level include list rather than replacing it, so disjoint lists return no results with no error raised.

Checklist

  • Unit tests added
  • Integration tests added (skip without an entitled account and a gateway)
  • Lint and format pass
  • No new dependencies

The SDK can create a web search connector target on a gateway, but there is no
way to call the resulting tool from Python. Anything that is not already an MCP
client has to hand-roll the SigV4 signed MCP handshake to use it.

WebSearchClient closes that gap:

    client = WebSearchClient(region="us-east-1", gateway_id="my-gateway-abc123")
    for result in client.search("what is agentcore", max_results=5):
        print(result.title, result.url)

Details:

- Transport sits behind a WebSearchBackend seam. GatewayMcpBackend speaks the
  slice of MCP streamable HTTP that one tool call needs (initialize, the
  initialized notification, optionally tools/list, then tools/call) using
  urllib3 and botocore.auth.SigV4Auth, so no new dependency is added. If the
  direct web search API arrives later, it is a second backend behind the same
  search() signature.
- Both response framings are handled, since a gateway may answer a POST with
  application/json or with text/event-stream.
- Tool name resolution accounts for Gateway prefixing every tool with its
  target name: target_name derives "<target>___WebSearch" directly, otherwise
  tools/list is walked (following nextCursor) and the WebSearch tool is picked,
  erroring when the choice is ambiguous.
- Inputs are validated against the documented limits before the call: query is
  required and capped at 200 characters, max_results at 1 to 25. Domain and
  published-date filters need connector version 1.2.0 or later on the target.
- Results are returned as WebSearchResult with url, title and published_date
  retained, because citations have to be displayable in anything shown to an
  end user.
- get_gateway_mcp_endpoint validates the gateway identifier as a DNS label
  before interpolating it into the hostname, matching the existing region
  validation, so a crafted identifier cannot redirect the request off AWS.
- A region outside the connector's availability warns instead of failing, so a
  stale constant never blocks a call to a newly added region.
@sundargthb
sundargthb requested a review from a team September 3, 2026 19:32
… needs

Web search takes no API key of its own: the caller's credentials need
bedrock-agentcore:InvokeGateway on the gateway ARN and the gateway service role
needs bedrock-agentcore:InvokeWebSearch on the connector. AccessDenied is almost
always the first of those, so say so where a caller will look.

Also spell out how request filters compose with the target's own domain rules.
A result is dropped if its domain is on either exclude list, and returned only
if it is on every include list that is set, so passing include_domains against a
target that already has an include list narrows to the intersection and disjoint
lists return nothing. That empty result is silent rather than an error, which is
worth knowing before debugging it.
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

✅ No Breaking Changes Detected

No public API breaking changes found in this PR.

@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Sep 3, 2026
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@tejaskash tejaskash left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the transport layer against the MCP streamable HTTP spec. Tests pass locally. Inline comments below, ordered roughly by severity. The first three matter once this runs against a real gateway.

Comment thread src/bedrock_agentcore/tools/web_search_client.py Outdated
Comment thread src/bedrock_agentcore/tools/web_search_client.py Outdated
Comment thread src/bedrock_agentcore/tools/web_search_client.py
Comment thread src/bedrock_agentcore/tools/web_search_client.py
Comment thread src/bedrock_agentcore/tools/web_search_client.py Outdated
Comment thread src/bedrock_agentcore/tools/web_search_client.py Outdated
Comment thread src/bedrock_agentcore/tools/web_search_client.py
Comment thread src/bedrock_agentcore/tools/web_search_client.py Outdated
Comment thread src/bedrock_agentcore/tools/__init__.py
Comment thread tests/bedrock_agentcore/tools/test_web_search_client.py Outdated
Match every JSON-RPC reply to the id of the request it answers, so a server
notification arriving ahead of the reply is not read as the answer, and require
a reply to carry a result or an error. A JSON-RPC error is still surfaced
whatever id it carries, since the spec allows a null id there.

Parse SSE the way the specification defines it: consecutive data lines of one
event join with a newline and a blank line ends the event, so a message split
across lines decodes instead of being dropped.

Wrap urllib3 transport failures in WebSearchError, so a connection drop or a
read timeout cannot escape as a urllib3 exception from a method documented to
raise WebSearchError.

Treat HTTP 404 against a session we hold as the spec's signal that the session
is gone, and drop it so the next call hands shake again. Roll the initialized
flag back if the notifications/initialized POST fails, rather than leaving the
client claiming a session the gateway never acknowledged.

Apply the SDK's endpoint host check to a caller-supplied gateway_endpoint,
since signed requests carry the caller's credentials either way.

Reject a domain list longer than the documented maximum of 100 before calling,
create the default boto3 session only when a region has to come from one,
export GatewayMcpBackend, type _parse_gateway_arn as Tuple[str, str] and drop
DEFAULT_TARGET_NAME, which duplicated a default that belongs to the gateway
helper.

Tests: replies are now numbered from the request they answer, so the fixtures
no longer hard-code ids that did not match. Adds coverage for each fix above.
Drops the signing test that only asserted botocore's own behavior.
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 6, 2026
@sundargthb
sundargthb requested a review from tejaskash September 6, 2026 23:08
sundargthb pushed a commit to aws/bedrock-agentcore-sdk-typescript that referenced this pull request Sep 6, 2026
Applies the findings from the review of the equivalent Python client
(aws/bedrock-agentcore-sdk-python#658) to this one, since both speak the
same subset of MCP streamable HTTP.

- Parse SSE per the specification. Consecutive data lines belonging to one
  event are joined with a newline and one leading space after the colon is
  framing, so a reply split over several lines decodes instead of being
  dropped. A final event with no trailing blank line is still read.
- Match a reply to the id of the request it answers, so a server
  notification arriving ahead of the reply is not taken for the answer. A
  JSON-RPC error is always surfaced, since it may carry a null id.
- Raise on a reply that carries neither a result nor an error, rather than
  reading a missing result as an empty one.
- Wrap a fetch rejection, meaning a refused connection, a DNS or TLS
  failure or an expired timeout, in WebSearchError with the original kept
  as cause. WebSearchError now takes ErrorOptions.
- Forget the MCP session on HTTP 404 with a session held, which is the
  transport's signal that the session is gone, so the next search redoes
  the handshake.
- Set initialized before the initialized notification, because every later
  request carries mcp-protocol-version, and roll it back if that POST
  fails.
- Check that an endpoint resolves to an AWS host before anything is signed
  for it, for a URL built from a gateway id and for one passed in. The
  BEDROCK_AGENTCORE_GATEWAY_ENDPOINT override stays as given so a local
  mock can still be used.
- Drop DEFAULT_TARGET_NAME, which nothing read.

@tejaskash tejaskash left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the first three items in my inline review above (#658 (review)):

  1. SSE decoder returns the first JSON-RPC message rather than the response matching the request id (line 459).
  2. urllib3 transport exceptions escape _post despite the WebSearchError contract in the docstring (line 314).
  3. An expired session (HTTP 404) leaves the backend permanently failing until close() (line 327).

The remaining comments are non-blocking.

@tejaskash tejaskash left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 12 items addressed at 0ed088e. Verified locally: 171 tests pass, and I exercised _decode_jsonrpc directly with a notification ahead of the reply, a reply to another id, and a message split across two data: lines. The reordered _initialized handling with rollback is the right call given the protocol version header dependency. Approving.

@sundargthb
sundargthb merged commit cf71b14 into main Sep 8, 2026
67 of 70 checks passed
@sundargthb
sundargthb deleted the feat/web-search-client branch September 8, 2026 17:02
sundargthb added a commit to aws/bedrock-agentcore-sdk-typescript that referenced this pull request Sep 9, 2026
* feat(tools): add WebSearchClient for AgentCore Web Search

Web search is reachable today as an AgentCore Gateway connector target,
which an agent calls as an MCP tool. There was no way to reach it from
this SDK, so a TypeScript caller had to hand-roll SigV4 signing plus a
JSON-RPC MCP conversation.

WebSearchClient wraps that behind a plain search(query, options) that
returns typed results. The transport sits behind a WebSearchBackend
interface so a different access path can be added later without changing
callers.

Also adds getGatewayMcpEndpoint to _utils/endpoints, mirroring
get_gateway_mcp_endpoint in the Python SDK.

* fix(tools): harden the web search MCP transport

Applies the findings from the review of the equivalent Python client
(aws/bedrock-agentcore-sdk-python#658) to this one, since both speak the
same subset of MCP streamable HTTP.

- Parse SSE per the specification. Consecutive data lines belonging to one
  event are joined with a newline and one leading space after the colon is
  framing, so a reply split over several lines decodes instead of being
  dropped. A final event with no trailing blank line is still read.
- Match a reply to the id of the request it answers, so a server
  notification arriving ahead of the reply is not taken for the answer. A
  JSON-RPC error is always surfaced, since it may carry a null id.
- Raise on a reply that carries neither a result nor an error, rather than
  reading a missing result as an empty one.
- Wrap a fetch rejection, meaning a refused connection, a DNS or TLS
  failure or an expired timeout, in WebSearchError with the original kept
  as cause. WebSearchError now takes ErrorOptions.
- Forget the MCP session on HTTP 404 with a session held, which is the
  transport's signal that the session is gone, so the next search redoes
  the handshake.
- Set initialized before the initialized notification, because every later
  request carries mcp-protocol-version, and roll it back if that POST
  fails.
- Check that an endpoint resolves to an AWS host before anything is signed
  for it, for a URL built from a gateway id and for one passed in. The
  BEDROCK_AGENTCORE_GATEWAY_ENDPOINT override stays as given so a local
  mock can still be used.
- Drop DEFAULT_TARGET_NAME, which nothing read.

* fix(tools): address review on the web search client

Handshake, signing and discovery fixes from review of #257:

- A second search now waits for the initialized notification instead of
  overtaking it, so a tool call can no longer go out on a session that is
  about to be reset.
- The endpoint query string is part of the canonical request, so a gateway
  URL that carries one signs correctly.
- Endpoints must be HTTPS, since signed headers carry an access key id and a
  session token.
- clientInfo reports the SDK version, read from package.json, rather than
  repeating the protocol version.
- A region that disagrees with the region in a gateway ARN throws instead of
  one of them silently winning, and an ARN for another service is rejected.
- Concurrent first searches share one tools/list, and pagination stops after
  50 pages or on a repeated cursor.
- Dropped the console.warn for an unlaunched region, which callers cannot
  silence. KNOWN_REGIONS is still exported for anyone who wants to check.

---------

Co-authored-by: Sundar Raghavan <sdraghav@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants