Skip to content

Add scoped Azure Blob payload account access - #171

Merged
Tomer Rosenthal (torosent) merged 6 commits into
mainfrom
azure-blob-token-read-coverage
Nov 3, 2026
Merged

Tomer Rosenthal (torosent) merged 6 commits into
mainfrom
azure-blob-token-read-coverage

Conversation

@berndverst

@berndverst Bernd Verst (berndverst) commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Replace the unshipped AllowedHosts option with independent AllowAllContainersInConfiguredAccount and AdditionalAccountURLs settings.
  • Keep reads limited to the configured endpoint and container by default. The boolean permits all primary-account containers with existing identity, key, or SAS authentication. Additional HTTPS account endpoints permit all their containers using the primary token credential, without widening the primary container scope.
  • Match scheme, hostname, effective port, and SDK-recognized path-style account paths. Validate and snapshot account configuration, deduplicate equivalent additional endpoints, and verify constructed blob destinations before credential acquisition or HTTP requests. Disable redirects and preserve Storage authorization errors without account or credential fallback.
  • Preserve v2-by-default writes and v1/v2 reads. Ignore legacy v2 token query and fragment fields while retaining configured authentication. Uploads stay at the primary endpoint and configured container and emit credential-free tokens.
  • Apply the same read policy to native tokens, generic Resolve locations, and client/worker hydration. Keep samples on the default scope and document the options and migration behavior.
  • Expand offline real-pipeline coverage for access-scope combinations, authentication modes, endpoint matching, configuration validation and mutation, escaped names, legacy query handling, uploads, redirects, and zero-request/zero-token denials.

Validation

  • go test ./payload ./internal/largepayload ./client ./durabletaskscheduler
  • go test ./... -run '^$' (compile all packages)

Document host-wide AllowedHosts behavior and cover default, opt-in, and connection-string payload reads with offline tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 08:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the Azure Blob read-boundary documentation and offline test additions.

No open issues 🎉

What changed in this PR

Clarifies Azure Blob payload read boundaries and adds offline coverage for existing behavior without changing runtime logic.

Changes:

  • Documents hostname-wide AllowedHosts permissions, identity requirements, and default endpoint/container restrictions.
  • Adds real Azure client pipeline tests for v1/v2 reads, allowed destinations, and connection-string authentication.
  • Verifies rejected references trigger neither HTTP requests nor token acquisition.
File Description
payload/​azure_blob.go Expands AllowedHosts API documentation and security guidance.
payload/​azure_blob_access_test.go Adds offline tests for read boundaries and authentication behavior.
durabletaskscheduler/​README.md Explains opt-in native token reads versus restricted generic Resolve locations.

Review effort: Balanced

Replace hostname permissions with independent primary-container and additional-account options. Preserve v1/v2 token compatibility, align legacy query handling, and add offline pipeline coverage and documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 18:48
@berndverst Bernd Verst (berndverst) changed the title Expand Azure Blob payload token read coverage Add scoped Azure Blob payload account access Nov 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Go’s transparent gzip decompression bypasses the compressed-length check in Azure Blob downloads, allowing responses larger than twice the payload limit to be accepted when their decoded payload is small.

1 open issue

Review effort: Balanced

Comment thread payload/azure_blob.go Resolved
Replace two conditional chains to satisfy gocritic without changing payload access behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 19:15
Keep v1 literal names and v2 escaped names unchanged through the Storage pipeline while retaining account, container, and traversal validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the Azure Blob read-access scope changes.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 3, 2026 19:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Automatic HTTP gzip decompression strips headers needed for the compressed-size check, allowing oversized blobs to bypass the limit and consume unbounded bandwidth and decompression work.

1 open issue

Review effort: Balanced

Disable transparent HTTP gzip decompression while retaining redirect rejection, and cover explicit decoding and payload size limits through local TLS pipeline tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 19:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unchecked type assertion in payload/azure_blob.go:115 can panic whenever an application replaces http.DefaultTransport with a tracing or proxy wrapper.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread payload/azure_blob.go Resolved
Avoid depending on the concrete type of http.DefaultTransport while retaining proxy support, connection defaults, explicit decompression, and redirect rejection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 21:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the Azure Blob read-scope changes.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@YunchuWang wangbill (YunchuWang) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the scoped Azure Blob payload implementation at 0f45b23, including the latest private-transport change. The earlier gzip decompression and DefaultTransport assertion findings are addressed, and I found no additional blocking issue in the reviewed changes. Current-head required CI remains a separate merge gate.

@torosent
Tomer Rosenthal (torosent) merged commit c19f832 into main Nov 3, 2026
9 checks passed
@torosent
Tomer Rosenthal (torosent) deleted the azure-blob-token-read-coverage branch November 3, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants