Skip to content

Use ESRP14 for gated NuGet package distribution - #823

Closed
wangbill (YunchuWang) wants to merge 6 commits into
mainfrom
yunchuwang-esrp-nuget-publishing
Closed

wangbill (YunchuWang) wants to merge 6 commits into
mainfrom
yunchuwang-esrp-nuget-publishing

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

  • Use ESRP14 for public NuGet distribution through a shared, explicitly @self-qualified template. Preserve internal feed staging, the manual approval, per-package submissions and dependency order.
  • Keep nuGetOrganization explicit and alreadyPublishedPackages empty by default. Confirmed-public versions complete a logged successful no-op without staging or ESRP submission; other packages are selected by exact nuspec ID and copied unchanged.
  • Remove the unnecessary holdback parameter, local scheduler simulation and trial diary. Restore the original job success conditions, keep direct script tests, and shorten the release guide and approval instructions.

Why is this change needed?

  • Direct public pushes are blocked by CFSClean isolation. ESRP14 supports distribution of the existing signed packages without changing SDK payloads or bypassing release approval.

Issues / work items


Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to release_notes.md
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: N/A — no SDK/API/package changes.
      • Migration guidance: N/A for SDK consumers; publisher inputs are documented in the release guide.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: eng/publish/publish.yml, eng/publish/nuget-release-steps.yml, eng/publish/publish.Tests.ps1, doc/release_process.md.
  • What you changed after AI output: Simplified scope after review and checked the retained staging, recovery and approval behavior. Human attestations below are left unchanged.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Result: 17 focused tests passed, 0 failed with Invoke-Pester -Path .\eng\publish\publish.Tests.ps1 -CI. Tests execute the actual staging/no-op scripts and check migration wiring; they do not simulate Azure's scheduler.
  • git -c core.whitespace=trailing-space,space-before-tab,cr-at-eol diff --cached --check — passed.
  • YAML/base comparison — passed: original 19 jobs, public success conditions, dependency/artifact graph and internal staging retained; helper unchanged by cleanup.
  • Parent preview-only checks passed defaults, confirmed Generator and Blob-only recovery (17/16/1 ESRP tasks, 0/1/16 no-ops): ADO expanded the exact candidate root override against the byte-identical actual mirrored helper. The candidate itself was not mirrored or executed.
  • Full SDK/application tests were not rerun for this publisher-only cleanup.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, .NET version, components): Windows, PowerShell 7, Pester 5, powershell-yaml; no SDK rebuild or signing.
  • Steps + observed results:
    1. Executed local staging against the approved 18-package signed drop.
    2. Confirmed 16 candidate copies match original SHA256s; all 18 originals remain unchanged, with internal/test/symbol packages excluded.
    3. Executed the Generator confirmation no-op; no package staged and no ESRP task invoked.
  • Evidence (optional): Detailed local and preview receipts retained in session artifacts.

Notes for reviewers

  • ESRP end-to-end publishing remains untested. Preview-only checks do not establish runtime access, onboarding, organization, service-connection or Trusted Publishing permissions; confirm these before approval.
  • The gate explicitly warns that approval starts real public submissions. No merge, live pipeline, approval, permission change or package publication is part of this cleanup.

Stage one exact public nuspec ID per ESRP submission, preserve internal feed staging and the manual/dependency gates, and require an explicit onboarded NuGet organization. Add run-scoped package exclusions and focused staging checks without changing SDK package payloads.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 22, 2026 16:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Production publishing requires human validation of server-side template compatibility, service-connection authorization, and ESRP/NuGet onboarding.

1 open issue
What changed in this PR

Migrates the Durable Task SDK’s public NuGet publishing path to ESRP14 while retaining signed artifacts, internal feed staging, and manual approval.

Changes:

  • Replaces 17 direct public push tasks with a shared template that stages packages by exact nuspec ID.
  • Adds an explicit NuGet organization input and run-scoped package exclusions.
  • Adds local staging tests and documents onboarding prerequisites and controlled-trial procedures.
File Description
eng/​publish/​publish.yml Integrates ESRP submissions and exclusions; makes approval wait for internal staging.
eng/​publish/​publish.Tests.ps1 Tests pipeline structure, package selection, unchanged copies, and staging failures.
eng/​publish/​nuget-release-steps.yml Adds exact-ID staging and shared ESRP14 distribution settings.
doc/​release_process.md Documents release selection, prerequisites, trial exclusions, and local validation.

Review effort: Balanced

Comment thread eng/publish/publish.yml Outdated
Display the selected organization at the manual gate and require onboarding, publisher allowlisting and Trusted Publishing confirmation before public submission. Clarify automatic mirror filters versus separately authorized manual mirroring without changing mirror policy or invoking a workflow.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 22, 2026 16:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Server-side pipeline validation and Durable-specific ESRP/NuGet authorization remain unverified for this public-release path.

1 open issue

Review effort: Balanced

Use /eng/publish/nuget-release-steps.yml@self for all 17 public-package jobs so the nested steps resolve in the original repository when expanded by external 1ES templates. Add a regression demonstrated failing on the former relative references. Leave staging logic, gates, profile inputs and package bytes unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 22, 2026 16:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Excluding already-published prerequisites also suppresses selected dependent packages, preventing partial-release recovery.

1 open issue

Review effort: Balanced

Clarify the verified REST string-map serialization for skipPackages and the narrow Edit queue build configuration action required for nondefault pinned inputs. Record actual-source compilation at261b and the pre-run TF215106 authorization blocker without changing runtime publisher defaults, helpers, tests or permissions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 22, 2026 17:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

This public distribution migration requires human verification of NuGet onboarding, service-connection authorization, queue permissions, and Trusted Publishing.

1 open issue

Review effort: Balanced

@YunchuWang

Copy link
Copy Markdown
Member Author

Varshitha Bachu (@bachuv) Queue-configuration permission unblock for the gated ESRP trial

Could you route the narrow Azure DevOps authorization request in this PR's Notes for reviewers to an authorized azfunc/internal administrator? The correctly pinned publisher 1100 request is denied with TF215106 for Edit queue build configuration (EditPipelineQueueConfigurationPermission, bit 65536). Only that pipeline-scoped grant is requested, not a broad administrator role or a policy/default/identity workaround.

The publisher source and explicit trial inputs server-compile; final public NuGet approval remains with Bill. No public submission has been approved or executed. Connection authorization and NuGet onboarding remain separate checks. The PR also documents the two hosted-CI failures in unchanged C# tests for maintainer triage; no assertions or timeouts have been weakened.

Add alreadyPublishedPackages confirmations that emit only a logged successful no-op, without staging or ESRP submission. Keep skipPackages as intentional holds with precedence over confirmations so existing success, approval, failure and cancellation gates remain fail-closed. Preserve the 19-job graph, root-qualified templates and unchanged publishing branch. Add functional partial-recovery tests and update trial/operator guidance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Final recovery and holdback variants still need server-side compilation, and release authorization and ESRP/NuGet onboarding remain unverified.

1 open issue

Review effort: Balanced

Remove the unneeded intentional-hold input and restore the original job success conditions. Keep confirmed-public no-ops, exact signed-package staging and the manual approval safeguard. Replace the local scheduler simulation with direct script tests and migration invariants, and remove trial history from the release guide.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c12fa023-523f-4bee-8664-769050035626
Copilot AI balanced review requested due to automatic review settings October 27, 2026 17:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The reviewed source lacks the promised intentional-hold controls needed to prevent held packages and their dependents from publishing.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread eng/publish/publish.yml
Comment on lines +10 to +13
- name: alreadyPublishedPackages
displayName: Public package IDs whose selected versions are confirmed on NuGet.org (successful no-op)
type: object
default: []
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants