Use ESRP14 for gated NuGet package distribution

doc/release_process.md

@@ -21,19 +21,23 @@ Its package and assembly name is `Microsoft.DurableTask.LargePayloadPurge.Abstra
 standard `Microsoft.DurableTask.*.dll` signing pattern. The existing source traversal, SBOM inclusion,
 NuGet signing and per-package approval-gated publication steps apply.
 
-Contract publication also waits for successful Client and Abstractions publication. If either prerequisite
-fails, including a duplicate-version upload failure, or is skipped or canceled, contract publication is
-skipped rather than treating that result as success.
+Contract publication also waits for successful Client and Abstractions jobs. Each prerequisite must
+either publish successfully or complete an explicit successful no-op because the operator confirmed its
+selected artifact version is already on NuGet.org using `alreadyPublishedPackages`. If either job fails,
+including a duplicate-version upload failure, or is skipped or canceled, contract
+publication is skipped rather than treating that result as success.
 
 `Microsoft.DurableTask.Extensions.AzureBlobPayloads` depends on the contract package at every target
 framework, so its publication job also waits for successful contract publication (in addition to approval).
-If contract publication fails for any reason above, including a duplicate-version upload failure, or is
-skipped or canceled, Blob publication is skipped rather than treating that result as success. This extends
-the same fail-closed chain: approval, then Client and Abstractions, then the contract, then Blob. A
-prerequisite that deliberately skips — for example because its exact version is already published — still
-skips the dependent job; this is not a general-purpose publication-idempotency mechanism, just the minimum
-ordering these two packages require. Other packages retain their own independent publication jobs gated
-only on approval.
+If the contract job fails for any reason above, including a duplicate-version upload failure, or is
+skipped or canceled, Blob publication is skipped rather than treating that result
+as success. This extends the same fail-closed chain: approval, then Client and Abstractions, then the
+contract, then Blob. An already-published confirmation succeeds without staging or submitting that
+package, allowing recovery of a partial release: confirm Client/Abstractions to retry the contract and
+Blob. For a Blob-only retry, confirm all selected versions already published, including Client,
+Abstractions and the contract; their jobs still follow the dependency chain but complete as no-ops.
+The confirmation list defaults to empty and applies only to the selected run. Other packages retain
+their own independent publication jobs gated only on approval.
 
 Keep its `RELEASENOTES.md`: `eng/targets/Release.targets` reads it into NuGet package metadata, whereas the
 root `CHANGELOG.md` retains repository release history. The shared target also appends a link using the
@@ -117,14 +121,57 @@ After the PR is merged, follow the **Publishing** steps below.
 
 ## Publishing (After Release PR is Merged)
 
-1. Kick off [ADO release build](https://dev.azure.com/durabletaskframework/Durable%20Task%20Framework%20CI/_build?definitionId=29) (use the tag as the build target, enter `refs/tags/<tag>`)
-2. Validate signing, package contents (if build changes were made.)
-3. Create ADO release.
-    - From successful ADO release build, click 3 dots in top right → 'Release'.
-4. Release to ADO feed.
-5. Release to NuGet once validated (and dependencies are also released to NuGet.)
-6. Publish GitHub draft release.
-7. Delete contents of all `RELEASENOTES.md` files.
+1. Select a successful, signed [official build](https://dev.azure.com/azfunc/internal/_build?definitionId=657).
+   The publisher uses its `drop` unchanged; it does not rebuild or repack.
+2. Validate the selected build's source commit, signatures, package contents, and exact versions.
+3. Use [`durabletask-dotnet.publish`](https://dev.azure.com/azfunc/internal/_build?definitionId=1100)
+   (`eng/publish/publish.yml`) with the reviewed publisher source. Select the signed build separately
+   through the `officialPipeline` artifact resource; do not rely on "latest" or assume the release tag
+   contains the current publisher.
+4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding.
+   There is no organization default; an unset or unresolved value fails before any ESRP submission.
+   Set `alreadyPublishedPackages` to exact public package IDs only after verifying their **selected
+   artifact versions** already exist on NuGet.org. Those jobs log a successful no-op, without package
+   staging or an ESRP task, so dependent jobs can proceed.
+5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that
+   succeeds does `nugetApproval` wait for manual approval. The gate displays the selected organization
+   and confirmed-public IDs. Verify onboarding, the exact selected versions and dependency availability
+   before approving. Stop at the gate for validation-only runs.
+   Rejecting or timing out the gate prevents all public publication.
+6. After approval, each public package not already confirmed published is copied unchanged into
+   its own ESRP folder using the exact nuspec ID, then submitted with `EsrpRelease@14` and
+   `WaitForReleaseCompletion: true`.
+   Internal-only/test packages and symbol packages are not submitted. Missing or multiple matching
+   packages fail explicitly. Each package retains its independent job and the dependency gates above;
+   ESRP does not guarantee ordering within a multi-package submission.
+7. Publish GitHub draft release.
+8. Delete contents of all `RELEASENOTES.md` files.
+
+### ESRP NuGet prerequisites
+
+Follow the [ESRP NuGet onboarding guide](https://eng.ms/docs/microsoft-security/identity/trust-and-security-services/tss-release-distribute/tss-release-esrp-parent/oss-publishing/releasing-open-source/nuget).
+Confirm the organization owns the selected packages, the Durable client is allowlisted for
+`ESRPRELPACMAN`, and Trusted Publishing covers the package IDs/prefixes and required operations.
+The template reuses `dtfx-internal-esrp-prod`; access to that connection and appropriate
+[pipeline permissions](https://learn.microsoft.com/azure/devops/pipelines/policies/permissions?view=azure-devops)
+are separate prerequisites. Owners/approvers use the individual `Build.RequestedForEmail`, not a group.
+This source change does not establish those permissions or onboarding. ESRP failures fail the job;
+there is no direct NuGet push fallback.
+
+For Azure DevOps CLI SDK REST requests, `templateParameters` is a string map: serialize
+`alreadyPublishedPackages` as a JSON string rather than a native array. Verify the resolved publisher
+source, artifact resource and confirmed-public versions before running.
+
+### Local publisher checks
+
+For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run:
+
+```powershell
+Invoke-Pester -Path .\eng\publish\publish.Tests.ps1 -CI
+```
+
+These checks execute local staging/no-op scripts and check pipeline wiring, not Azure's scheduler.
+They do not compile server-side templates or validate end-to-end publishing.
 
 ## Manual Release Process (Alternative)
 

eng/publish/nuget-release-steps.yml

@@ -0,0 +1,97 @@
+parameters:
+- name: packageId
+  type: string
+- name: organizationName
+  type: string
+- name: alreadyPublished
+  type: boolean
+  default: false
+
+steps:
+- ${{ if parameters.alreadyPublished }}:
+  - pwsh: |
+      Write-Host "Operator confirmed the selected version of '$env:PACKAGE_ID' is already published on NuGet.org. No package will be staged or submitted to ESRP."
+    displayName: Already published (${{ parameters.packageId }})
+    env:
+      PACKAGE_ID: ${{ parameters.packageId }}
+
+- ${{ else }}:
+  - pwsh: |
+      $ErrorActionPreference = 'Stop'
+
+      if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') {
+        throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.'
+      }
+
+      $packages = @(
+        foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) {
+          if ($file.Name -like '*.symbols.nupkg') {
+            continue
+          }
+
+          $archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName)
+          try {
+            $nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' })
+            if ($nuspecs.Count -ne 1) {
+              throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)."
+            }
+
+            $reader = [System.IO.StreamReader]::new($nuspecs[0].Open())
+            try {
+              [xml]$nuspec = $reader.ReadToEnd()
+            }
+            finally {
+              $reader.Dispose()
+            }
+
+            if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) {
+              throw "Missing package ID in '$($file.Name)'."
+            }
+
+            if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) {
+              $file
+            }
+          }
+          finally {
+            $archive.Dispose()
+          }
+        }
+      )
+
+      if ($packages.Count -ne 1) {
+        throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)."
+      }
+
+      if (Test-Path -LiteralPath $env:STAGING_FOLDER) {
+        throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content."
+      }
+
+      New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null
+      Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER
+      Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'."
+    displayName: Stage ${{ parameters.packageId }} for ESRP
+    env:
+      PACKAGE_ID: ${{ parameters.packageId }}
+      NUGET_ORGANIZATION: ${{ parameters.organizationName }}
+      SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop
+      STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
+
+  - task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14
+    displayName: ESRP NuGet release (${{ parameters.packageId }})
+    inputs:
+      ConnectedServiceName: dtfx-internal-esrp-prod
+      UseManagedIdentity: true
+      KeyVaultName: durable-esrp-akv
+      SignCertName: dts-esrp-cert
+      ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89
+      DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d
+      Intent: PackageDistribution
+      ContentType: NuGet
+      OrganizationName: ${{ parameters.organizationName }}
+      ContentSource: Folder
+      FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
+      WaitForReleaseCompletion: true
+      Owners: $(Build.RequestedForEmail)
+      Approvers: $(Build.RequestedForEmail)
+      ServiceEndpointUrl: https://api.esrp.microsoft.com
+      MainPublisher: ESRPRELPACMAN