From bf2607c531b437a4c99a336f230a9a1e65ca83b8 Mon Sep 17 00:00:00 2001 From: wangbill Date: Thu, 22 Oct 2026 12:14:16 -0400 Subject: [PATCH 1/6] Use ESRP14 for gated NuGet distribution Stage one exact public nuspec ID per ESRP submission, preserve internal feed staging and the manual/dependency gates, and require an explicit onboarded NuGet organization. Add run-scoped package exclusions and focused staging checks without changing SDK package payloads. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- doc/release_process.md | 79 +++++++- eng/publish/nuget-release-steps.yml | 86 ++++++++ eng/publish/publish.Tests.ps1 | 304 ++++++++++++++++++++++++++++ eng/publish/publish.yml | 284 +++++++++----------------- 4 files changed, 561 insertions(+), 192 deletions(-) create mode 100644 eng/publish/nuget-release-steps.yml create mode 100644 eng/publish/publish.Tests.ps1 diff --git a/doc/release_process.md b/doc/release_process.md index 6ac034a1..9ea62a65 100644 --- a/doc/release_process.md +++ b/doc/release_process.md @@ -117,14 +117,77 @@ After the PR is merged, follow the **Publishing** steps below. ## Publishing (After Release PR is Merged) -1. Kick off [ADO release build](https://dev.azure.com/durabletaskframework/Durable%20Task%20Framework%20CI/_build?definitionId=29) (use the tag as the build target, enter `refs/tags/`) -2. Validate signing, package contents (if build changes were made.) -3. Create ADO release. - - From successful ADO release build, click 3 dots in top right → 'Release'. -4. Release to ADO feed. -5. Release to NuGet once validated (and dependencies are also released to NuGet.) -6. Publish GitHub draft release. -7. Delete contents of all `RELEASENOTES.md` files. +1. Select a successful, signed `durabletask-dotnet.official` build in the `azfunc/internal` project. + Reuse the approved build's `drop`; the publisher does not rebuild, repack, or change package versions. +2. Validate the selected build's source commit, signatures, package contents, and exact versions. +3. Use [`durabletask-dotnet.publish`](https://dev.azure.com/azfunc/internal/_build?definitionId=1100) + (`eng/publish/publish.yml`). The publisher source ref and the `officialPipeline` artifact resource + are separate selections: explicitly pin the official run, not just its branch or "latest". + Use the reviewed publisher-source commit on `main` after the normal code-mirror pipeline has + mirrored it to ADO. Individual user branches are not mirrored; do not expand the mirror allowlist + for a trial. Selecting the old release tag as the publisher source loads the old publisher YAML, + even when the correct signed official artifact run is selected. +4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding. + There is no organization default; an unset or unresolved value fails before any ESRP submission. + Set `skipPackages` to an array of exact public package IDs whose selected versions are already + published or are intentionally held back. This is a per-run exclusion, not a permanent package removal. +5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that + succeeds does `nugetApproval` wait for manual approval. Do not approve until the selected artifacts, + exclusion list, dependency availability, and ESRP prerequisites below have been checked. + Rejecting or timing out the gate prevents all public publication. +6. After approval, each selected public package is copied unchanged into its own ESRP folder using + the exact nuspec ID, then submitted with `EsrpRelease@14` and `WaitForReleaseCompletion: true`. + Internal-only/test packages and symbol packages are not submitted. Missing or multiple matching + packages fail explicitly. Each package retains its independent job and the dependency gates above; + ESRP does not guarantee ordering within a multi-package submission. +7. Publish GitHub draft release. +8. Delete contents of all `RELEASENOTES.md` files. + +### ESRP NuGet prerequisites + +The [ESRP NuGet guide](https://eng.ms/docs/microsoft-security/identity/trust-and-security-services/tss-release-distribute/tss-release-esrp-parent/oss-publishing/releasing-open-source/nuget) +requires channel onboarding and a NuGet.org Trusted Publishing policy for the customer's organization, +ESRP client identity, package IDs or delegated prefixes, and required operations (new IDs and/or new +versions). Confirm organization ownership and any collaborator access required by that onboarding. +Existing ESRP customers use the existing-customer onboarding process to enable the NuGet channel. + +The template reuses the Durable ESRP service connection `dtfx-internal-esrp-prod`, managed identity +`0b3ed1a4-0727-4a50-b82a-02c2bd9dec89`, tenant `33e01921-4d64-4f8c-a055-5bdaffd5e33d`, +vault `durable-esrp-akv`, and signing certificate `dts-esrp-cert`. NuGet distribution uses +`MainPublisher: ESRPRELPACMAN`, not the `durabletask-java` publisher used for other registries. +The client must be allowlisted for this publisher. `Owners` and `Approvers` use the individual +Microsoft identity of the person who queued the run (`Build.RequestedForEmail`), not a group alias. +The publishing pipeline must also be authorized to use this service connection; that ADO authorization +is separate from ESRP publisher allowlisting and NuGet Trusted Publishing permissions. + +**Durable-specific NuGet onboarding, organization choice, publisher allowlisting, and Trusted Publishing +permissions have not been verified by this source migration.** Package owner names alone do not prove +that either owner is the onboarded organization. Confirm these prerequisites with the ESRP/NuGet teams +before approving publication. A failed ESRP submission fails the job; there is no direct NuGet push fallback. + +### v1.27.1 trial selection + +The signed official drop has 18 `.nupkg` files: 14 public stable packages at `1.27.1`, +`ExportHistory` and `ScheduledTasks` at `1.27.1-preview.2`, the unchanged public +`Microsoft.DurableTask.Generators` at `2.1.0-preview.2`, and internal-only +`Microsoft.DurableTask.InProcessTestHost` at `0.2.3-preview.1`. +For this trial, set `skipPackages: [Microsoft.DurableTask.Generators]` because that exact generator +version is already public. This leaves 16 public targets; the test host has no public job. +The default exclusion list is empty so future generator releases remain enabled. +Keep `officialPipeline` pinned to the approved signed run `311120` (build number `20261021.2`, +source `2a409da4ac7dcf03c4f25985337eb4623ea6394e`), while the publisher uses the new reviewed and mirrored +`main` commit containing this migration. A run ID and build number are not interchangeable. +Recheck exact-version absence for every intended target immediately before any submission; +search indexing alone is not proof that a version has never been published. + +For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run: + +```powershell +Invoke-Pester -Path .\eng\publish\publish.Tests.ps1 -CI +``` + +These checks exercise local staging and pipeline structure only. They do not compile the server-side +1ES templates, validate backend permissions, queue a release, or publish a package. ## Manual Release Process (Alternative) diff --git a/eng/publish/nuget-release-steps.yml b/eng/publish/nuget-release-steps.yml new file mode 100644 index 00000000..b61b1bb3 --- /dev/null +++ b/eng/publish/nuget-release-steps.yml @@ -0,0 +1,86 @@ +parameters: +- name: packageId + type: string +- name: organizationName + type: string + +steps: +- pwsh: | + $ErrorActionPreference = 'Stop' + + if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') { + throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.' + } + + $packages = @( + foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) { + if ($file.Name -like '*.symbols.nupkg') { + continue + } + + $archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName) + try { + $nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' }) + if ($nuspecs.Count -ne 1) { + throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)." + } + + $reader = [System.IO.StreamReader]::new($nuspecs[0].Open()) + try { + [xml]$nuspec = $reader.ReadToEnd() + } + finally { + $reader.Dispose() + } + + if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) { + throw "Missing package ID in '$($file.Name)'." + } + + if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) { + $file + } + } + finally { + $archive.Dispose() + } + } + ) + + if ($packages.Count -ne 1) { + throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)." + } + + if (Test-Path -LiteralPath $env:STAGING_FOLDER) { + throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content." + } + + New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null + Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER + Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'." + displayName: Stage ${{ parameters.packageId }} for ESRP + env: + PACKAGE_ID: ${{ parameters.packageId }} + NUGET_ORGANIZATION: ${{ parameters.organizationName }} + SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop + STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} + +- task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14 + displayName: ESRP NuGet release (${{ parameters.packageId }}) + inputs: + ConnectedServiceName: dtfx-internal-esrp-prod + UseManagedIdentity: true + KeyVaultName: durable-esrp-akv + SignCertName: dts-esrp-cert + ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89 + DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d + Intent: PackageDistribution + ContentType: NuGet + OrganizationName: ${{ parameters.organizationName }} + ContentSource: Folder + FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} + WaitForReleaseCompletion: true + Owners: $(Build.RequestedForEmail) + Approvers: $(Build.RequestedForEmail) + ServiceEndpointUrl: https://api.esrp.microsoft.com + MainPublisher: ESRPRELPACMAN diff --git a/eng/publish/publish.Tests.ps1 b/eng/publish/publish.Tests.ps1 new file mode 100644 index 00000000..4057f750 --- /dev/null +++ b/eng/publish/publish.Tests.ps1 @@ -0,0 +1,304 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +BeforeAll { + Import-Module powershell-yaml -ErrorAction Stop + $pipeline = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\publish.yml" -Raw) + $releaseTemplate = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\nuget-release-steps.yml" -Raw) + $stagePackage = [scriptblock]::Create($releaseTemplate.steps[0].pwsh) + $jobs = $pipeline.extends.parameters.stages[0].jobs + $publicJobs = @($jobs | Where-Object job -Like 'nugetRelease_*') + $publicPackageIds = @( + 'Microsoft.DurableTask.Abstractions' + 'Microsoft.DurableTask.Client' + 'Microsoft.DurableTask.Client.Grpc' + 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim' + 'Microsoft.DurableTask.Generators' + 'Microsoft.DurableTask.Analyzers' + 'Microsoft.DurableTask.Grpc' + 'Microsoft.DurableTask.Worker' + 'Microsoft.DurableTask.Worker.Grpc' + 'Microsoft.DurableTask.Client.AzureManaged' + 'Microsoft.DurableTask.Client.AzureManaged.Sandboxes' + 'Microsoft.DurableTask.Worker.AzureManaged' + 'Microsoft.DurableTask.Worker.AzureManaged.Sandboxes' + 'Microsoft.DurableTask.ScheduledTasks' + 'Microsoft.DurableTask.ExportHistory' + 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' + 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' + ) + + function New-TestPackage { + param( + [string]$Folder, + [string]$PackageId, + [string]$Version = '1.27.1', + [string]$FileName = "$PackageId.$Version.nupkg", + [string]$Nuspec = @" + + $PackageId$Version + +"@ + ) + + $path = Join-Path $Folder $FileName + $archive = [System.IO.Compression.ZipFile]::Open($path, [System.IO.Compression.ZipArchiveMode]::Create) + try { + $writer = [System.IO.StreamWriter]::new($archive.CreateEntry("$PackageId.nuspec").Open()) + try { + $writer.Write($Nuspec) + } + finally { + $writer.Dispose() + } + } + finally { + $archive.Dispose() + } + } +} + +Describe 'Publisher pipeline' { + It 'retains exactly the existing 17 public package jobs and signed artifact inputs' { + # Arrange / Act + $actualIds = @($publicJobs | ForEach-Object { $_.steps[0].parameters.packageId }) + + # Assert + $publicJobs.Count | Should -Be 17 + ($actualIds | Sort-Object) -join ',' | Should -Be (($publicPackageIds | Sort-Object) -join ',') + foreach ($job in $publicJobs) { + $packageId = $job.steps[0].parameters.packageId + $job.job | Should -Be ("nugetRelease_" + $packageId.Replace('.', '_')) + $job.templateContext.type | Should -Be 'releaseJob' + $job.templateContext.isProduction | Should -BeTrue + $job.templateContext.inputs[0].pipeline | Should -Be 'officialPipeline' + $job.templateContext.inputs[0].artifactName | Should -Be 'drop' + $job.steps[0].template | Should -Be 'nuget-release-steps.yml' + $job.steps[0].parameters.organizationName | Should -Be '${{ parameters.nuGetOrganization }}' + } + } + + It 'preserves internal staging and waits for it before manual NuGet approval' { + # Arrange + $ado = $jobs | Where-Object job -EQ 'AdoRelease' + $approval = $jobs | Where-Object job -EQ 'nugetApproval' + + # Act / Assert + $pipeline.trigger | Should -Be 'none' + $pipeline.pr | Should -Be 'none' + $pipeline.resources.pipelines[0].source | Should -Be 'durabletask-dotnet.official' + $ado.steps[0].task | Should -Be '1ES.PublishNuget@1' + $ado.steps[0].inputs.publishVstsFeed | Should -Be '3f99e810-c336-441f-8892-84983093ad7f/c895696b-ce37-4fe7-b7ce-74333a04f8bf' + $ado.steps[0].inputs.allowPackageConflicts | Should -BeTrue + $ado.steps[0].inputs.packagesToPush | Should -Be '$(System.DefaultWorkingDirectory)/**/*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' + $approval.dependsOn | Should -Be 'AdoRelease' + $approval.pool | Should -Be 'server' + $approval.steps[0].task | Should -Be 'ManualValidation@1' + $approval.steps[0].inputs.onTimeout | Should -Be 'reject' + } + + It 'retains fail-closed dependency gates and run-scoped exclusions for every public job' { + # Arrange / Act / Assert + foreach ($job in $publicJobs) { + $packageId = $job.steps[0].parameters.packageId + @($job.dependsOn) | Should -Contain 'nugetApproval' + $job.condition | Should -Match ([regex]::Escape("not(containsValue(parameters.skipPackages, '$packageId'))")) + $job.condition | Should -Match '^and\(succeeded\(' + } + + $contract = $jobs | Where-Object job -EQ 'nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions' + ($contract.dependsOn -join ',') | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_Abstractions,nugetRelease_Microsoft_DurableTask_Client' + $contract.condition | Should -Match '^and\(succeeded\(\),' + $blob = $jobs | Where-Object job -EQ 'nugetRelease_Microsoft_DurableTask_Extensions_AzureBlobPayloads' + ($blob.dependsOn -join ',') | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions' + $blob.condition | Should -Match '^and\(succeeded\(\),' + } + + It 'requires an explicit organization and leaves future generator releases enabled by default' { + # Arrange / Act + $organization = $pipeline.parameters | Where-Object name -EQ 'nuGetOrganization' + $exclusions = $pipeline.parameters | Where-Object name -EQ 'skipPackages' + + # Assert + $organization.default | Should -BeNullOrEmpty + $exclusions.type | Should -Be 'object' + $exclusions.default.Count | Should -Be 0 + $publicPackageIds | Should -Contain 'Microsoft.DurableTask.Generators' + } + + It 'uses the Durable ESRP14 NuGet profile without a direct public-push fallback' { + # Arrange / Act + $release = $releaseTemplate.steps[1] + + # Assert + $releaseTemplate.steps.Count | Should -Be 2 + $release.task | Should -Be 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14' + $release.inputs.ConnectedServiceName | Should -Be 'dtfx-internal-esrp-prod' + $release.inputs.ClientId | Should -Be '0b3ed1a4-0727-4a50-b82a-02c2bd9dec89' + $release.inputs.DomainTenantId | Should -Be '33e01921-4d64-4f8c-a055-5bdaffd5e33d' + $release.inputs.UseManagedIdentity | Should -BeTrue + $release.inputs.KeyVaultName | Should -Be 'durable-esrp-akv' + $release.inputs.SignCertName | Should -Be 'dts-esrp-cert' + $release.inputs.Intent | Should -Be 'PackageDistribution' + $release.inputs.ContentType | Should -Be 'NuGet' + $release.inputs.OrganizationName | Should -Be '${{ parameters.organizationName }}' + $release.inputs.ContentSource | Should -Be 'Folder' + $release.inputs.FolderLocation | Should -Be $releaseTemplate.steps[0].env.STAGING_FOLDER + $release.inputs.WaitForReleaseCompletion | Should -BeTrue + $release.inputs.MainPublisher | Should -Be 'ESRPRELPACMAN' + $release.inputs.Owners | Should -Be '$(Build.RequestedForEmail)' + $release.inputs.Approvers | Should -Be '$(Build.RequestedForEmail)' + @($jobs.steps | Where-Object task -EQ '1ES.PublishNuget@1').Count | Should -Be 1 + } +} + +Describe 'ESRP package staging' { + BeforeAll { + $environmentNames = 'PACKAGE_ID', 'NUGET_ORGANIZATION', 'SOURCE_FOLDER', 'STAGING_FOLDER' + $savedEnvironment = @{} + foreach ($name in $environmentNames) { + $savedEnvironment[$name] = [Environment]::GetEnvironmentVariable($name) + } + } + + BeforeEach { + $fixture = Join-Path $TestDrive ([guid]::NewGuid().ToString()) + $env:SOURCE_FOLDER = Join-Path $fixture 'drop' + $env:STAGING_FOLDER = Join-Path $fixture 'esrp' + $env:PACKAGE_ID = 'Microsoft.DurableTask.Client' + $env:NUGET_ORGANIZATION = 'validation-only-organization' + New-Item -ItemType Directory -Path $env:SOURCE_FOLDER | Out-Null + foreach ($id in $publicPackageIds) { + $version = switch ($id) { + 'Microsoft.DurableTask.Generators' { '2.1.0-preview.2' } + 'Microsoft.DurableTask.ScheduledTasks' { '1.27.1-preview.2' } + 'Microsoft.DurableTask.ExportHistory' { '1.27.1-preview.2' } + default { '1.27.1' } + } + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $id -Version $version + } + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.InProcessTestHost' -Version '0.2.3-preview.1' + } + + AfterAll { + foreach ($name in $environmentNames) { + [Environment]::SetEnvironmentVariable($name, $savedEnvironment[$name]) + } + } + + It 'stages exactly 16 unchanged trial packages from the 18-package drop, excluding the old generator and test host' { + # Arrange + $skipPackages = @('Microsoft.DurableTask.Generators') + $selectedJobs = @($publicJobs | Where-Object { $_.steps[0].parameters.packageId -notin $skipPackages }) + @(Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg').Count | Should -Be 18 + + # Act / Assert + $selectedJobs.Count | Should -Be 16 + foreach ($job in $selectedJobs) { + $env:PACKAGE_ID = $job.steps[0].parameters.packageId + $env:STAGING_FOLDER = Join-Path $fixture $env:PACKAGE_ID + & $stagePackage 6>$null + $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) + $staged.Count | Should -Be 1 + $original = Join-Path $env:SOURCE_FOLDER $staged[0].Name + (Get-FileHash -LiteralPath $staged[0].FullName).Hash | Should -Be (Get-FileHash -LiteralPath $original).Hash + $staged[0].Name | Should -Not -Match 'Generators|InProcessTestHost|symbols' + } + } + + It 'selects the exact nuspec ID rather than a filename prefix or a misleading filename' { + # Arrange + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.Client.2.Tools' + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.InProcessTestHost' -FileName 'Microsoft.DurableTask.Client.100.0.0.nupkg' + + # Act + & $stagePackage 6>$null + + # Assert + $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) + $staged.Count | Should -Be 1 + $staged[0].Name | Should -Be 'Microsoft.DurableTask.Client.1.27.1.nupkg' + } + + It 'excludes legacy symbols and snupkg files even when their package ID matches' { + # Arrange + Set-Content -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Client.1.27.1.symbols.nupkg" -Value 'not a product package' + Set-Content -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Client.1.27.1.snupkg" -Value 'not a product package' + + # Act + & $stagePackage 6>$null + + # Assert + @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File).Count | Should -Be 1 + Test-Path -LiteralPath "$env:STAGING_FOLDER\Microsoft.DurableTask.Client.1.27.1.nupkg" | Should -BeTrue + } + + It 'can stage a future generator version without a permanent generator exclusion' { + # Arrange + $env:PACKAGE_ID = 'Microsoft.DurableTask.Generators' + Remove-Item -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Generators.2.1.0-preview.2.nupkg" + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $env:PACKAGE_ID -Version '2.1.0-preview.3' + + # Act + & $stagePackage 6>$null + + # Assert + $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) + $staged.Count | Should -Be 1 + $staged[0].Name | Should -Be 'Microsoft.DurableTask.Generators.2.1.0-preview.3.nupkg' + } + + It 'rejects an unset or unresolved organization: ' -TestCases @( + @{ Organization = '' } + @{ Organization = ' ' } + @{ Organization = '$(NuGetOrganizationName)' } + @{ Organization = '${{ parameters.organizationName }}' } + ) { + param($Organization) + # Arrange + $env:NUGET_ORGANIZATION = $Organization + + # Act / Assert + { & $stagePackage 6>$null } | Should -Throw '*Set nuGetOrganization*' + Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + } + + It 'rejects a missing package without creating a submission folder' { + # Arrange + $env:PACKAGE_ID = 'Microsoft.DurableTask.Missing' + + # Act / Assert + { & $stagePackage 6>$null } | Should -Throw '*found 0*' + Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + } + + It 'rejects multiple versions of a package rather than choosing one silently' { + # Arrange + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $env:PACKAGE_ID -Version '1.27.0' + + # Act / Assert + { & $stagePackage 6>$null } | Should -Throw '*found 2*' + Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + } + + It 'rejects malformed package metadata' { + # Arrange + New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Invalid' -Nuspec '' + + # Act / Assert + { & $stagePackage 6>$null } | Should -Throw "*Missing package ID in 'Invalid.1.27.1.nupkg'*" + Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + } + + It 'refuses to reuse or delete a nonempty staging folder' { + # Arrange + New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null + $stalePackage = Join-Path $env:STAGING_FOLDER 'stale.nupkg' + Set-Content -LiteralPath $stalePackage -Value 'stale content' + + # Act / Assert + { & $stagePackage 6>$null } | Should -Throw '*refusing to reuse*' + Get-Content -LiteralPath $stalePackage | Should -Be 'stale content' + @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File).Count | Should -Be 1 + } +} diff --git a/eng/publish/publish.yml b/eng/publish/publish.yml index 03fa64ba..fc4eb4ad 100644 --- a/eng/publish/publish.yml +++ b/eng/publish/publish.yml @@ -2,6 +2,16 @@ # When executed, it automatically publishes the output of the 'official pipeline' (the nupkgs) to our internal ADO feed. # It may optionally also publish the packages to NuGet, but that is gated behind a manual approval. +parameters: +- name: nuGetOrganization + displayName: NuGet.org organization confirmed during ESRP onboarding (required) + type: string + default: '' +- name: skipPackages + displayName: Public package IDs to skip for this run (already published or intentionally held back) + type: object + default: [] + trigger: none # only trigger is manual pr: none # only trigger is manual @@ -56,6 +66,7 @@ extends: # NuGet approval gate - job: nugetApproval displayName: NuGetApproval + dependsOn: AdoRelease pool: server # This task only works when executed on serverl pools, so this needs to be specified steps: # Wait for manual approval. @@ -65,37 +76,16 @@ extends: onTimeout: 'reject' #================== NuGet publishing section ================== - # Next, we publish the nupkgs to NuGet, one at a time; for good reason. - # - # When publishing to NuGet, the `1ES.PublishNuget@1` will fail the ADO job if we attempt - # to push any package versions that were already uploaded to NuGet. - # For instance, imagine the latest DTFx packages on NuGet are: DTFx.Core@v1.1 and DTFx.AzureStorage@v1.0 - # and that our official pipeline generated these packages: DTFx.Core@v1.1 and DTFx.AzureStorage@v1.1. - # In the end, we want NuGet to have both packages at version v1.1. - # - # In this case, if we try to publish both packages in the same ADO job, it will fail as soon as it detects - # we're trying to push DTFx.Core@v1.1 as that already exists on NuGet, and this could prevent - # DTFx.AzureStorage@v1.1 from being published in the first place. - # - # Unfortunately, skipping duplicate package versions is only supported for publishing to ADO feeds, which explains - # why we did not have this issue when pushing to ADO. - # - # So we split each package upload to a separate job, so they be uploaded independently. In our example, the - # DTFx.Core upload task would fail (as expected, it was already uploaded), - # but the DTFx.AzureStorage upload would succeed. - - # Implementor's note: - # What follows could use some refactoring as there's a lot of repetition, perhaps an ADO template could hide the repeated details. - # Pay close attention to the `packagesToPush` property of each job. We need to make sure the pattern specified there uniquely identifies - # the package we want to push, and this is difficult when the full name of one package is the prefix of another. - # For example: `Microsoft.DurableTask.Client` and `Microsoft.DurableTask.Client.Grpc`. - # The trick is to exclude the `Microsoft.DurableTask.Client.Grpc` in the `packagesToPush` property for `Microsoft.DurableTask.Client` + # Keep one package per ESRP submission: duplicate versions fail independently, and ESRP + # does not guarantee publication order within a submission. Preserve the dependency jobs below. + # skipPackages applies only to this run; it does not remove future releases of a package. + # The shared steps select the exact nuspec ID, not a filename prefix, from the signed drop. # NuGet release (Microsoft.DurableTask.Abstractions) - job: nugetRelease_Microsoft_DurableTask_Abstractions displayName: NuGet Release (Microsoft.DurableTask.Abstractions) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Abstractions')) }}) templateContext: type: releaseJob isProduction: true @@ -105,20 +95,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Abstractions)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Abstractions.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Abstractions + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Client) - job: nugetRelease_Microsoft_DurableTask_Client displayName: NuGet Release (Microsoft.DurableTask.Client) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client')) }}) templateContext: type: releaseJob isProduction: true @@ -128,25 +114,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Client)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - # the packages to push pattern explicitly excludes: - # - 'Microsoft.DurableTask.Client.Grpc' - # - 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim' - # - 'Microsoft.DurableTask.Client.AzureManaged.*' - # which are pushed by their respective jobs - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.Grpc.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.OrchestrationServiceClientShim.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.AzureManaged.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Client + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Client.Grpc) - job: nugetRelease_Microsoft_DurableTask_Client_Grpc displayName: NuGet Release (Microsoft.DurableTask.Client.Grpc) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.Grpc')) }}) templateContext: type: releaseJob isProduction: true @@ -156,20 +133,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Client.Grpc)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.Grpc.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Client.Grpc + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Client.OrchestrationServiceClientShim) - job: nugetRelease_Microsoft_DurableTask_Client_OrchestrationServiceClientShim displayName: NuGet Release (Microsoft.DurableTask.Client.OrchestrationServiceClientShim) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim')) }}) templateContext: type: releaseJob isProduction: true @@ -179,20 +152,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Client.OrchestrationServiceClientShim)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.OrchestrationServiceClientShim.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Client.OrchestrationServiceClientShim + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Generators) - job: nugetRelease_Microsoft_DurableTask_Generators displayName: NuGet Release (Microsoft.DurableTask.Generators) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Generators')) }}) templateContext: type: releaseJob isProduction: true @@ -202,20 +171,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Generators)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Generators.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Generators + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Analyzers) - job: nugetRelease_Microsoft_DurableTask_Analyzers displayName: NuGet Release (Microsoft.DurableTask.Analyzers) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Analyzers')) }}) templateContext: type: releaseJob isProduction: true @@ -225,20 +190,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Analyzers)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Analyzers.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Analyzers + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Grpc) - job: nugetRelease_Microsoft_DurableTask_Grpc displayName: NuGet Release (Microsoft.DurableTask.Grpc) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Grpc')) }}) templateContext: type: releaseJob isProduction: true @@ -248,20 +209,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Grpc)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Grpc.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Grpc + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Worker) - job: nugetRelease_Microsoft_DurableTask_Worker displayName: NuGet Release (Microsoft.DurableTask.Worker) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker')) }}) templateContext: type: releaseJob isProduction: true @@ -271,21 +228,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Worker)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - # the packages to push pattern explicitly excludes 'Microsoft.DurableTask.Worker.Grpc' and 'Microsoft.DurableTask.Worker.AzureManaged', which are pushed by another job in this file - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.Grpc*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.AzureManaged.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Worker + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Worker.Grpc) - job: nugetRelease_Microsoft_DurableTask_Worker_Grpc displayName: NuGet Release (Microsoft.DurableTask.Worker.Grpc) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.Grpc')) }}) templateContext: type: releaseJob isProduction: true @@ -295,20 +247,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Worker.Grpc)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.Grpc.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Worker.Grpc + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Client.AzureManaged) - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged displayName: NuGet Release (Microsoft.DurableTask.Client.AzureManaged) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.AzureManaged')) }}) templateContext: type: releaseJob isProduction: true @@ -318,20 +266,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Client.AzureManaged)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.AzureManaged.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.AzureManaged.Sandboxes.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Client.AzureManaged + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Client.AzureManaged.Sandboxes) - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged_Sandboxes displayName: NuGet Release (Microsoft.DurableTask.Client.AzureManaged.Sandboxes) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.AzureManaged.Sandboxes')) }}) templateContext: type: releaseJob isProduction: true @@ -341,20 +285,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Client.AzureManaged.Sandboxes)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Client.AzureManaged.Sandboxes.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Client.AzureManaged.Sandboxes + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Worker.AzureManaged) - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged displayName: NuGet Release (Microsoft.DurableTask.Worker.AzureManaged) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.AzureManaged')) }}) templateContext: type: releaseJob isProduction: true @@ -364,20 +304,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Worker.AzureManaged)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.AzureManaged.*.nupkg;!$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.AzureManaged.Sandboxes.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Worker.AzureManaged + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.Worker.AzureManaged.Sandboxes) - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged_Sandboxes displayName: NuGet Release (Microsoft.DurableTask.Worker.AzureManaged.Sandboxes) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.AzureManaged.Sandboxes')) }}) templateContext: type: releaseJob isProduction: true @@ -387,20 +323,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Worker.AzureManaged.Sandboxes)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Worker.AzureManaged.Sandboxes.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Worker.AzureManaged.Sandboxes + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.ScheduledTasks) - job: nugetRelease_Microsoft_DurableTask_ScheduledTasks displayName: NuGet Release (Microsoft.DurableTask.ScheduledTasks) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.ScheduledTasks')) }}) templateContext: type: releaseJob isProduction: true @@ -410,20 +342,16 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.ScheduledTasks)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.ScheduledTasks.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.ScheduledTasks + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.ExportHistory) - job: nugetRelease_Microsoft_DurableTask_ExportHistory displayName: NuGet Release (Microsoft.DurableTask.ExportHistory) dependsOn: nugetApproval - condition: succeeded('nugetApproval') # nuget packages need to be on ADO first + condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.ExportHistory')) }}) templateContext: type: releaseJob isProduction: true @@ -433,14 +361,10 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.ExportHistory)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.ExportHistory.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.ExportHistory + organizationName: ${{ parameters.nuGetOrganization }} # add it for Microsoft.DurableTask.Extensions.AzureBlobPayloads - job: nugetRelease_Microsoft_DurableTask_Extensions_AzureBlobPayloads @@ -448,7 +372,7 @@ extends: dependsOn: - nugetApproval - nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions - condition: succeeded() # the Blob package depends on the new contract package, which must publish first + condition: and(succeeded(), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Extensions.AzureBlobPayloads')) }}) templateContext: type: releaseJob isProduction: true @@ -458,14 +382,10 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.Extensions.AzureBlobPayloads)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.Extensions.AzureBlobPayloads.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' # Despite this being a custom command, we need to keep this for 1ES validation - packageParentPath: $(System.DefaultWorkingDirectory) # This needs to be set to some prefix of the `packagesToPush` parameter. Apparently it helps with SDL tooling + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.Extensions.AzureBlobPayloads + organizationName: ${{ parameters.nuGetOrganization }} # NuGet release (Microsoft.DurableTask.LargePayloadPurge.Abstractions) - job: nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions @@ -474,7 +394,7 @@ extends: - nugetApproval - nugetRelease_Microsoft_DurableTask_Abstractions - nugetRelease_Microsoft_DurableTask_Client - condition: succeeded() + condition: and(succeeded(), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.LargePayloadPurge.Abstractions')) }}) templateContext: type: releaseJob isProduction: true @@ -484,11 +404,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - task: 1ES.PublishNuget@1 - displayName: 'NuGet push (Microsoft.DurableTask.LargePayloadPurge.Abstractions)' - inputs: - command: push - nuGetFeedType: external - publishFeedCredentials: 'DurableTask org NuGet API Key' - packagesToPush: '$(System.DefaultWorkingDirectory)/drop/Microsoft.DurableTask.LargePayloadPurge.Abstractions.*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' - packageParentPath: $(System.DefaultWorkingDirectory) \ No newline at end of file + - template: nuget-release-steps.yml + parameters: + packageId: Microsoft.DurableTask.LargePayloadPurge.Abstractions + organizationName: ${{ parameters.nuGetOrganization }} \ No newline at end of file From 62a133de41f7fd6d1eaa071fcca7b82ae06e3691 Mon Sep 17 00:00:00 2001 From: wangbill Date: Thu, 22 Oct 2026 12:20:30 -0400 Subject: [PATCH 2/6] Make NuGet approval confirm the selected ESRP organization Display the selected organization at the manual gate and require onboarding, publisher allowlisting and Trusted Publishing confirmation before public submission. Clarify automatic mirror filters versus separately authorized manual mirroring without changing mirror policy or invoking a workflow. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- doc/release_process.md | 19 ++++++++++++------- eng/publish/publish.Tests.ps1 | 5 +++++ eng/publish/publish.yml | 9 ++++++++- 3 files changed, 25 insertions(+), 8 deletions(-) diff --git a/doc/release_process.md b/doc/release_process.md index 9ea62a65..cb6c13d4 100644 --- a/doc/release_process.md +++ b/doc/release_process.md @@ -123,17 +123,21 @@ After the PR is merged, follow the **Publishing** steps below. 3. Use [`durabletask-dotnet.publish`](https://dev.azure.com/azfunc/internal/_build?definitionId=1100) (`eng/publish/publish.yml`). The publisher source ref and the `officialPipeline` artifact resource are separate selections: explicitly pin the official run, not just its branch or "latest". - Use the reviewed publisher-source commit on `main` after the normal code-mirror pipeline has - mirrored it to ADO. Individual user branches are not mirrored; do not expand the mirror allowlist - for a trial. Selecting the old release tag as the publisher source loads the old publisher YAML, - even when the correct signed official artifact run is selected. + Normally use the reviewed publisher-source commit on `main` after the code-mirror pipeline has + mirrored it to ADO. The branch allowlist controls automatic mirror triggers; it does not rule out + a separately authorized manual invocation of the existing mirror workflow for an exact trial ref. + Do not expand the trigger allowlist or change mirror policies for a trial. Selecting the old + release tag as the publisher source loads the old publisher YAML, even when the correct signed + official artifact run is selected. 4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding. There is no organization default; an unset or unresolved value fails before any ESRP submission. Set `skipPackages` to an array of exact public package IDs whose selected versions are already published or are intentionally held back. This is a per-run exclusion, not a permanent package removal. 5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that - succeeds does `nugetApproval` wait for manual approval. Do not approve until the selected artifacts, - exclusion list, dependency availability, and ESRP prerequisites below have been checked. + succeeds does `nugetApproval` wait for manual approval. The gate displays the selected organization + and requires confirmation of NuGet onboarding, publisher allowlisting and Trusted Publishing. + Do not approve until those prerequisites, the selected artifacts, exclusion list and dependency + availability have been checked. Stop at the gate for a validation-only trial. Rejecting or timing out the gate prevents all public publication. 6. After approval, each selected public package is copied unchanged into its own ESRP folder using the exact nuspec ID, then submitted with `EsrpRelease@14` and `WaitForReleaseCompletion: true`. @@ -176,7 +180,8 @@ version is already public. This leaves 16 public targets; the test host has no p The default exclusion list is empty so future generator releases remain enabled. Keep `officialPipeline` pinned to the approved signed run `311120` (build number `20261021.2`, source `2a409da4ac7dcf03c4f25985337eb4623ea6394e`), while the publisher uses the new reviewed and mirrored -`main` commit containing this migration. A run ID and build number are not interchangeable. +source containing this migration (normally the reviewed `main` commit). A run ID and build number +are not interchangeable. Recheck exact-version absence for every intended target immediately before any submission; search indexing alone is not proof that a version has never been published. diff --git a/eng/publish/publish.Tests.ps1 b/eng/publish/publish.Tests.ps1 index 4057f750..68fec79d 100644 --- a/eng/publish/publish.Tests.ps1 +++ b/eng/publish/publish.Tests.ps1 @@ -95,6 +95,11 @@ Describe 'Publisher pipeline' { $approval.pool | Should -Be 'server' $approval.steps[0].task | Should -Be 'ManualValidation@1' $approval.steps[0].inputs.onTimeout | Should -Be 'reject' + $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape('${{ parameters.nuGetOrganization }}')) + $approval.steps[0].inputs.instructions | Should -Match 'ESRP NuGet onboarding' + $approval.steps[0].inputs.instructions | Should -Match 'ESRPRELPACMAN' + $approval.steps[0].inputs.instructions | Should -Match 'Trusted Publishing' + $approval.steps[0].inputs.instructions | Should -Match 'Stop here for a validation-only trial' } It 'retains fail-closed dependency gates and run-scoped exclusions for every public job' { diff --git a/eng/publish/publish.yml b/eng/publish/publish.yml index fc4eb4ad..c6a3280a 100644 --- a/eng/publish/publish.yml +++ b/eng/publish/publish.yml @@ -72,7 +72,14 @@ extends: # Wait for manual approval. - task: ManualValidation@1 inputs: - instructions: Confirm you want to push to NuGet + instructions: | + Selected NuGet.org organization: ${{ parameters.nuGetOrganization }} + Confirm that this is the organization approved during ESRP NuGet onboarding. + Confirm the Durable ESRP client is allowlisted for ESRPRELPACMAN and the NuGet + Trusted Publishing policy covers the selected package IDs/prefixes and operations. + Validate the selected signed official build, package exclusions and dependency availability. + Stop here for a validation-only trial or if any prerequisite remains unverified. + Approving this gate starts public ESRP NuGet submissions. onTimeout: 'reject' #================== NuGet publishing section ================== From 261b233a828211dae2439afba6a9cf3601d79c5c Mon Sep 17 00:00:00 2001 From: wangbill Date: Thu, 22 Oct 2026 12:42:48 -0400 Subject: [PATCH 3/6] Anchor ESRP step templates to the self repository Use /eng/publish/nuget-release-steps.yml@self for all 17 public-package jobs so the nested steps resolve in the original repository when expanded by external 1ES templates. Add a regression demonstrated failing on the former relative references. Leave staging logic, gates, profile inputs and package bytes unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- eng/publish/publish.Tests.ps1 | 15 ++++++++++++++- eng/publish/publish.yml | 34 +++++++++++++++++----------------- 2 files changed, 31 insertions(+), 18 deletions(-) diff --git a/eng/publish/publish.Tests.ps1 b/eng/publish/publish.Tests.ps1 index 68fec79d..2a5e7886 100644 --- a/eng/publish/publish.Tests.ps1 +++ b/eng/publish/publish.Tests.ps1 @@ -73,11 +73,24 @@ Describe 'Publisher pipeline' { $job.templateContext.isProduction | Should -BeTrue $job.templateContext.inputs[0].pipeline | Should -Be 'officialPipeline' $job.templateContext.inputs[0].artifactName | Should -Be 'drop' - $job.steps[0].template | Should -Be 'nuget-release-steps.yml' $job.steps[0].parameters.organizationName | Should -Be '${{ parameters.nuGetOrganization }}' } } + It 'anchors every NuGet step template to self when passed through the external 1ES stage template' { + # Arrange + $expectedTemplate = '/eng/publish/nuget-release-steps.yml@self' + + # Act + $references = @($publicJobs | ForEach-Object { $_.steps[0].template }) + + # Assert + $references.Count | Should -Be 17 + foreach ($reference in $references) { + $reference | Should -Be $expectedTemplate + } + } + It 'preserves internal staging and waits for it before manual NuGet approval' { # Arrange $ado = $jobs | Where-Object job -EQ 'AdoRelease' diff --git a/eng/publish/publish.yml b/eng/publish/publish.yml index c6a3280a..89386bb2 100644 --- a/eng/publish/publish.yml +++ b/eng/publish/publish.yml @@ -102,7 +102,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Abstractions organizationName: ${{ parameters.nuGetOrganization }} @@ -121,7 +121,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Client organizationName: ${{ parameters.nuGetOrganization }} @@ -140,7 +140,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Client.Grpc organizationName: ${{ parameters.nuGetOrganization }} @@ -159,7 +159,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Client.OrchestrationServiceClientShim organizationName: ${{ parameters.nuGetOrganization }} @@ -178,7 +178,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Generators organizationName: ${{ parameters.nuGetOrganization }} @@ -197,7 +197,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Analyzers organizationName: ${{ parameters.nuGetOrganization }} @@ -216,7 +216,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Grpc organizationName: ${{ parameters.nuGetOrganization }} @@ -235,7 +235,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Worker organizationName: ${{ parameters.nuGetOrganization }} @@ -254,7 +254,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Worker.Grpc organizationName: ${{ parameters.nuGetOrganization }} @@ -273,7 +273,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Client.AzureManaged organizationName: ${{ parameters.nuGetOrganization }} @@ -292,7 +292,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Client.AzureManaged.Sandboxes organizationName: ${{ parameters.nuGetOrganization }} @@ -311,7 +311,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Worker.AzureManaged organizationName: ${{ parameters.nuGetOrganization }} @@ -330,7 +330,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Worker.AzureManaged.Sandboxes organizationName: ${{ parameters.nuGetOrganization }} @@ -349,7 +349,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.ScheduledTasks organizationName: ${{ parameters.nuGetOrganization }} @@ -368,7 +368,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.ExportHistory organizationName: ${{ parameters.nuGetOrganization }} @@ -389,7 +389,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.Extensions.AzureBlobPayloads organizationName: ${{ parameters.nuGetOrganization }} @@ -411,7 +411,7 @@ extends: artifactName: drop targetPath: $(System.DefaultWorkingDirectory)/drop steps: - - template: nuget-release-steps.yml + - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.LargePayloadPurge.Abstractions organizationName: ${{ parameters.nuGetOrganization }} \ No newline at end of file From 7eca4a1ea173eeb7f2a3444f19eff30bdc01cb1f Mon Sep 17 00:00:00 2001 From: wangbill Date: Thu, 22 Oct 2026 13:25:59 -0400 Subject: [PATCH 4/6] Document pinned NuGet trial requests and queue permission Clarify the verified REST string-map serialization for skipPackages and the narrow Edit queue build configuration action required for nondefault pinned inputs. Record actual-source compilation at261b and the pre-run TF215106 authorization blocker without changing runtime publisher defaults, helpers, tests or permissions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- doc/release_process.md | 53 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/doc/release_process.md b/doc/release_process.md index cb6c13d4..157c48f3 100644 --- a/doc/release_process.md +++ b/doc/release_process.md @@ -185,6 +185,59 @@ are not interchangeable. Recheck exact-version absence for every intended target immediately before any submission; search indexing alone is not proof that a version has never been published. +#### Pinned trial request and queue permissions + +The pipeline's `skipPackages` parameter is a YAML/UI list, for example +`[Microsoft.DurableTask.Generators]`. For the verified REST request through the installed Azure DevOps +CLI SDK, however, `RunPipelineParameters.template_parameters` is typed as `{str}`: send +`templateParameters` as a map of strings, with the list serialized as JSON **inside a string**. +This request fragment illustrates the wire value; `` is a placeholder, not a default: + +```json +{ + "templateParameters": { + "nuGetOrganization": "", + "skipPackages": "[\"Microsoft.DurableTask.Generators\"]" + } +} +``` + +Do not send `skipPackages` as a native JSON array through that client. In the trial, that request +invalidated or ignored overrides and resolved default source/artifact selections instead. +Verify the resolved publisher source, official artifact run, organization bindings and exclusion +conditions in the actual expanded preview before attempting a run; an HTTP 200 alone is insufficient. + +Selecting nondefault pinned trial inputs requires **Edit queue build configuration** on +`internal` project pipeline `1100` (`durabletask-dotnet.publish`): Build security action +`EditPipelineQueueConfigurationPermission` (bit `65536`), distinct from **Queue builds** (bit `128`) +and **Edit build pipeline** (bit `2048`). If Azure DevOps returns +`TF215106` requesting that permission, stop and have an ADO administrator grant that narrow permission +to the intended requester before another attempt. Do not change pipeline defaults, switch +credentials/APIs, or loosen policies to bypass the denial. This queue-configuration permission is +separate from authorization to use `dtfx-internal-esrp-prod`, ESRP `ESRPRELPACMAN` allowlisting, +NuGet channel onboarding and Trusted Publishing. + +#### Observed trial status (2026-10-22) + +The tested runtime source is `261b233a828211dae2439afba6a9cf3601d79c5c`. Mirror run `311345` +succeeded and its locked internal ref matched that commit. The actual-source publisher preview +returned HTTP 200; scalar positive-control and corrected string-map previews confirmed all 17 +organization/environment bindings, the visible selected-organization/STOP approval instructions, +the Generator job condition set to `False`, and the remaining 16 runnable jobs with dependency-success +gates preserved. The organization used for compilation was a candidate, not verified onboarding. + +An earlier malformed array-valued request created run `311347` with default old `main` source +`2a409da4`, latest official run `311261`, and empty template parameters. It was canceled before any +manual approval; it is not evidence of a correctly pinned trial. +With the corrected request, official build `20261021.2`, and fresh exact-version HTTP 404s for all +16 targets, the queue attempt returned HTTP 500 / `TF215106` before assigning a run ID: +the requester lacked **Edit queue build configuration** on pipeline `1100`. +This is a hard ADO authorization blocker, not a source compilation failure or an observed +ESRP/backend/NuGet failure. No permission grant, policy change, approval or public distribution was +performed. Post-denial run inventory confirmed that the corrected request created no run and that +`311347` was completed/canceled; no accidental publisher remained active. +Later documentation-only commits do not replace `261b233a` as the exact runtime tested. + For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run: ```powershell From 245cfd8e0ec0fcc786962c1a12a62ae344e79972 Mon Sep 17 00:00:00 2001 From: wangbill Date: Thu, 22 Oct 2026 14:32:26 -0400 Subject: [PATCH 5/6] Distinguish already-published NuGet versions from intentional holds Add alreadyPublishedPackages confirmations that emit only a logged successful no-op, without staging or ESRP submission. Keep skipPackages as intentional holds with precedence over confirmations so existing success, approval, failure and cancellation gates remain fail-closed. Preserve the 19-job graph, root-qualified templates and unchanged publishing branch. Add functional partial-recovery tests and update trial/operator guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- doc/release_process.md | 68 +++--- eng/publish/nuget-release-steps.yml | 141 ++++++------ eng/publish/publish.Tests.ps1 | 333 ++++++++++++++++++++++++++-- eng/publish/publish.yml | 35 ++- 4 files changed, 467 insertions(+), 110 deletions(-) diff --git a/doc/release_process.md b/doc/release_process.md index 157c48f3..de2ec2e1 100644 --- a/doc/release_process.md +++ b/doc/release_process.md @@ -21,19 +21,25 @@ Its package and assembly name is `Microsoft.DurableTask.LargePayloadPurge.Abstra standard `Microsoft.DurableTask.*.dll` signing pattern. The existing source traversal, SBOM inclusion, NuGet signing and per-package approval-gated publication steps apply. -Contract publication also waits for successful Client and Abstractions publication. If either prerequisite -fails, including a duplicate-version upload failure, or is skipped or canceled, contract publication is -skipped rather than treating that result as success. +Contract publication also waits for successful Client and Abstractions jobs. Each prerequisite must +either publish successfully or complete an explicit successful no-op because the operator confirmed its +selected artifact version is already on NuGet.org using `alreadyPublishedPackages`. If either job fails, +including a duplicate-version upload failure, or is intentionally held back, skipped or canceled, contract +publication is skipped rather than treating that result as success. `Microsoft.DurableTask.Extensions.AzureBlobPayloads` depends on the contract package at every target framework, so its publication job also waits for successful contract publication (in addition to approval). -If contract publication fails for any reason above, including a duplicate-version upload failure, or is -skipped or canceled, Blob publication is skipped rather than treating that result as success. This extends -the same fail-closed chain: approval, then Client and Abstractions, then the contract, then Blob. A -prerequisite that deliberately skips — for example because its exact version is already published — still -skips the dependent job; this is not a general-purpose publication-idempotency mechanism, just the minimum -ordering these two packages require. Other packages retain their own independent publication jobs gated -only on approval. +If the contract job fails for any reason above, including a duplicate-version upload failure, or is +intentionally held back, skipped or canceled, Blob publication is skipped rather than treating that result +as success. This extends the same fail-closed chain: approval, then Client and Abstractions, then the +contract, then Blob. An already-published confirmation succeeds without staging or submitting that +package, allowing recovery of a partial release: confirm Client/Abstractions to retry the contract and +Blob. For a Blob-only retry, confirm all selected versions already published, including Client, +Abstractions and the contract; their jobs still follow the dependency chain but complete as no-ops. +Intentional holds belong in `skipPackages` and still block dependents. If a package appears in both +lists, the hold takes precedence; it is not permission to publish dependents. Both lists default to +empty and apply only to the selected run. Other packages retain their own independent publication +jobs gated only on approval. Keep its `RELEASENOTES.md`: `eng/targets/Release.targets` reads it into NuGet package metadata, whereas the root `CHANGELOG.md` retains repository release history. The shared target also appends a link using the @@ -131,16 +137,21 @@ After the PR is merged, follow the **Publishing** steps below. official artifact run is selected. 4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding. There is no organization default; an unset or unresolved value fails before any ESRP submission. - Set `skipPackages` to an array of exact public package IDs whose selected versions are already - published or are intentionally held back. This is a per-run exclusion, not a permanent package removal. + Set `alreadyPublishedPackages` to exact public package IDs only after verifying their **selected + artifact versions** already exist on NuGet.org. Those jobs log a successful no-op, without package + staging or an ESRP task, so dependent jobs can proceed. + Use `skipPackages` only for intentional holds: those jobs are skipped and still block dependents. + A hold takes precedence if a package is in both lists. These are per-run inputs, not permanent removals. 5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that succeeds does `nugetApproval` wait for manual approval. The gate displays the selected organization - and requires confirmation of NuGet onboarding, publisher allowlisting and Trusted Publishing. - Do not approve until those prerequisites, the selected artifacts, exclusion list and dependency + and both package lists, and requires confirmation of NuGet onboarding, publisher allowlisting, + Trusted Publishing and each exact already-published version. + Do not approve until those prerequisites, the selected artifacts, both lists and dependency availability have been checked. Stop at the gate for a validation-only trial. Rejecting or timing out the gate prevents all public publication. -6. After approval, each selected public package is copied unchanged into its own ESRP folder using - the exact nuspec ID, then submitted with `EsrpRelease@14` and `WaitForReleaseCompletion: true`. +6. After approval, each public package not held or already confirmed published is copied unchanged into + its own ESRP folder using the exact nuspec ID, then submitted with `EsrpRelease@14` and + `WaitForReleaseCompletion: true`. Internal-only/test packages and symbol packages are not submitted. Missing or multiple matching packages fail explicitly. Each package retains its independent job and the dependency gates above; ESRP does not guarantee ordering within a multi-package submission. @@ -175,9 +186,10 @@ The signed official drop has 18 `.nupkg` files: 14 public stable packages at `1. `ExportHistory` and `ScheduledTasks` at `1.27.1-preview.2`, the unchanged public `Microsoft.DurableTask.Generators` at `2.1.0-preview.2`, and internal-only `Microsoft.DurableTask.InProcessTestHost` at `0.2.3-preview.1`. -For this trial, set `skipPackages: [Microsoft.DurableTask.Generators]` because that exact generator -version is already public. This leaves 16 public targets; the test host has no public job. -The default exclusion list is empty so future generator releases remain enabled. +For this trial, set `alreadyPublishedPackages: [Microsoft.DurableTask.Generators]` and +`skipPackages: []` because that exact generator version is already public. Its job succeeds through a +logged no-op, not an ESRP submission. This leaves 16 public submission targets; the test host has no +public job. Both default lists are empty so future generator releases remain enabled. Keep `officialPipeline` pinned to the approved signed run `311120` (build number `20261021.2`, source `2a409da4ac7dcf03c4f25985337eb4623ea6394e`), while the publisher uses the new reviewed and mirrored source containing this migration (normally the reviewed `main` commit). A run ID and build number @@ -187,8 +199,9 @@ search indexing alone is not proof that a version has never been published. #### Pinned trial request and queue permissions -The pipeline's `skipPackages` parameter is a YAML/UI list, for example -`[Microsoft.DurableTask.Generators]`. For the verified REST request through the installed Azure DevOps +The pipeline's `skipPackages` and `alreadyPublishedPackages` parameters are YAML/UI lists. For the +trial, use an empty hold list and `[Microsoft.DurableTask.Generators]` as the already-published list. +For the verified REST request through the installed Azure DevOps CLI SDK, however, `RunPipelineParameters.template_parameters` is typed as `{str}`: send `templateParameters` as a map of strings, with the list serialized as JSON **inside a string**. This request fragment illustrates the wire value; `` is a placeholder, not a default: @@ -197,12 +210,13 @@ This request fragment illustrates the wire value; `` is a placeho { "templateParameters": { "nuGetOrganization": "", - "skipPackages": "[\"Microsoft.DurableTask.Generators\"]" + "skipPackages": "[]", + "alreadyPublishedPackages": "[\"Microsoft.DurableTask.Generators\"]" } } ``` -Do not send `skipPackages` as a native JSON array through that client. In the trial, that request +Do not send either list as a native JSON array through that client. In the earlier trial, that request invalidated or ignored overrides and resolved default source/artifact selections instead. Verify the resolved publisher source, official artifact run, organization bindings and exclusion conditions in the actual expanded preview before attempting a run; an HTTP 200 alone is insufficient. @@ -217,7 +231,7 @@ credentials/APIs, or loosen policies to bypass the denial. This queue-configurat separate from authorization to use `dtfx-internal-esrp-prod`, ESRP `ESRPRELPACMAN` allowlisting, NuGet channel onboarding and Trusted Publishing. -#### Observed trial status (2026-10-22) +#### Observed trial status before partial-release recovery fix (2026-10-22) The tested runtime source is `261b233a828211dae2439afba6a9cf3601d79c5c`. Mirror run `311345` succeeded and its locked internal ref matched that commit. The actual-source publisher preview @@ -236,7 +250,11 @@ This is a hard ADO authorization blocker, not a source compilation failure or an ESRP/backend/NuGet failure. No permission grant, policy change, approval or public distribution was performed. Post-denial run inventory confirmed that the corrected request created no run and that `311347` was completed/canceled; no accidental publisher remained active. -Later documentation-only commits do not replace `261b233a` as the exact runtime tested. +Later documentation-only commits did not replace `261b233a` as the exact runtime tested in that preview. +The subsequent source change adds `alreadyPublishedPackages` to distinguish successful no-ops from +intentional holds. It is not covered by that earlier server preview; the final recovery/holdback variants +need their own actual-source compilation before use. The queue-configuration and downstream onboarding +prerequisites remain separate and unresolved; no trial parameter change bypasses them. For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run: diff --git a/eng/publish/nuget-release-steps.yml b/eng/publish/nuget-release-steps.yml index b61b1bb3..a1f520f0 100644 --- a/eng/publish/nuget-release-steps.yml +++ b/eng/publish/nuget-release-steps.yml @@ -3,84 +3,95 @@ parameters: type: string - name: organizationName type: string +- name: alreadyPublished + type: boolean + default: false steps: -- pwsh: | - $ErrorActionPreference = 'Stop' +- ${{ if parameters.alreadyPublished }}: + - pwsh: | + Write-Host "Operator confirmed the selected version of '$env:PACKAGE_ID' is already published on NuGet.org. No package will be staged or submitted to ESRP." + displayName: Already published (${{ parameters.packageId }}) + env: + PACKAGE_ID: ${{ parameters.packageId }} - if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') { - throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.' - } +- ${{ else }}: + - pwsh: | + $ErrorActionPreference = 'Stop' - $packages = @( - foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) { - if ($file.Name -like '*.symbols.nupkg') { - continue - } + if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') { + throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.' + } - $archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName) - try { - $nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' }) - if ($nuspecs.Count -ne 1) { - throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)." + $packages = @( + foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) { + if ($file.Name -like '*.symbols.nupkg') { + continue } - $reader = [System.IO.StreamReader]::new($nuspecs[0].Open()) + $archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName) try { - [xml]$nuspec = $reader.ReadToEnd() - } - finally { - $reader.Dispose() - } + $nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' }) + if ($nuspecs.Count -ne 1) { + throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)." + } - if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) { - throw "Missing package ID in '$($file.Name)'." - } + $reader = [System.IO.StreamReader]::new($nuspecs[0].Open()) + try { + [xml]$nuspec = $reader.ReadToEnd() + } + finally { + $reader.Dispose() + } + + if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) { + throw "Missing package ID in '$($file.Name)'." + } - if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) { - $file + if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) { + $file + } + } + finally { + $archive.Dispose() } } - finally { - $archive.Dispose() - } - } - ) + ) - if ($packages.Count -ne 1) { - throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)." - } + if ($packages.Count -ne 1) { + throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)." + } - if (Test-Path -LiteralPath $env:STAGING_FOLDER) { - throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content." - } + if (Test-Path -LiteralPath $env:STAGING_FOLDER) { + throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content." + } - New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null - Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER - Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'." - displayName: Stage ${{ parameters.packageId }} for ESRP - env: - PACKAGE_ID: ${{ parameters.packageId }} - NUGET_ORGANIZATION: ${{ parameters.organizationName }} - SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop - STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} + New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null + Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER + Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'." + displayName: Stage ${{ parameters.packageId }} for ESRP + env: + PACKAGE_ID: ${{ parameters.packageId }} + NUGET_ORGANIZATION: ${{ parameters.organizationName }} + SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop + STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} -- task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14 - displayName: ESRP NuGet release (${{ parameters.packageId }}) - inputs: - ConnectedServiceName: dtfx-internal-esrp-prod - UseManagedIdentity: true - KeyVaultName: durable-esrp-akv - SignCertName: dts-esrp-cert - ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89 - DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d - Intent: PackageDistribution - ContentType: NuGet - OrganizationName: ${{ parameters.organizationName }} - ContentSource: Folder - FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} - WaitForReleaseCompletion: true - Owners: $(Build.RequestedForEmail) - Approvers: $(Build.RequestedForEmail) - ServiceEndpointUrl: https://api.esrp.microsoft.com - MainPublisher: ESRPRELPACMAN + - task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14 + displayName: ESRP NuGet release (${{ parameters.packageId }}) + inputs: + ConnectedServiceName: dtfx-internal-esrp-prod + UseManagedIdentity: true + KeyVaultName: durable-esrp-akv + SignCertName: dts-esrp-cert + ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89 + DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d + Intent: PackageDistribution + ContentType: NuGet + OrganizationName: ${{ parameters.organizationName }} + ContentSource: Folder + FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }} + WaitForReleaseCompletion: true + Owners: $(Build.RequestedForEmail) + Approvers: $(Build.RequestedForEmail) + ServiceEndpointUrl: https://api.esrp.microsoft.com + MainPublisher: ESRPRELPACMAN diff --git a/eng/publish/publish.Tests.ps1 b/eng/publish/publish.Tests.ps1 index 2a5e7886..90257c7d 100644 --- a/eng/publish/publish.Tests.ps1 +++ b/eng/publish/publish.Tests.ps1 @@ -5,7 +5,8 @@ BeforeAll { Import-Module powershell-yaml -ErrorAction Stop $pipeline = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\publish.yml" -Raw) $releaseTemplate = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\nuget-release-steps.yml" -Raw) - $stagePackage = [scriptblock]::Create($releaseTemplate.steps[0].pwsh) + $publishSteps = @($releaseTemplate.steps[1]['${{ else }}']) + $stagePackage = [scriptblock]::Create($publishSteps[0].pwsh) $jobs = $pipeline.extends.parameters.stages[0].jobs $publicJobs = @($jobs | Where-Object job -Like 'nugetRelease_*') $publicPackageIds = @( @@ -56,6 +57,142 @@ BeforeAll { $archive.Dispose() } } + + function Test-PackageListExpression { + param([string]$Expression, [hashtable]$Parameters) + + $expressionBody = $Expression -replace '^\$\{\{\s*|\s*\}\}$', '' + $negated = $expressionBody.StartsWith('not(') + if ($negated) { + $expressionBody = $expressionBody.Substring(4, $expressionBody.Length - 5) + } + $match = [regex]::Match($expressionBody, "^containsValue\(parameters\.(?\w+), '(?[^']+)'\)$") + if (-not $match.Success -or -not $Parameters.ContainsKey($match.Groups['list'].Value)) { + throw "Unsupported package-list expression: $Expression" + } + $included = @($Parameters[$match.Groups['list'].Value]) -contains $match.Groups['id'].Value + if ($negated) { + return -not $included + } + return $included + } + + function Get-ReleaseSteps { + param([System.Collections.IDictionary]$Job, [hashtable]$Parameters) + + $alreadyPublished = Test-PackageListExpression $Job.steps[0].parameters.alreadyPublished $Parameters + $branch = if ($alreadyPublished) { '${{ if parameters.alreadyPublished }}' } else { '${{ else }}' } + $branches = @($releaseTemplate.steps | Where-Object { $_.ContainsKey($branch) }) + if ($branches.Count -ne 1) { + throw "Expected one template branch for alreadyPublished=$alreadyPublished." + } + return $branches[0][$branch] + } + + # Model only the conditions emitted by this pipeline; unexpected expressions fail the local harness. + function Test-ReleaseJobCondition { + param([System.Collections.IDictionary]$Job, [hashtable]$Parameters, [hashtable]$Results) + + $condition = [regex]::Replace($Job.condition, '\$\{\{.*?\}\}', { + param($match) + (Test-PackageListExpression $match.Value $Parameters).ToString() + }) + $match = [regex]::Match($condition, "^and\(succeeded\((?:'(?[^']+)')?\), (?True|False)\)$") + if (-not $match.Success) { + throw "Unsupported job condition: $condition" + } + if ($match.Groups['included'].Value -eq 'False') { + return $false + } + $dependencies = if ($match.Groups['dependency'].Success) { @($match.Groups['dependency'].Value) } else { @($Job.dependsOn) } + return @($dependencies | Where-Object { $Results[$_] -notin 'Succeeded', 'SucceededWithIssues' }).Count -eq 0 + } + + function Invoke-TestReleasePlan { + param( + [string]$Root, + [string[]]$AlreadyPublishedPackages = @(), + [string[]]$SkipPackages = @(), + [hashtable]$ResultOverrides = @{}, + [string]$ApprovalResult = 'Succeeded', + [string]$AdoResult = 'Succeeded' + ) + + $parameters = @{ alreadyPublishedPackages = $AlreadyPublishedPackages; skipPackages = $SkipPackages } + $results = @{ AdoRelease = $AdoResult; nugetApproval = $(if ($AdoResult -eq 'Succeeded') { $ApprovalResult } else { 'Skipped' }) } + $pending = [Collections.Generic.List[object]]::new() + $publicJobs | ForEach-Object { $pending.Add($_) } + $order = [Collections.Generic.List[string]]::new() + $submissions = [Collections.Generic.List[string]]::new() + $logs = [Collections.Generic.List[string]]::new() + $errors = @{} + $savedEnvironment = @{} + foreach ($name in 'PACKAGE_ID', 'NUGET_ORGANIZATION', 'SOURCE_FOLDER', 'STAGING_FOLDER') { + $savedEnvironment[$name] = [Environment]::GetEnvironmentVariable($name) + } + try { + while ($pending.Count) { + $ready = @($pending | Where-Object { @($_.dependsOn | Where-Object { -not $results.ContainsKey($_) }).Count -eq 0 }) + if ($ready.Count -eq 0) { + throw 'The public job graph cannot make progress.' + } + foreach ($job in $ready) { + $packageId = $job.steps[0].parameters.packageId + if (-not (Test-ReleaseJobCondition $job $parameters $results)) { + $results[$job.job] = 'Skipped' + } + elseif ($ResultOverrides.ContainsKey($packageId)) { + $results[$job.job] = $ResultOverrides[$packageId] + } + else { + $order.Add($packageId) + try { + foreach ($step in Get-ReleaseSteps $job $parameters) { + if ($step.pwsh) { + foreach ($entry in $step.env.GetEnumerator()) { + $value = $entry.Value.Replace('${{ parameters.packageId }}', $packageId). + Replace('${{ parameters.organizationName }}', 'validation-only-organization'). + Replace('$(System.DefaultWorkingDirectory)', $Root) + [Environment]::SetEnvironmentVariable($entry.Key, $value) + } + & ([scriptblock]::Create($step.pwsh)) 6>&1 | ForEach-Object { $logs.Add([string]$_) } + } + elseif ($step.task -eq 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14') { + # Record the emitted task, never invoke a distribution tool in local tests. + $submissions.Add($packageId) + } + else { + throw 'Unsupported release step in local harness.' + } + } + $results[$job.job] = 'Succeeded' + } + catch { + $results[$job.job] = 'Failed' + $errors[$packageId] = $_.Exception.Message + } + } + [void]$pending.Remove($job) + } + } + } + finally { + foreach ($name in $savedEnvironment.Keys) { + [Environment]::SetEnvironmentVariable($name, $savedEnvironment[$name]) + } + } + $packageResults = @{} + foreach ($job in $publicJobs) { + $packageResults[$job.steps[0].parameters.packageId] = $results[$job.job] + } + return [pscustomobject]@{ + Results = $packageResults + Order = $order.ToArray() + Submissions = $submissions.ToArray() + Logs = $logs.ToArray() + Errors = $errors + } + } } Describe 'Publisher pipeline' { @@ -74,6 +211,8 @@ Describe 'Publisher pipeline' { $job.templateContext.inputs[0].pipeline | Should -Be 'officialPipeline' $job.templateContext.inputs[0].artifactName | Should -Be 'drop' $job.steps[0].parameters.organizationName | Should -Be '${{ parameters.nuGetOrganization }}' + Test-PackageListExpression $job.steps[0].parameters.alreadyPublished @{ alreadyPublishedPackages = @($packageId) } | Should -BeTrue + Test-PackageListExpression $job.steps[0].parameters.alreadyPublished @{ alreadyPublishedPackages = @() } | Should -BeFalse } } @@ -113,6 +252,10 @@ Describe 'Publisher pipeline' { $approval.steps[0].inputs.instructions | Should -Match 'ESRPRELPACMAN' $approval.steps[0].inputs.instructions | Should -Match 'Trusted Publishing' $approval.steps[0].inputs.instructions | Should -Match 'Stop here for a validation-only trial' + $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape("join(', ', parameters.skipPackages)")) + $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape("join(', ', parameters.alreadyPublishedPackages)")) + $approval.steps[0].inputs.instructions | Should -Match 'exact selected artifact version' + $approval.steps[0].inputs.instructions | Should -Match 'Intentional holds take precedence' } It 'retains fail-closed dependency gates and run-scoped exclusions for every public job' { @@ -135,21 +278,27 @@ Describe 'Publisher pipeline' { It 'requires an explicit organization and leaves future generator releases enabled by default' { # Arrange / Act $organization = $pipeline.parameters | Where-Object name -EQ 'nuGetOrganization' - $exclusions = $pipeline.parameters | Where-Object name -EQ 'skipPackages' + $holds = $pipeline.parameters | Where-Object name -EQ 'skipPackages' + $published = $pipeline.parameters | Where-Object name -EQ 'alreadyPublishedPackages' + $noOp = $releaseTemplate.parameters | Where-Object name -EQ 'alreadyPublished' # Assert $organization.default | Should -BeNullOrEmpty - $exclusions.type | Should -Be 'object' - $exclusions.default.Count | Should -Be 0 + $holds.type | Should -Be 'object' + $holds.default.Count | Should -Be 0 + $published.type | Should -Be 'object' + $published.default.Count | Should -Be 0 + $noOp.type | Should -Be 'boolean' + $noOp.default | Should -BeFalse $publicPackageIds | Should -Contain 'Microsoft.DurableTask.Generators' } It 'uses the Durable ESRP14 NuGet profile without a direct public-push fallback' { # Arrange / Act - $release = $releaseTemplate.steps[1] + $release = $publishSteps[1] # Assert - $releaseTemplate.steps.Count | Should -Be 2 + $publishSteps.Count | Should -Be 2 $release.task | Should -Be 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14' $release.inputs.ConnectedServiceName | Should -Be 'dtfx-internal-esrp-prod' $release.inputs.ClientId | Should -Be '0b3ed1a4-0727-4a50-b82a-02c2bd9dec89' @@ -161,7 +310,7 @@ Describe 'Publisher pipeline' { $release.inputs.ContentType | Should -Be 'NuGet' $release.inputs.OrganizationName | Should -Be '${{ parameters.organizationName }}' $release.inputs.ContentSource | Should -Be 'Folder' - $release.inputs.FolderLocation | Should -Be $releaseTemplate.steps[0].env.STAGING_FOLDER + $release.inputs.FolderLocation | Should -Be $publishSteps[0].env.STAGING_FOLDER $release.inputs.WaitForReleaseCompletion | Should -BeTrue $release.inputs.MainPublisher | Should -Be 'ESRPRELPACMAN' $release.inputs.Owners | Should -Be '$(Build.RequestedForEmail)' @@ -206,17 +355,19 @@ Describe 'ESRP package staging' { It 'stages exactly 16 unchanged trial packages from the 18-package drop, excluding the old generator and test host' { # Arrange - $skipPackages = @('Microsoft.DurableTask.Generators') - $selectedJobs = @($publicJobs | Where-Object { $_.steps[0].parameters.packageId -notin $skipPackages }) @(Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg').Count | Should -Be 18 - # Act / Assert - $selectedJobs.Count | Should -Be 16 - foreach ($job in $selectedJobs) { - $env:PACKAGE_ID = $job.steps[0].parameters.packageId - $env:STAGING_FOLDER = Join-Path $fixture $env:PACKAGE_ID - & $stagePackage 6>$null - $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) + # Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages 'Microsoft.DurableTask.Generators' + + # Assert + $plan.Errors.Count | Should -Be 0 + $plan.Submissions.Count | Should -Be 16 + $plan.Results['Microsoft.DurableTask.Generators'] | Should -Be 'Succeeded' + $plan.Logs -join "`n" | Should -Match "Operator confirmed.*Microsoft.DurableTask.Generators.*No package will be staged or submitted" + Test-Path -LiteralPath "$fixture\esrp\Microsoft.DurableTask.Generators" | Should -BeFalse + foreach ($packageId in $plan.Submissions) { + $staged = @(Get-ChildItem -LiteralPath "$fixture\esrp\$packageId" -File) $staged.Count | Should -Be 1 $original = Join-Path $env:SOURCE_FOLDER $staged[0].Name (Get-FileHash -LiteralPath $staged[0].FullName).Hash | Should -Be (Get-FileHash -LiteralPath $original).Hash @@ -224,6 +375,156 @@ Describe 'ESRP package staging' { } } + It 'emits and executes only a logged no-op for already-public versions, even without a source folder' { + # Arrange + $noArtifacts = Join-Path $fixture 'no-artifacts' + + # Act + $plan = Invoke-TestReleasePlan -Root $noArtifacts -AlreadyPublishedPackages $publicPackageIds + + # Assert + $plan.Errors.Count | Should -Be 0 + @($plan.Results.Values | Where-Object { $_ -eq 'Succeeded' }).Count | Should -Be 17 + $plan.Submissions.Count | Should -Be 0 + @($plan.Logs | Where-Object { $_ -match 'Operator confirmed.*No package will be staged or submitted' }).Count | Should -Be 17 + Test-Path -LiteralPath $noArtifacts | Should -BeFalse + foreach ($job in $publicJobs) { + $steps = @(Get-ReleaseSteps $job @{ alreadyPublishedPackages = $publicPackageIds }) + $steps.Count | Should -Be 1 + @($steps | Where-Object task).Count | Should -Be 0 + } + } + + It 'runs the contract then Blob after successful no-ops for already-public Client and Abstractions' { + # Arrange + $prerequisites = 'Microsoft.DurableTask.Client', 'Microsoft.DurableTask.Abstractions' + $contract = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' + $blob = 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' + + # Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $prerequisites + + # Assert + $plan.Errors.Count | Should -Be 0 + foreach ($id in $prerequisites) { + $plan.Results[$id] | Should -Be 'Succeeded' + $plan.Submissions | Should -Not -Contain $id + Test-Path -LiteralPath "$fixture\esrp\$id" | Should -BeFalse + [array]::IndexOf($plan.Order, $id) | Should -BeLessThan ([array]::IndexOf($plan.Order, $contract)) + } + $plan.Results[$contract] | Should -Be 'Succeeded' + $plan.Results[$blob] | Should -Be 'Succeeded' + $plan.Submissions | Should -Contain $contract + $plan.Submissions | Should -Contain $blob + [array]::IndexOf($plan.Order, $contract) | Should -BeLessThan ([array]::IndexOf($plan.Order, $blob)) + } + + It 'can retry only Blob when the contract and other selected versions are already public' { + # Arrange + $blob = 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' + $published = @($publicPackageIds | Where-Object { $_ -ne $blob }) + + # Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $published + + # Assert + $plan.Errors.Count | Should -Be 0 + $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Succeeded' + $plan.Results[$blob] | Should -Be 'Succeeded' + $plan.Submissions.Count | Should -Be 1 + $plan.Submissions[0] | Should -Be $blob + @(Get-ChildItem -LiteralPath "$fixture\esrp" -Directory).Count | Should -Be 1 + Test-Path -LiteralPath "$fixture\esrp\Microsoft.DurableTask.LargePayloadPurge.Abstractions" | Should -BeFalse + } + + It 'gives a true hold precedence over an already-public confirmation for ' -TestCases @( + @{ Prerequisite = 'Microsoft.DurableTask.Client' } + @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } + @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } + ) { + param($Prerequisite) + # Arrange / Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $Prerequisite -SkipPackages $Prerequisite + + # Assert + $plan.Errors.Count | Should -Be 0 + $plan.Results[$Prerequisite] | Should -Be 'Skipped' + $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' + $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' + $plan.Submissions | Should -Not -Contain $Prerequisite + $plan.Logs -join "`n" | Should -Not -Match ([regex]::Escape("Operator confirmed the selected version of '$Prerequisite'")) + } + + It 'blocks dependents after a real staging failure in ' -TestCases @( + @{ Prerequisite = 'Microsoft.DurableTask.Client' } + @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } + @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } + ) { + param($Prerequisite) + # Arrange + Remove-Item -LiteralPath "$env:SOURCE_FOLDER\$Prerequisite.1.27.1.nupkg" + + # Act + $plan = Invoke-TestReleasePlan -Root $fixture + + # Assert + $plan.Results[$Prerequisite] | Should -Be 'Failed' + $plan.Errors[$Prerequisite] | Should -BeLike '*found 0*' + $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' + $plan.Submissions | Should -Not -Contain $Prerequisite + if ($Prerequisite -ne 'Microsoft.DurableTask.LargePayloadPurge.Abstractions') { + $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' + } + } + + It 'blocks dependents when the confirmed prerequisite job is canceled: ' -TestCases @( + @{ Prerequisite = 'Microsoft.DurableTask.Client' } + @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } + @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } + ) { + param($Prerequisite) + # Arrange + $overrides = @{} + $overrides[$Prerequisite] = 'Canceled' + + # Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $Prerequisite -ResultOverrides $overrides + + # Assert + $plan.Errors.Count | Should -Be 0 + $plan.Results[$Prerequisite] | Should -Be 'Canceled' + $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' + if ($Prerequisite -ne 'Microsoft.DurableTask.LargePayloadPurge.Abstractions') { + $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' + } + } + + It 'does not run even already-public no-ops when approval is ' -TestCases @( + @{ ApprovalResult = 'Failed' } + @{ ApprovalResult = 'Canceled' } + @{ ApprovalResult = 'Skipped' } + ) { + param($ApprovalResult) + # Arrange / Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $publicPackageIds -ApprovalResult $ApprovalResult + + # Assert + @($plan.Results.Values | Where-Object { $_ -eq 'Skipped' }).Count | Should -Be 17 + $plan.Submissions.Count | Should -Be 0 + $plan.Logs.Count | Should -Be 0 + Test-Path -LiteralPath "$fixture\esrp" | Should -BeFalse + } + + It 'blocks all public jobs when internal staging fails' { + # Arrange / Act + $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $publicPackageIds -AdoResult 'Failed' + + # Assert + @($plan.Results.Values | Where-Object { $_ -eq 'Skipped' }).Count | Should -Be 17 + $plan.Submissions.Count | Should -Be 0 + $plan.Logs.Count | Should -Be 0 + } + It 'selects the exact nuspec ID rather than a filename prefix or a misleading filename' { # Arrange New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.Client.2.Tools' diff --git a/eng/publish/publish.yml b/eng/publish/publish.yml index 89386bb2..07fcab84 100644 --- a/eng/publish/publish.yml +++ b/eng/publish/publish.yml @@ -8,7 +8,11 @@ parameters: type: string default: '' - name: skipPackages - displayName: Public package IDs to skip for this run (already published or intentionally held back) + displayName: Public package IDs intentionally held back (also blocks dependent releases) + type: object + default: [] +- name: alreadyPublishedPackages + displayName: Public package IDs whose selected versions are confirmed on NuGet.org (successful no-op) type: object default: [] @@ -77,7 +81,12 @@ extends: Confirm that this is the organization approved during ESRP NuGet onboarding. Confirm the Durable ESRP client is allowlisted for ESRPRELPACMAN and the NuGet Trusted Publishing policy covers the selected package IDs/prefixes and operations. - Validate the selected signed official build, package exclusions and dependency availability. + Intentional holds (skipPackages): ${{ join(', ', parameters.skipPackages) }} + Selected versions confirmed already public (alreadyPublishedPackages): ${{ join(', ', parameters.alreadyPublishedPackages) }} + Confirm the exact selected artifact version of each already-published package exists on NuGet.org. + Those jobs succeed without staging or ESRP submission so dependent releases can proceed. + Intentional holds take precedence over already-published confirmations and block dependents. + Validate the selected signed official build, both package lists and dependency availability. Stop here for a validation-only trial or if any prerequisite remains unverified. Approving this gate starts public ESRP NuGet submissions. onTimeout: 'reject' @@ -85,7 +94,8 @@ extends: #================== NuGet publishing section ================== # Keep one package per ESRP submission: duplicate versions fail independently, and ESRP # does not guarantee publication order within a submission. Preserve the dependency jobs below. - # skipPackages applies only to this run; it does not remove future releases of a package. + # Intentional holds skip jobs and block dependents; confirmed public versions use a successful + # no-op instead. A hold takes precedence if a package is in both per-run lists. # The shared steps select the exact nuspec ID, not a filename prefix, from the signed drop. # NuGet release (Microsoft.DurableTask.Abstractions) @@ -106,6 +116,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Abstractions organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Abstractions') }} # NuGet release (Microsoft.DurableTask.Client) - job: nugetRelease_Microsoft_DurableTask_Client @@ -125,6 +136,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Client organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Client') }} # NuGet release (Microsoft.DurableTask.Client.Grpc) - job: nugetRelease_Microsoft_DurableTask_Client_Grpc @@ -144,6 +156,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Client.Grpc organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Client.Grpc') }} # NuGet release (Microsoft.DurableTask.Client.OrchestrationServiceClientShim) - job: nugetRelease_Microsoft_DurableTask_Client_OrchestrationServiceClientShim @@ -163,6 +176,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Client.OrchestrationServiceClientShim organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim') }} # NuGet release (Microsoft.DurableTask.Generators) - job: nugetRelease_Microsoft_DurableTask_Generators @@ -182,6 +196,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Generators organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Generators') }} # NuGet release (Microsoft.DurableTask.Analyzers) - job: nugetRelease_Microsoft_DurableTask_Analyzers @@ -201,6 +216,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Analyzers organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Analyzers') }} # NuGet release (Microsoft.DurableTask.Grpc) - job: nugetRelease_Microsoft_DurableTask_Grpc @@ -220,6 +236,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Grpc organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Grpc') }} # NuGet release (Microsoft.DurableTask.Worker) - job: nugetRelease_Microsoft_DurableTask_Worker @@ -239,6 +256,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Worker organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Worker') }} # NuGet release (Microsoft.DurableTask.Worker.Grpc) - job: nugetRelease_Microsoft_DurableTask_Worker_Grpc @@ -258,6 +276,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Worker.Grpc organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Worker.Grpc') }} # NuGet release (Microsoft.DurableTask.Client.AzureManaged) - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged @@ -277,6 +296,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Client.AzureManaged organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Client.AzureManaged') }} # NuGet release (Microsoft.DurableTask.Client.AzureManaged.Sandboxes) - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged_Sandboxes @@ -296,6 +316,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Client.AzureManaged.Sandboxes organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Client.AzureManaged.Sandboxes') }} # NuGet release (Microsoft.DurableTask.Worker.AzureManaged) - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged @@ -315,6 +336,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Worker.AzureManaged organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Worker.AzureManaged') }} # NuGet release (Microsoft.DurableTask.Worker.AzureManaged.Sandboxes) - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged_Sandboxes @@ -334,6 +356,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Worker.AzureManaged.Sandboxes organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Worker.AzureManaged.Sandboxes') }} # NuGet release (Microsoft.DurableTask.ScheduledTasks) - job: nugetRelease_Microsoft_DurableTask_ScheduledTasks @@ -353,6 +376,7 @@ extends: parameters: packageId: Microsoft.DurableTask.ScheduledTasks organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.ScheduledTasks') }} # NuGet release (Microsoft.DurableTask.ExportHistory) - job: nugetRelease_Microsoft_DurableTask_ExportHistory @@ -372,6 +396,7 @@ extends: parameters: packageId: Microsoft.DurableTask.ExportHistory organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.ExportHistory') }} # add it for Microsoft.DurableTask.Extensions.AzureBlobPayloads - job: nugetRelease_Microsoft_DurableTask_Extensions_AzureBlobPayloads @@ -393,6 +418,7 @@ extends: parameters: packageId: Microsoft.DurableTask.Extensions.AzureBlobPayloads organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.Extensions.AzureBlobPayloads') }} # NuGet release (Microsoft.DurableTask.LargePayloadPurge.Abstractions) - job: nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions @@ -414,4 +440,5 @@ extends: - template: /eng/publish/nuget-release-steps.yml@self parameters: packageId: Microsoft.DurableTask.LargePayloadPurge.Abstractions - organizationName: ${{ parameters.nuGetOrganization }} \ No newline at end of file + organizationName: ${{ parameters.nuGetOrganization }} + alreadyPublished: ${{ containsValue(parameters.alreadyPublishedPackages, 'Microsoft.DurableTask.LargePayloadPurge.Abstractions') }} \ No newline at end of file From d09e80a048f93031baaf623659d59674fa7a4c1f Mon Sep 17 00:00:00 2001 From: wangbill Date: Tue, 27 Oct 2026 10:48:11 -0700 Subject: [PATCH 6/6] Trim ESRP publishing to the required release behavior Remove the unneeded intentional-hold input and restore the original job success conditions. Keep confirmed-public no-ops, exact signed-package staging and the manual approval safeguard. Replace the local scheduler simulation with direct script tests and migration invariants, and remove trial history from the release guide. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626 --- doc/release_process.md | 146 ++------ eng/publish/publish.Tests.ps1 | 646 ++++++---------------------------- eng/publish/publish.yml | 56 ++- 3 files changed, 163 insertions(+), 685 deletions(-) diff --git a/doc/release_process.md b/doc/release_process.md index de2ec2e1..63dfd3c2 100644 --- a/doc/release_process.md +++ b/doc/release_process.md @@ -24,22 +24,20 @@ NuGet signing and per-package approval-gated publication steps apply. Contract publication also waits for successful Client and Abstractions jobs. Each prerequisite must either publish successfully or complete an explicit successful no-op because the operator confirmed its selected artifact version is already on NuGet.org using `alreadyPublishedPackages`. If either job fails, -including a duplicate-version upload failure, or is intentionally held back, skipped or canceled, contract +including a duplicate-version upload failure, or is skipped or canceled, contract publication is skipped rather than treating that result as success. `Microsoft.DurableTask.Extensions.AzureBlobPayloads` depends on the contract package at every target framework, so its publication job also waits for successful contract publication (in addition to approval). If the contract job fails for any reason above, including a duplicate-version upload failure, or is -intentionally held back, skipped or canceled, Blob publication is skipped rather than treating that result +skipped or canceled, Blob publication is skipped rather than treating that result as success. This extends the same fail-closed chain: approval, then Client and Abstractions, then the contract, then Blob. An already-published confirmation succeeds without staging or submitting that package, allowing recovery of a partial release: confirm Client/Abstractions to retry the contract and Blob. For a Blob-only retry, confirm all selected versions already published, including Client, Abstractions and the contract; their jobs still follow the dependency chain but complete as no-ops. -Intentional holds belong in `skipPackages` and still block dependents. If a package appears in both -lists, the hold takes precedence; it is not permission to publish dependents. Both lists default to -empty and apply only to the selected run. Other packages retain their own independent publication -jobs gated only on approval. +The confirmation list defaults to empty and applies only to the selected run. Other packages retain +their own independent publication jobs gated only on approval. Keep its `RELEASENOTES.md`: `eng/targets/Release.targets` reads it into NuGet package metadata, whereas the root `CHANGELOG.md` retains repository release history. The shared target also appends a link using the @@ -123,33 +121,24 @@ After the PR is merged, follow the **Publishing** steps below. ## Publishing (After Release PR is Merged) -1. Select a successful, signed `durabletask-dotnet.official` build in the `azfunc/internal` project. - Reuse the approved build's `drop`; the publisher does not rebuild, repack, or change package versions. +1. Select a successful, signed [official build](https://dev.azure.com/azfunc/internal/_build?definitionId=657). + The publisher uses its `drop` unchanged; it does not rebuild or repack. 2. Validate the selected build's source commit, signatures, package contents, and exact versions. 3. Use [`durabletask-dotnet.publish`](https://dev.azure.com/azfunc/internal/_build?definitionId=1100) - (`eng/publish/publish.yml`). The publisher source ref and the `officialPipeline` artifact resource - are separate selections: explicitly pin the official run, not just its branch or "latest". - Normally use the reviewed publisher-source commit on `main` after the code-mirror pipeline has - mirrored it to ADO. The branch allowlist controls automatic mirror triggers; it does not rule out - a separately authorized manual invocation of the existing mirror workflow for an exact trial ref. - Do not expand the trigger allowlist or change mirror policies for a trial. Selecting the old - release tag as the publisher source loads the old publisher YAML, even when the correct signed - official artifact run is selected. + (`eng/publish/publish.yml`) with the reviewed publisher source. Select the signed build separately + through the `officialPipeline` artifact resource; do not rely on "latest" or assume the release tag + contains the current publisher. 4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding. There is no organization default; an unset or unresolved value fails before any ESRP submission. Set `alreadyPublishedPackages` to exact public package IDs only after verifying their **selected artifact versions** already exist on NuGet.org. Those jobs log a successful no-op, without package staging or an ESRP task, so dependent jobs can proceed. - Use `skipPackages` only for intentional holds: those jobs are skipped and still block dependents. - A hold takes precedence if a package is in both lists. These are per-run inputs, not permanent removals. 5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that succeeds does `nugetApproval` wait for manual approval. The gate displays the selected organization - and both package lists, and requires confirmation of NuGet onboarding, publisher allowlisting, - Trusted Publishing and each exact already-published version. - Do not approve until those prerequisites, the selected artifacts, both lists and dependency - availability have been checked. Stop at the gate for a validation-only trial. + and confirmed-public IDs. Verify onboarding, the exact selected versions and dependency availability + before approving. Stop at the gate for validation-only runs. Rejecting or timing out the gate prevents all public publication. -6. After approval, each public package not held or already confirmed published is copied unchanged into +6. After approval, each public package not already confirmed published is copied unchanged into its own ESRP folder using the exact nuspec ID, then submitted with `EsrpRelease@14` and `WaitForReleaseCompletion: true`. Internal-only/test packages and symbol packages are not submitted. Missing or multiple matching @@ -160,101 +149,20 @@ After the PR is merged, follow the **Publishing** steps below. ### ESRP NuGet prerequisites -The [ESRP NuGet guide](https://eng.ms/docs/microsoft-security/identity/trust-and-security-services/tss-release-distribute/tss-release-esrp-parent/oss-publishing/releasing-open-source/nuget) -requires channel onboarding and a NuGet.org Trusted Publishing policy for the customer's organization, -ESRP client identity, package IDs or delegated prefixes, and required operations (new IDs and/or new -versions). Confirm organization ownership and any collaborator access required by that onboarding. -Existing ESRP customers use the existing-customer onboarding process to enable the NuGet channel. - -The template reuses the Durable ESRP service connection `dtfx-internal-esrp-prod`, managed identity -`0b3ed1a4-0727-4a50-b82a-02c2bd9dec89`, tenant `33e01921-4d64-4f8c-a055-5bdaffd5e33d`, -vault `durable-esrp-akv`, and signing certificate `dts-esrp-cert`. NuGet distribution uses -`MainPublisher: ESRPRELPACMAN`, not the `durabletask-java` publisher used for other registries. -The client must be allowlisted for this publisher. `Owners` and `Approvers` use the individual -Microsoft identity of the person who queued the run (`Build.RequestedForEmail`), not a group alias. -The publishing pipeline must also be authorized to use this service connection; that ADO authorization -is separate from ESRP publisher allowlisting and NuGet Trusted Publishing permissions. - -**Durable-specific NuGet onboarding, organization choice, publisher allowlisting, and Trusted Publishing -permissions have not been verified by this source migration.** Package owner names alone do not prove -that either owner is the onboarded organization. Confirm these prerequisites with the ESRP/NuGet teams -before approving publication. A failed ESRP submission fails the job; there is no direct NuGet push fallback. - -### v1.27.1 trial selection - -The signed official drop has 18 `.nupkg` files: 14 public stable packages at `1.27.1`, -`ExportHistory` and `ScheduledTasks` at `1.27.1-preview.2`, the unchanged public -`Microsoft.DurableTask.Generators` at `2.1.0-preview.2`, and internal-only -`Microsoft.DurableTask.InProcessTestHost` at `0.2.3-preview.1`. -For this trial, set `alreadyPublishedPackages: [Microsoft.DurableTask.Generators]` and -`skipPackages: []` because that exact generator version is already public. Its job succeeds through a -logged no-op, not an ESRP submission. This leaves 16 public submission targets; the test host has no -public job. Both default lists are empty so future generator releases remain enabled. -Keep `officialPipeline` pinned to the approved signed run `311120` (build number `20261021.2`, -source `2a409da4ac7dcf03c4f25985337eb4623ea6394e`), while the publisher uses the new reviewed and mirrored -source containing this migration (normally the reviewed `main` commit). A run ID and build number -are not interchangeable. -Recheck exact-version absence for every intended target immediately before any submission; -search indexing alone is not proof that a version has never been published. - -#### Pinned trial request and queue permissions - -The pipeline's `skipPackages` and `alreadyPublishedPackages` parameters are YAML/UI lists. For the -trial, use an empty hold list and `[Microsoft.DurableTask.Generators]` as the already-published list. -For the verified REST request through the installed Azure DevOps -CLI SDK, however, `RunPipelineParameters.template_parameters` is typed as `{str}`: send -`templateParameters` as a map of strings, with the list serialized as JSON **inside a string**. -This request fragment illustrates the wire value; `` is a placeholder, not a default: - -```json -{ - "templateParameters": { - "nuGetOrganization": "", - "skipPackages": "[]", - "alreadyPublishedPackages": "[\"Microsoft.DurableTask.Generators\"]" - } -} -``` +Follow the [ESRP NuGet onboarding guide](https://eng.ms/docs/microsoft-security/identity/trust-and-security-services/tss-release-distribute/tss-release-esrp-parent/oss-publishing/releasing-open-source/nuget). +Confirm the organization owns the selected packages, the Durable client is allowlisted for +`ESRPRELPACMAN`, and Trusted Publishing covers the package IDs/prefixes and required operations. +The template reuses `dtfx-internal-esrp-prod`; access to that connection and appropriate +[pipeline permissions](https://learn.microsoft.com/azure/devops/pipelines/policies/permissions?view=azure-devops) +are separate prerequisites. Owners/approvers use the individual `Build.RequestedForEmail`, not a group. +This source change does not establish those permissions or onboarding. ESRP failures fail the job; +there is no direct NuGet push fallback. + +For Azure DevOps CLI SDK REST requests, `templateParameters` is a string map: serialize +`alreadyPublishedPackages` as a JSON string rather than a native array. Verify the resolved publisher +source, artifact resource and confirmed-public versions before running. -Do not send either list as a native JSON array through that client. In the earlier trial, that request -invalidated or ignored overrides and resolved default source/artifact selections instead. -Verify the resolved publisher source, official artifact run, organization bindings and exclusion -conditions in the actual expanded preview before attempting a run; an HTTP 200 alone is insufficient. - -Selecting nondefault pinned trial inputs requires **Edit queue build configuration** on -`internal` project pipeline `1100` (`durabletask-dotnet.publish`): Build security action -`EditPipelineQueueConfigurationPermission` (bit `65536`), distinct from **Queue builds** (bit `128`) -and **Edit build pipeline** (bit `2048`). If Azure DevOps returns -`TF215106` requesting that permission, stop and have an ADO administrator grant that narrow permission -to the intended requester before another attempt. Do not change pipeline defaults, switch -credentials/APIs, or loosen policies to bypass the denial. This queue-configuration permission is -separate from authorization to use `dtfx-internal-esrp-prod`, ESRP `ESRPRELPACMAN` allowlisting, -NuGet channel onboarding and Trusted Publishing. - -#### Observed trial status before partial-release recovery fix (2026-10-22) - -The tested runtime source is `261b233a828211dae2439afba6a9cf3601d79c5c`. Mirror run `311345` -succeeded and its locked internal ref matched that commit. The actual-source publisher preview -returned HTTP 200; scalar positive-control and corrected string-map previews confirmed all 17 -organization/environment bindings, the visible selected-organization/STOP approval instructions, -the Generator job condition set to `False`, and the remaining 16 runnable jobs with dependency-success -gates preserved. The organization used for compilation was a candidate, not verified onboarding. - -An earlier malformed array-valued request created run `311347` with default old `main` source -`2a409da4`, latest official run `311261`, and empty template parameters. It was canceled before any -manual approval; it is not evidence of a correctly pinned trial. -With the corrected request, official build `20261021.2`, and fresh exact-version HTTP 404s for all -16 targets, the queue attempt returned HTTP 500 / `TF215106` before assigning a run ID: -the requester lacked **Edit queue build configuration** on pipeline `1100`. -This is a hard ADO authorization blocker, not a source compilation failure or an observed -ESRP/backend/NuGet failure. No permission grant, policy change, approval or public distribution was -performed. Post-denial run inventory confirmed that the corrected request created no run and that -`311347` was completed/canceled; no accidental publisher remained active. -Later documentation-only commits did not replace `261b233a` as the exact runtime tested in that preview. -The subsequent source change adds `alreadyPublishedPackages` to distinguish successful no-ops from -intentional holds. It is not covered by that earlier server preview; the final recovery/holdback variants -need their own actual-source compilation before use. The queue-configuration and downstream onboarding -prerequisites remain separate and unresolved; no trial parameter change bypasses them. +### Local publisher checks For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run: @@ -262,8 +170,8 @@ For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell Invoke-Pester -Path .\eng\publish\publish.Tests.ps1 -CI ``` -These checks exercise local staging and pipeline structure only. They do not compile the server-side -1ES templates, validate backend permissions, queue a release, or publish a package. +These checks execute local staging/no-op scripts and check pipeline wiring, not Azure's scheduler. +They do not compile server-side templates or validate end-to-end publishing. ## Manual Release Process (Alternative) diff --git a/eng/publish/publish.Tests.ps1 b/eng/publish/publish.Tests.ps1 index 90257c7d..215d9925 100644 --- a/eng/publish/publish.Tests.ps1 +++ b/eng/publish/publish.Tests.ps1 @@ -3,331 +3,113 @@ BeforeAll { Import-Module powershell-yaml -ErrorAction Stop - $pipeline = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\publish.yml" -Raw) - $releaseTemplate = ConvertFrom-Yaml (Get-Content -LiteralPath "$PSScriptRoot\nuget-release-steps.yml" -Raw) - $publishSteps = @($releaseTemplate.steps[1]['${{ else }}']) + $pipeline = ConvertFrom-Yaml (Get-Content "$PSScriptRoot\publish.yml" -Raw) + $template = ConvertFrom-Yaml (Get-Content "$PSScriptRoot\nuget-release-steps.yml" -Raw) + $publishSteps = @($template.steps[1]['${{ else }}']) + $noOpSteps = @($template.steps[0]['${{ if parameters.alreadyPublished }}']) $stagePackage = [scriptblock]::Create($publishSteps[0].pwsh) + $confirmPublished = [scriptblock]::Create($noOpSteps[0].pwsh) $jobs = $pipeline.extends.parameters.stages[0].jobs $publicJobs = @($jobs | Where-Object job -Like 'nugetRelease_*') - $publicPackageIds = @( - 'Microsoft.DurableTask.Abstractions' - 'Microsoft.DurableTask.Client' - 'Microsoft.DurableTask.Client.Grpc' - 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim' - 'Microsoft.DurableTask.Generators' - 'Microsoft.DurableTask.Analyzers' - 'Microsoft.DurableTask.Grpc' - 'Microsoft.DurableTask.Worker' - 'Microsoft.DurableTask.Worker.Grpc' - 'Microsoft.DurableTask.Client.AzureManaged' - 'Microsoft.DurableTask.Client.AzureManaged.Sandboxes' - 'Microsoft.DurableTask.Worker.AzureManaged' - 'Microsoft.DurableTask.Worker.AzureManaged.Sandboxes' - 'Microsoft.DurableTask.ScheduledTasks' - 'Microsoft.DurableTask.ExportHistory' - 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' - 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' - ) function New-TestPackage { param( - [string]$Folder, - [string]$PackageId, - [string]$Version = '1.27.1', - [string]$FileName = "$PackageId.$Version.nupkg", - [string]$Nuspec = @" - - $PackageId$Version - -"@ + [string]$Id, + [string]$Version = '1.0.0', + [string]$FileName = "$Id.$Version.nupkg", + [string]$Nuspec = "$Id$Version" ) - - $path = Join-Path $Folder $FileName - $archive = [System.IO.Compression.ZipFile]::Open($path, [System.IO.Compression.ZipArchiveMode]::Create) + $archive = [IO.Compression.ZipFile]::Open((Join-Path $env:SOURCE_FOLDER $FileName), 'Create') try { - $writer = [System.IO.StreamWriter]::new($archive.CreateEntry("$PackageId.nuspec").Open()) - try { - $writer.Write($Nuspec) - } - finally { - $writer.Dispose() - } - } - finally { - $archive.Dispose() - } - } - - function Test-PackageListExpression { - param([string]$Expression, [hashtable]$Parameters) - - $expressionBody = $Expression -replace '^\$\{\{\s*|\s*\}\}$', '' - $negated = $expressionBody.StartsWith('not(') - if ($negated) { - $expressionBody = $expressionBody.Substring(4, $expressionBody.Length - 5) - } - $match = [regex]::Match($expressionBody, "^containsValue\(parameters\.(?\w+), '(?[^']+)'\)$") - if (-not $match.Success -or -not $Parameters.ContainsKey($match.Groups['list'].Value)) { - throw "Unsupported package-list expression: $Expression" - } - $included = @($Parameters[$match.Groups['list'].Value]) -contains $match.Groups['id'].Value - if ($negated) { - return -not $included - } - return $included - } - - function Get-ReleaseSteps { - param([System.Collections.IDictionary]$Job, [hashtable]$Parameters) - - $alreadyPublished = Test-PackageListExpression $Job.steps[0].parameters.alreadyPublished $Parameters - $branch = if ($alreadyPublished) { '${{ if parameters.alreadyPublished }}' } else { '${{ else }}' } - $branches = @($releaseTemplate.steps | Where-Object { $_.ContainsKey($branch) }) - if ($branches.Count -ne 1) { - throw "Expected one template branch for alreadyPublished=$alreadyPublished." - } - return $branches[0][$branch] - } - - # Model only the conditions emitted by this pipeline; unexpected expressions fail the local harness. - function Test-ReleaseJobCondition { - param([System.Collections.IDictionary]$Job, [hashtable]$Parameters, [hashtable]$Results) - - $condition = [regex]::Replace($Job.condition, '\$\{\{.*?\}\}', { - param($match) - (Test-PackageListExpression $match.Value $Parameters).ToString() - }) - $match = [regex]::Match($condition, "^and\(succeeded\((?:'(?[^']+)')?\), (?True|False)\)$") - if (-not $match.Success) { - throw "Unsupported job condition: $condition" - } - if ($match.Groups['included'].Value -eq 'False') { - return $false - } - $dependencies = if ($match.Groups['dependency'].Success) { @($match.Groups['dependency'].Value) } else { @($Job.dependsOn) } - return @($dependencies | Where-Object { $Results[$_] -notin 'Succeeded', 'SucceededWithIssues' }).Count -eq 0 - } - - function Invoke-TestReleasePlan { - param( - [string]$Root, - [string[]]$AlreadyPublishedPackages = @(), - [string[]]$SkipPackages = @(), - [hashtable]$ResultOverrides = @{}, - [string]$ApprovalResult = 'Succeeded', - [string]$AdoResult = 'Succeeded' - ) - - $parameters = @{ alreadyPublishedPackages = $AlreadyPublishedPackages; skipPackages = $SkipPackages } - $results = @{ AdoRelease = $AdoResult; nugetApproval = $(if ($AdoResult -eq 'Succeeded') { $ApprovalResult } else { 'Skipped' }) } - $pending = [Collections.Generic.List[object]]::new() - $publicJobs | ForEach-Object { $pending.Add($_) } - $order = [Collections.Generic.List[string]]::new() - $submissions = [Collections.Generic.List[string]]::new() - $logs = [Collections.Generic.List[string]]::new() - $errors = @{} - $savedEnvironment = @{} - foreach ($name in 'PACKAGE_ID', 'NUGET_ORGANIZATION', 'SOURCE_FOLDER', 'STAGING_FOLDER') { - $savedEnvironment[$name] = [Environment]::GetEnvironmentVariable($name) - } - try { - while ($pending.Count) { - $ready = @($pending | Where-Object { @($_.dependsOn | Where-Object { -not $results.ContainsKey($_) }).Count -eq 0 }) - if ($ready.Count -eq 0) { - throw 'The public job graph cannot make progress.' - } - foreach ($job in $ready) { - $packageId = $job.steps[0].parameters.packageId - if (-not (Test-ReleaseJobCondition $job $parameters $results)) { - $results[$job.job] = 'Skipped' - } - elseif ($ResultOverrides.ContainsKey($packageId)) { - $results[$job.job] = $ResultOverrides[$packageId] - } - else { - $order.Add($packageId) - try { - foreach ($step in Get-ReleaseSteps $job $parameters) { - if ($step.pwsh) { - foreach ($entry in $step.env.GetEnumerator()) { - $value = $entry.Value.Replace('${{ parameters.packageId }}', $packageId). - Replace('${{ parameters.organizationName }}', 'validation-only-organization'). - Replace('$(System.DefaultWorkingDirectory)', $Root) - [Environment]::SetEnvironmentVariable($entry.Key, $value) - } - & ([scriptblock]::Create($step.pwsh)) 6>&1 | ForEach-Object { $logs.Add([string]$_) } - } - elseif ($step.task -eq 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14') { - # Record the emitted task, never invoke a distribution tool in local tests. - $submissions.Add($packageId) - } - else { - throw 'Unsupported release step in local harness.' - } - } - $results[$job.job] = 'Succeeded' - } - catch { - $results[$job.job] = 'Failed' - $errors[$packageId] = $_.Exception.Message - } - } - [void]$pending.Remove($job) - } - } - } - finally { - foreach ($name in $savedEnvironment.Keys) { - [Environment]::SetEnvironmentVariable($name, $savedEnvironment[$name]) - } - } - $packageResults = @{} - foreach ($job in $publicJobs) { - $packageResults[$job.steps[0].parameters.packageId] = $results[$job.job] - } - return [pscustomobject]@{ - Results = $packageResults - Order = $order.ToArray() - Submissions = $submissions.ToArray() - Logs = $logs.ToArray() - Errors = $errors + $writer = [IO.StreamWriter]::new($archive.CreateEntry("$Id.nuspec").Open()) + try { $writer.Write($Nuspec) } + finally { $writer.Dispose() } } + finally { $archive.Dispose() } } } -Describe 'Publisher pipeline' { - It 'retains exactly the existing 17 public package jobs and signed artifact inputs' { - # Arrange / Act - $actualIds = @($publicJobs | ForEach-Object { $_.steps[0].parameters.packageId }) - - # Assert +Describe 'Publisher wiring' { + It 'retains 17 unique public identities with explicit self templates and per-package confirmations' { + $jobs.Count | Should -Be 19 $publicJobs.Count | Should -Be 17 - ($actualIds | Sort-Object) -join ',' | Should -Be (($publicPackageIds | Sort-Object) -join ',') + $ids = @($publicJobs | ForEach-Object { $_.steps[0].parameters.packageId }) + @($ids | Select-Object -Unique).Count | Should -Be 17 + $ids | Should -Not -Contain 'Microsoft.DurableTask.InProcessTestHost' foreach ($job in $publicJobs) { - $packageId = $job.steps[0].parameters.packageId - $job.job | Should -Be ("nugetRelease_" + $packageId.Replace('.', '_')) - $job.templateContext.type | Should -Be 'releaseJob' - $job.templateContext.isProduction | Should -BeTrue + $id = $job.steps[0].parameters.packageId + $job.job | Should -Be ("nugetRelease_" + $id.Replace('.', '_')) + $job.steps.Count | Should -Be 1 + $job.steps[0].template | Should -Be '/eng/publish/nuget-release-steps.yml@self' + $job.steps[0].parameters.organizationName | Should -Be '${{ parameters.nuGetOrganization }}' + $job.steps[0].parameters.alreadyPublished | Should -Be ("`${{ containsValue(parameters.alreadyPublishedPackages, '$id') }}") $job.templateContext.inputs[0].pipeline | Should -Be 'officialPipeline' $job.templateContext.inputs[0].artifactName | Should -Be 'drop' - $job.steps[0].parameters.organizationName | Should -Be '${{ parameters.nuGetOrganization }}' - Test-PackageListExpression $job.steps[0].parameters.alreadyPublished @{ alreadyPublishedPackages = @($packageId) } | Should -BeTrue - Test-PackageListExpression $job.steps[0].parameters.alreadyPublished @{ alreadyPublishedPackages = @() } | Should -BeFalse } + ($pipeline.parameters | Where-Object name -EQ 'nuGetOrganization').default | Should -BeNullOrEmpty + ($pipeline.parameters | Where-Object name -EQ 'alreadyPublishedPackages').default.Count | Should -Be 0 + ($template.parameters | Where-Object name -EQ 'alreadyPublished').default | Should -BeFalse } - It 'anchors every NuGet step template to self when passed through the external 1ES stage template' { - # Arrange - $expectedTemplate = '/eng/publish/nuget-release-steps.yml@self' - - # Act - $references = @($publicJobs | ForEach-Object { $_.steps[0].template }) - - # Assert - $references.Count | Should -Be 17 - foreach ($reference in $references) { - $reference | Should -Be $expectedTemplate - } - } - - It 'preserves internal staging and waits for it before manual NuGet approval' { - # Arrange - $ado = $jobs | Where-Object job -EQ 'AdoRelease' - $approval = $jobs | Where-Object job -EQ 'nugetApproval' - - # Act / Assert + It 'preserves internal staging, approval and dependency success gates' { $pipeline.trigger | Should -Be 'none' $pipeline.pr | Should -Be 'none' - $pipeline.resources.pipelines[0].source | Should -Be 'durabletask-dotnet.official' + $ado = $jobs | Where-Object job -EQ 'AdoRelease' $ado.steps[0].task | Should -Be '1ES.PublishNuget@1' - $ado.steps[0].inputs.publishVstsFeed | Should -Be '3f99e810-c336-441f-8892-84983093ad7f/c895696b-ce37-4fe7-b7ce-74333a04f8bf' $ado.steps[0].inputs.allowPackageConflicts | Should -BeTrue - $ado.steps[0].inputs.packagesToPush | Should -Be '$(System.DefaultWorkingDirectory)/**/*.nupkg;!$(System.DefaultWorkingDirectory)/**/*.symbols.nupkg' + $approval = $jobs | Where-Object job -EQ 'nugetApproval' $approval.dependsOn | Should -Be 'AdoRelease' $approval.pool | Should -Be 'server' $approval.steps[0].task | Should -Be 'ManualValidation@1' $approval.steps[0].inputs.onTimeout | Should -Be 'reject' - $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape('${{ parameters.nuGetOrganization }}')) - $approval.steps[0].inputs.instructions | Should -Match 'ESRP NuGet onboarding' - $approval.steps[0].inputs.instructions | Should -Match 'ESRPRELPACMAN' - $approval.steps[0].inputs.instructions | Should -Match 'Trusted Publishing' - $approval.steps[0].inputs.instructions | Should -Match 'Stop here for a validation-only trial' - $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape("join(', ', parameters.skipPackages)")) - $approval.steps[0].inputs.instructions | Should -Match ([regex]::Escape("join(', ', parameters.alreadyPublishedPackages)")) - $approval.steps[0].inputs.instructions | Should -Match 'exact selected artifact version' - $approval.steps[0].inputs.instructions | Should -Match 'Intentional holds take precedence' - } - - It 'retains fail-closed dependency gates and run-scoped exclusions for every public job' { - # Arrange / Act / Assert + $approval.steps[0].inputs.instructions | Should -Match 'validation-only' + $approval.steps[0].inputs.instructions | Should -Match 'real public ESRP' foreach ($job in $publicJobs) { - $packageId = $job.steps[0].parameters.packageId @($job.dependsOn) | Should -Contain 'nugetApproval' - $job.condition | Should -Match ([regex]::Escape("not(containsValue(parameters.skipPackages, '$packageId'))")) - $job.condition | Should -Match '^and\(succeeded\(' + $expected = if (@($job.dependsOn).Count -eq 1) { "succeeded('nugetApproval')" } else { 'succeeded()' } + $job.condition | Should -Be $expected } - $contract = $jobs | Where-Object job -EQ 'nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions' - ($contract.dependsOn -join ',') | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_Abstractions,nugetRelease_Microsoft_DurableTask_Client' - $contract.condition | Should -Match '^and\(succeeded\(\),' + $contract.dependsOn -join ',' | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_Abstractions,nugetRelease_Microsoft_DurableTask_Client' $blob = $jobs | Where-Object job -EQ 'nugetRelease_Microsoft_DurableTask_Extensions_AzureBlobPayloads' - ($blob.dependsOn -join ',') | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions' - $blob.condition | Should -Match '^and\(succeeded\(\),' - } - - It 'requires an explicit organization and leaves future generator releases enabled by default' { - # Arrange / Act - $organization = $pipeline.parameters | Where-Object name -EQ 'nuGetOrganization' - $holds = $pipeline.parameters | Where-Object name -EQ 'skipPackages' - $published = $pipeline.parameters | Where-Object name -EQ 'alreadyPublishedPackages' - $noOp = $releaseTemplate.parameters | Where-Object name -EQ 'alreadyPublished' - - # Assert - $organization.default | Should -BeNullOrEmpty - $holds.type | Should -Be 'object' - $holds.default.Count | Should -Be 0 - $published.type | Should -Be 'object' - $published.default.Count | Should -Be 0 - $noOp.type | Should -Be 'boolean' - $noOp.default | Should -BeFalse - $publicPackageIds | Should -Contain 'Microsoft.DurableTask.Generators' + $blob.dependsOn -join ',' | Should -Be 'nugetApproval,nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions' } - It 'uses the Durable ESRP14 NuGet profile without a direct public-push fallback' { - # Arrange / Act - $release = $publishSteps[1] - - # Assert + It 'keeps distribution only in the normal branch with the Durable ESRP14 profile' { + $noOpSteps.Count | Should -Be 1 + @($noOpSteps | Where-Object task).Count | Should -Be 0 $publishSteps.Count | Should -Be 2 - $release.task | Should -Be 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14' - $release.inputs.ConnectedServiceName | Should -Be 'dtfx-internal-esrp-prod' - $release.inputs.ClientId | Should -Be '0b3ed1a4-0727-4a50-b82a-02c2bd9dec89' - $release.inputs.DomainTenantId | Should -Be '33e01921-4d64-4f8c-a055-5bdaffd5e33d' - $release.inputs.UseManagedIdentity | Should -BeTrue - $release.inputs.KeyVaultName | Should -Be 'durable-esrp-akv' - $release.inputs.SignCertName | Should -Be 'dts-esrp-cert' - $release.inputs.Intent | Should -Be 'PackageDistribution' - $release.inputs.ContentType | Should -Be 'NuGet' - $release.inputs.OrganizationName | Should -Be '${{ parameters.organizationName }}' - $release.inputs.ContentSource | Should -Be 'Folder' - $release.inputs.FolderLocation | Should -Be $publishSteps[0].env.STAGING_FOLDER - $release.inputs.WaitForReleaseCompletion | Should -BeTrue - $release.inputs.MainPublisher | Should -Be 'ESRPRELPACMAN' - $release.inputs.Owners | Should -Be '$(Build.RequestedForEmail)' - $release.inputs.Approvers | Should -Be '$(Build.RequestedForEmail)' - @($jobs.steps | Where-Object task -EQ '1ES.PublishNuget@1').Count | Should -Be 1 + $publishSteps[1].task | Should -Be 'SFP.release-tasks.custom-build-release-task.EsrpRelease@14' + $expected = @{ + ConnectedServiceName = 'dtfx-internal-esrp-prod' + ClientId = '0b3ed1a4-0727-4a50-b82a-02c2bd9dec89' + DomainTenantId = '33e01921-4d64-4f8c-a055-5bdaffd5e33d' + UseManagedIdentity = $true + KeyVaultName = 'durable-esrp-akv' + SignCertName = 'dts-esrp-cert' + Intent = 'PackageDistribution' + ContentType = 'NuGet' + ContentSource = 'Folder' + OrganizationName = '${{ parameters.organizationName }}' + FolderLocation = $publishSteps[0].env.STAGING_FOLDER + MainPublisher = 'ESRPRELPACMAN' + WaitForReleaseCompletion = $true + Owners = '$(Build.RequestedForEmail)' + Approvers = '$(Build.RequestedForEmail)' + } + foreach ($key in $expected.Keys) { + $publishSteps[1].inputs[$key] | Should -Be $expected[$key] + } } } -Describe 'ESRP package staging' { +Describe 'Actual package scripts' { BeforeAll { - $environmentNames = 'PACKAGE_ID', 'NUGET_ORGANIZATION', 'SOURCE_FOLDER', 'STAGING_FOLDER' $savedEnvironment = @{} - foreach ($name in $environmentNames) { + foreach ($name in 'PACKAGE_ID', 'NUGET_ORGANIZATION', 'SOURCE_FOLDER', 'STAGING_FOLDER') { $savedEnvironment[$name] = [Environment]::GetEnvironmentVariable($name) } } - BeforeEach { $fixture = Join-Path $TestDrive ([guid]::NewGuid().ToString()) $env:SOURCE_FOLDER = Join-Path $fixture 'drop' @@ -335,289 +117,89 @@ Describe 'ESRP package staging' { $env:PACKAGE_ID = 'Microsoft.DurableTask.Client' $env:NUGET_ORGANIZATION = 'validation-only-organization' New-Item -ItemType Directory -Path $env:SOURCE_FOLDER | Out-Null - foreach ($id in $publicPackageIds) { - $version = switch ($id) { - 'Microsoft.DurableTask.Generators' { '2.1.0-preview.2' } - 'Microsoft.DurableTask.ScheduledTasks' { '1.27.1-preview.2' } - 'Microsoft.DurableTask.ExportHistory' { '1.27.1-preview.2' } - default { '1.27.1' } - } - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $id -Version $version - } - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.InProcessTestHost' -Version '0.2.3-preview.1' + New-TestPackage $env:PACKAGE_ID } - AfterAll { - foreach ($name in $environmentNames) { + foreach ($name in $savedEnvironment.Keys) { [Environment]::SetEnvironmentVariable($name, $savedEnvironment[$name]) } } - It 'stages exactly 16 unchanged trial packages from the 18-package drop, excluding the old generator and test host' { - # Arrange - @(Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg').Count | Should -Be 18 - - # Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages 'Microsoft.DurableTask.Generators' + It 'copies only the exact nuspec ID unchanged, excluding prefixes, disguised internal packages and symbols' { + $original = Join-Path $env:SOURCE_FOLDER "$env:PACKAGE_ID.1.0.0.nupkg" + $hash = (Get-FileHash $original).Hash + New-TestPackage 'Microsoft.DurableTask.Client.2.Tools' + New-TestPackage 'Microsoft.DurableTask.InProcessTestHost' -FileName 'Microsoft.DurableTask.Client.100.0.0.nupkg' + New-TestPackage $env:PACKAGE_ID -FileName "$env:PACKAGE_ID.1.0.0.symbols.nupkg" + Set-Content "$env:SOURCE_FOLDER\$env:PACKAGE_ID.1.0.0.snupkg" 'not a product package' - # Assert - $plan.Errors.Count | Should -Be 0 - $plan.Submissions.Count | Should -Be 16 - $plan.Results['Microsoft.DurableTask.Generators'] | Should -Be 'Succeeded' - $plan.Logs -join "`n" | Should -Match "Operator confirmed.*Microsoft.DurableTask.Generators.*No package will be staged or submitted" - Test-Path -LiteralPath "$fixture\esrp\Microsoft.DurableTask.Generators" | Should -BeFalse - foreach ($packageId in $plan.Submissions) { - $staged = @(Get-ChildItem -LiteralPath "$fixture\esrp\$packageId" -File) - $staged.Count | Should -Be 1 - $original = Join-Path $env:SOURCE_FOLDER $staged[0].Name - (Get-FileHash -LiteralPath $staged[0].FullName).Hash | Should -Be (Get-FileHash -LiteralPath $original).Hash - $staged[0].Name | Should -Not -Match 'Generators|InProcessTestHost|symbols' - } - } - - It 'emits and executes only a logged no-op for already-public versions, even without a source folder' { - # Arrange - $noArtifacts = Join-Path $fixture 'no-artifacts' - - # Act - $plan = Invoke-TestReleasePlan -Root $noArtifacts -AlreadyPublishedPackages $publicPackageIds - - # Assert - $plan.Errors.Count | Should -Be 0 - @($plan.Results.Values | Where-Object { $_ -eq 'Succeeded' }).Count | Should -Be 17 - $plan.Submissions.Count | Should -Be 0 - @($plan.Logs | Where-Object { $_ -match 'Operator confirmed.*No package will be staged or submitted' }).Count | Should -Be 17 - Test-Path -LiteralPath $noArtifacts | Should -BeFalse - foreach ($job in $publicJobs) { - $steps = @(Get-ReleaseSteps $job @{ alreadyPublishedPackages = $publicPackageIds }) - $steps.Count | Should -Be 1 - @($steps | Where-Object task).Count | Should -Be 0 - } - } - - It 'runs the contract then Blob after successful no-ops for already-public Client and Abstractions' { - # Arrange - $prerequisites = 'Microsoft.DurableTask.Client', 'Microsoft.DurableTask.Abstractions' - $contract = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' - $blob = 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' - - # Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $prerequisites - - # Assert - $plan.Errors.Count | Should -Be 0 - foreach ($id in $prerequisites) { - $plan.Results[$id] | Should -Be 'Succeeded' - $plan.Submissions | Should -Not -Contain $id - Test-Path -LiteralPath "$fixture\esrp\$id" | Should -BeFalse - [array]::IndexOf($plan.Order, $id) | Should -BeLessThan ([array]::IndexOf($plan.Order, $contract)) - } - $plan.Results[$contract] | Should -Be 'Succeeded' - $plan.Results[$blob] | Should -Be 'Succeeded' - $plan.Submissions | Should -Contain $contract - $plan.Submissions | Should -Contain $blob - [array]::IndexOf($plan.Order, $contract) | Should -BeLessThan ([array]::IndexOf($plan.Order, $blob)) - } - - It 'can retry only Blob when the contract and other selected versions are already public' { - # Arrange - $blob = 'Microsoft.DurableTask.Extensions.AzureBlobPayloads' - $published = @($publicPackageIds | Where-Object { $_ -ne $blob }) - - # Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $published - - # Assert - $plan.Errors.Count | Should -Be 0 - $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Succeeded' - $plan.Results[$blob] | Should -Be 'Succeeded' - $plan.Submissions.Count | Should -Be 1 - $plan.Submissions[0] | Should -Be $blob - @(Get-ChildItem -LiteralPath "$fixture\esrp" -Directory).Count | Should -Be 1 - Test-Path -LiteralPath "$fixture\esrp\Microsoft.DurableTask.LargePayloadPurge.Abstractions" | Should -BeFalse - } - - It 'gives a true hold precedence over an already-public confirmation for ' -TestCases @( - @{ Prerequisite = 'Microsoft.DurableTask.Client' } - @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } - @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } - ) { - param($Prerequisite) - # Arrange / Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $Prerequisite -SkipPackages $Prerequisite - - # Assert - $plan.Errors.Count | Should -Be 0 - $plan.Results[$Prerequisite] | Should -Be 'Skipped' - $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' - $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' - $plan.Submissions | Should -Not -Contain $Prerequisite - $plan.Logs -join "`n" | Should -Not -Match ([regex]::Escape("Operator confirmed the selected version of '$Prerequisite'")) - } - - It 'blocks dependents after a real staging failure in ' -TestCases @( - @{ Prerequisite = 'Microsoft.DurableTask.Client' } - @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } - @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } - ) { - param($Prerequisite) - # Arrange - Remove-Item -LiteralPath "$env:SOURCE_FOLDER\$Prerequisite.1.27.1.nupkg" - - # Act - $plan = Invoke-TestReleasePlan -Root $fixture - - # Assert - $plan.Results[$Prerequisite] | Should -Be 'Failed' - $plan.Errors[$Prerequisite] | Should -BeLike '*found 0*' - $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' - $plan.Submissions | Should -Not -Contain $Prerequisite - if ($Prerequisite -ne 'Microsoft.DurableTask.LargePayloadPurge.Abstractions') { - $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' - } - } - - It 'blocks dependents when the confirmed prerequisite job is canceled: ' -TestCases @( - @{ Prerequisite = 'Microsoft.DurableTask.Client' } - @{ Prerequisite = 'Microsoft.DurableTask.Abstractions' } - @{ Prerequisite = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } - ) { - param($Prerequisite) - # Arrange - $overrides = @{} - $overrides[$Prerequisite] = 'Canceled' - - # Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $Prerequisite -ResultOverrides $overrides - - # Assert - $plan.Errors.Count | Should -Be 0 - $plan.Results[$Prerequisite] | Should -Be 'Canceled' - $plan.Results['Microsoft.DurableTask.Extensions.AzureBlobPayloads'] | Should -Be 'Skipped' - if ($Prerequisite -ne 'Microsoft.DurableTask.LargePayloadPurge.Abstractions') { - $plan.Results['Microsoft.DurableTask.LargePayloadPurge.Abstractions'] | Should -Be 'Skipped' - } - } - - It 'does not run even already-public no-ops when approval is ' -TestCases @( - @{ ApprovalResult = 'Failed' } - @{ ApprovalResult = 'Canceled' } - @{ ApprovalResult = 'Skipped' } - ) { - param($ApprovalResult) - # Arrange / Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $publicPackageIds -ApprovalResult $ApprovalResult - - # Assert - @($plan.Results.Values | Where-Object { $_ -eq 'Skipped' }).Count | Should -Be 17 - $plan.Submissions.Count | Should -Be 0 - $plan.Logs.Count | Should -Be 0 - Test-Path -LiteralPath "$fixture\esrp" | Should -BeFalse - } - - It 'blocks all public jobs when internal staging fails' { - # Arrange / Act - $plan = Invoke-TestReleasePlan -Root $fixture -AlreadyPublishedPackages $publicPackageIds -AdoResult 'Failed' - - # Assert - @($plan.Results.Values | Where-Object { $_ -eq 'Skipped' }).Count | Should -Be 17 - $plan.Submissions.Count | Should -Be 0 - $plan.Logs.Count | Should -Be 0 - } - - It 'selects the exact nuspec ID rather than a filename prefix or a misleading filename' { - # Arrange - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.Client.2.Tools' - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Microsoft.DurableTask.InProcessTestHost' -FileName 'Microsoft.DurableTask.Client.100.0.0.nupkg' - - # Act & $stagePackage 6>$null - # Assert - $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) + $staged = @(Get-ChildItem $env:STAGING_FOLDER -File) $staged.Count | Should -Be 1 - $staged[0].Name | Should -Be 'Microsoft.DurableTask.Client.1.27.1.nupkg' + $staged[0].Name | Should -Be "$env:PACKAGE_ID.1.0.0.nupkg" + (Get-FileHash $staged[0].FullName).Hash | Should -Be $hash + (Get-FileHash $original).Hash | Should -Be $hash } - It 'excludes legacy symbols and snupkg files even when their package ID matches' { - # Arrange - Set-Content -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Client.1.27.1.symbols.nupkg" -Value 'not a product package' - Set-Content -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Client.1.27.1.snupkg" -Value 'not a product package' - - # Act - & $stagePackage 6>$null - - # Assert - @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File).Count | Should -Be 1 - Test-Path -LiteralPath "$env:STAGING_FOLDER\Microsoft.DurableTask.Client.1.27.1.nupkg" | Should -BeTrue + It 'logs an already-public without reading or staging packages' -TestCases @( + @{ Id = 'Microsoft.DurableTask.Client' } + @{ Id = 'Microsoft.DurableTask.Abstractions' } + @{ Id = 'Microsoft.DurableTask.LargePayloadPurge.Abstractions' } + @{ Id = 'Microsoft.DurableTask.Generators' } + ) { + param($Id) + $env:PACKAGE_ID = $Id + $env:SOURCE_FOLDER = Join-Path $fixture 'not-created' + $output = @(& $confirmPublished 6>&1) -join "`n" + $output | Should -Match ([regex]::Escape("Operator confirmed the selected version of '$Id'")) + $output | Should -Match 'No package will be staged or submitted to ESRP' + Test-Path $env:SOURCE_FOLDER | Should -BeFalse + Test-Path $env:STAGING_FOLDER | Should -BeFalse } - It 'can stage a future generator version without a permanent generator exclusion' { - # Arrange + It 'stages a new Generator version through the normal branch' { $env:PACKAGE_ID = 'Microsoft.DurableTask.Generators' - Remove-Item -LiteralPath "$env:SOURCE_FOLDER\Microsoft.DurableTask.Generators.2.1.0-preview.2.nupkg" - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $env:PACKAGE_ID -Version '2.1.0-preview.3' - - # Act + New-TestPackage $env:PACKAGE_ID -Version '2.0.0-preview.1' & $stagePackage 6>$null - - # Assert - $staged = @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File) - $staged.Count | Should -Be 1 - $staged[0].Name | Should -Be 'Microsoft.DurableTask.Generators.2.1.0-preview.3.nupkg' + @(Get-ChildItem $env:STAGING_FOLDER -File).Name | Should -Be 'Microsoft.DurableTask.Generators.2.0.0-preview.1.nupkg' } - It 'rejects an unset or unresolved organization: ' -TestCases @( + It 'rejects unset or unresolved organization ' -TestCases @( @{ Organization = '' } @{ Organization = ' ' } @{ Organization = '$(NuGetOrganizationName)' } @{ Organization = '${{ parameters.organizationName }}' } ) { param($Organization) - # Arrange $env:NUGET_ORGANIZATION = $Organization - - # Act / Assert { & $stagePackage 6>$null } | Should -Throw '*Set nuGetOrganization*' - Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + Test-Path $env:STAGING_FOLDER | Should -BeFalse } - It 'rejects a missing package without creating a submission folder' { - # Arrange - $env:PACKAGE_ID = 'Microsoft.DurableTask.Missing' - - # Act / Assert - { & $stagePackage 6>$null } | Should -Throw '*found 0*' - Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse - } - - It 'rejects multiple versions of a package rather than choosing one silently' { - # Arrange - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId $env:PACKAGE_ID -Version '1.27.0' - - # Act / Assert - { & $stagePackage 6>$null } | Should -Throw '*found 2*' - Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + It 'rejects packages' -TestCases @( + @{ Problem = 'missing'; Count = 0 } + @{ Problem = 'ambiguous'; Count = 2 } + ) { + param($Problem, $Count) + if ($Problem -eq 'missing') { $env:PACKAGE_ID = 'Microsoft.DurableTask.Missing' } + else { New-TestPackage $env:PACKAGE_ID -Version '1.0.1' } + { & $stagePackage 6>$null } | Should -Throw "*found $Count*" + Test-Path $env:STAGING_FOLDER | Should -BeFalse } - It 'rejects malformed package metadata' { - # Arrange - New-TestPackage -Folder $env:SOURCE_FOLDER -PackageId 'Invalid' -Nuspec '' - - # Act / Assert - { & $stagePackage 6>$null } | Should -Throw "*Missing package ID in 'Invalid.1.27.1.nupkg'*" - Test-Path -LiteralPath $env:STAGING_FOLDER | Should -BeFalse + It 'rejects missing package metadata' { + New-TestPackage 'Invalid' -Nuspec '' + { & $stagePackage 6>$null } | Should -Throw '*Missing package ID*' + Test-Path $env:STAGING_FOLDER | Should -BeFalse } - It 'refuses to reuse or delete a nonempty staging folder' { - # Arrange + It 'refuses a stale staging folder without deleting its contents' { New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null - $stalePackage = Join-Path $env:STAGING_FOLDER 'stale.nupkg' - Set-Content -LiteralPath $stalePackage -Value 'stale content' - - # Act / Assert + $stale = Join-Path $env:STAGING_FOLDER 'stale.nupkg' + Set-Content $stale 'stale content' { & $stagePackage 6>$null } | Should -Throw '*refusing to reuse*' - Get-Content -LiteralPath $stalePackage | Should -Be 'stale content' - @(Get-ChildItem -LiteralPath $env:STAGING_FOLDER -File).Count | Should -Be 1 + Get-Content $stale | Should -Be 'stale content' + @(Get-ChildItem $env:STAGING_FOLDER -File).Count | Should -Be 1 } } diff --git a/eng/publish/publish.yml b/eng/publish/publish.yml index 07fcab84..1f424d7b 100644 --- a/eng/publish/publish.yml +++ b/eng/publish/publish.yml @@ -7,10 +7,6 @@ parameters: displayName: NuGet.org organization confirmed during ESRP onboarding (required) type: string default: '' -- name: skipPackages - displayName: Public package IDs intentionally held back (also blocks dependent releases) - type: object - default: [] - name: alreadyPublishedPackages displayName: Public package IDs whose selected versions are confirmed on NuGet.org (successful no-op) type: object @@ -78,31 +74,23 @@ extends: inputs: instructions: | Selected NuGet.org organization: ${{ parameters.nuGetOrganization }} - Confirm that this is the organization approved during ESRP NuGet onboarding. - Confirm the Durable ESRP client is allowlisted for ESRPRELPACMAN and the NuGet - Trusted Publishing policy covers the selected package IDs/prefixes and operations. - Intentional holds (skipPackages): ${{ join(', ', parameters.skipPackages) }} - Selected versions confirmed already public (alreadyPublishedPackages): ${{ join(', ', parameters.alreadyPublishedPackages) }} - Confirm the exact selected artifact version of each already-published package exists on NuGet.org. - Those jobs succeed without staging or ESRP submission so dependent releases can proceed. - Intentional holds take precedence over already-published confirmations and block dependents. - Validate the selected signed official build, both package lists and dependency availability. - Stop here for a validation-only trial or if any prerequisite remains unverified. - Approving this gate starts public ESRP NuGet submissions. + Confirmed public package IDs: ${{ join(', ', parameters.alreadyPublishedPackages) }} + Verify the signed official artifacts, dependencies and exact selected versions of the confirmed public IDs. + Confirm ESRP NuGet onboarding, ESRPRELPACMAN allowlisting and Trusted Publishing for the selected organization. + Stop here for validation-only runs or any unverified prerequisite. + Approval starts real public ESRP NuGet submissions. onTimeout: 'reject' #================== NuGet publishing section ================== # Keep one package per ESRP submission: duplicate versions fail independently, and ESRP # does not guarantee publication order within a submission. Preserve the dependency jobs below. - # Intentional holds skip jobs and block dependents; confirmed public versions use a successful - # no-op instead. A hold takes precedence if a package is in both per-run lists. # The shared steps select the exact nuspec ID, not a filename prefix, from the signed drop. # NuGet release (Microsoft.DurableTask.Abstractions) - job: nugetRelease_Microsoft_DurableTask_Abstractions displayName: NuGet Release (Microsoft.DurableTask.Abstractions) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Abstractions')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -122,7 +110,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Client displayName: NuGet Release (Microsoft.DurableTask.Client) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -142,7 +130,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Client_Grpc displayName: NuGet Release (Microsoft.DurableTask.Client.Grpc) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.Grpc')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -162,7 +150,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Client_OrchestrationServiceClientShim displayName: NuGet Release (Microsoft.DurableTask.Client.OrchestrationServiceClientShim) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.OrchestrationServiceClientShim')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -182,7 +170,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Generators displayName: NuGet Release (Microsoft.DurableTask.Generators) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Generators')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -202,7 +190,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Analyzers displayName: NuGet Release (Microsoft.DurableTask.Analyzers) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Analyzers')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -222,7 +210,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Grpc displayName: NuGet Release (Microsoft.DurableTask.Grpc) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Grpc')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -242,7 +230,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Worker displayName: NuGet Release (Microsoft.DurableTask.Worker) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -262,7 +250,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Worker_Grpc displayName: NuGet Release (Microsoft.DurableTask.Worker.Grpc) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.Grpc')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -282,7 +270,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged displayName: NuGet Release (Microsoft.DurableTask.Client.AzureManaged) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.AzureManaged')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -302,7 +290,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Client_AzureManaged_Sandboxes displayName: NuGet Release (Microsoft.DurableTask.Client.AzureManaged.Sandboxes) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Client.AzureManaged.Sandboxes')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -322,7 +310,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged displayName: NuGet Release (Microsoft.DurableTask.Worker.AzureManaged) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.AzureManaged')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -342,7 +330,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_Worker_AzureManaged_Sandboxes displayName: NuGet Release (Microsoft.DurableTask.Worker.AzureManaged.Sandboxes) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Worker.AzureManaged.Sandboxes')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -362,7 +350,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_ScheduledTasks displayName: NuGet Release (Microsoft.DurableTask.ScheduledTasks) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.ScheduledTasks')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -382,7 +370,7 @@ extends: - job: nugetRelease_Microsoft_DurableTask_ExportHistory displayName: NuGet Release (Microsoft.DurableTask.ExportHistory) dependsOn: nugetApproval - condition: and(succeeded('nugetApproval'), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.ExportHistory')) }}) + condition: succeeded('nugetApproval') templateContext: type: releaseJob isProduction: true @@ -404,7 +392,7 @@ extends: dependsOn: - nugetApproval - nugetRelease_Microsoft_DurableTask_LargePayloadPurge_Abstractions - condition: and(succeeded(), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.Extensions.AzureBlobPayloads')) }}) + condition: succeeded() templateContext: type: releaseJob isProduction: true @@ -427,7 +415,7 @@ extends: - nugetApproval - nugetRelease_Microsoft_DurableTask_Abstractions - nugetRelease_Microsoft_DurableTask_Client - condition: and(succeeded(), ${{ not(containsValue(parameters.skipPackages, 'Microsoft.DurableTask.LargePayloadPurge.Abstractions')) }}) + condition: succeeded() templateContext: type: releaseJob isProduction: true