Simplify DocumentExtraction CI and enable releases

.github/copilot-instructions.md

@@ -1,12 +1,14 @@
 # NuGet Publishing
 
-Packages are published via GitHub Actions triggered by git tags. Each component workflow (e.g. `mevd.yml`) owns its build, test, and pack steps; on tag pushes it calls the generic `sign-and-publish.yml` reusable workflow to sign and publish to NuGet.org.
+Packages are published via GitHub Actions triggered by git tags. Each component workflow (e.g. `mevd.yml` or `document-extraction.yml`) owns its build, test, and pack steps; on tag pushes it calls the generic `release.yml` reusable workflow to sign and publish to NuGet.org.
 
 ## Tag format
 
 - `<component>/v<version>` — release all packages in a component (e.g. `mevd/v1.0.0-preview.1`)
 - `<component>/<subcomponent>/v<version>` — release a single package (e.g. `mevd/PgVector/v1.0.0-preview.1`)
 
+DocumentExtraction uses the `document-extraction` tag prefix (e.g. `document-extraction/PdfPig/v1.0.0-preview.1`).
+
 The subcomponent name must match the directory name under the component's `src/` folder exactly (e.g. `PgVector`, `Qdrant`, `InMemory`).
 
 Preview/prerelease versions use standard NuGet prerelease suffixes in the tag (e.g. `v1.0.0-preview.1`).
@@ -15,15 +17,15 @@ Preview/prerelease versions use standard NuGet prerelease suffixes in the tag (e
 
 When changing C# files under `MEVD/src/<provider>/`, also update the `<Version>` in the provider's `.csproj`. Choose the appropriate SemVer increment for the change. Changes limited to tests, documentation, shared code, or central build and dependency files do not require a provider version update.
 
-Release tags must exactly match project versions. A single-provider tag must match that provider's version, while a component-wide `mevd/v<version>` tag requires every MEVD provider to have that version.
+Release tags must exactly match project versions. A single-provider tag must match that provider's version, while a component-wide `<component>/v<version>` tag requires every provider in that component to have that version.
 
 ## Release flow
 
-Pushing a matching tag triggers the component workflow, which runs the full test suite, packs NuGet packages, then hands them to `sign-and-publish.yml` for signing (Azure Key Vault) → GitHub Release → NuGet.org (gated by manual approval on the `nuget-release-gate` environment).
+Pushing a matching tag triggers the component workflow, which runs the full test suite, packs NuGet packages, then hands them to `release.yml` for signing (Azure Key Vault) → GitHub Release → NuGet.org (gated by manual approval on the `nuget-release-gate` environment).
 
 ## Adding a new component
 
 Create a `<component>.yml` workflow that:
 1. Triggers on its own tag prefix (e.g. `mycomp/v*`)
 2. Handles all component-specific build and test logic
-3. On tag pushes, calls `package.yml` then `sign-and-publish.yml`
+3. On tag pushes, packs NuGet packages and calls `release.yml`

.github/skills/publish-providers/SKILL.md

@@ -1,11 +1,17 @@
 ---
 name: publish-providers
-description: Publish one or more MEVD providers when the user explicitly requests a release, for example "publish Chroma and PgVector". Never publish when merely discussing or editing this skill.
+description: Publish one or more MEVD or DocumentExtraction providers when the user explicitly requests a release, for example "publish Chroma and PgVector" or "publish PdfPig and Mistral". Never publish when merely discussing or editing this skill.
 ---
 
 1. Require explicitly named providers; never infer additional releases. Verify `origin` points to `CommunityToolkit/AI`.
 2. Fetch `origin main --tags` and pin the release commit to `origin/main`. Do not switch branches, commit changes, or bump versions.
-3. At that commit, resolve each provider to its exact directory name under `MEVD/src`. Read its literal `<Version>` from its `.csproj`; form `mevd/<Provider>/v<Version>`. Stop on unknown providers or ambiguous versions.
+3. At that commit, resolve each provider to its exact directory name using the component table below. Read its literal `<Version>` from its `.csproj`; form `<tag-prefix>/<Provider>/v<Version>`. Stop on unknown or ambiguous providers or versions; require a component name if a provider exists in both components.
+
+   | Component | Provider directories | Tag prefix | Workflow |
+   | --- | --- | --- | --- |
+   | MEVD | `MEVD/src` | `mevd` | `mevd.yml` |
+   | DocumentExtraction | `DocumentExtraction/src` | `document-extraction` | `document-extraction.yml` |
+
 4. Preflight every requested tag locally and remotely before making changes. Stop on any tag pointing to another commit; skip remote tags already targeting the release commit. Reuse matching local tags.
 5. For each remaining tag, create it at the pinned commit if missing, then run `git push origin refs/tags/<tag>`. Push tags individually so each triggers GitHub Actions. Never force-push, push branches, or use `--tags`. Stop on failure and report any tags already pushed.
 6. Verify the remote tag targets and report the provider/version, tag, and workflow link. Distinguish a triggered release from a published package: NuGet publishing still requires approval of `nuget-release-gate`.

.github/workflows/document-extraction.yml

@@ -3,18 +3,18 @@ name: Document extraction
 on:
   push:
     branches: [main]
+    tags:
+      - "document-extraction/v*"
+      - "document-extraction/*/v*"
     paths:
       - 'DocumentExtraction/**'
       - 'Directory.Build.props'
       - 'Directory.Build.targets'
       - 'Directory.Packages.props'
       - 'CommunityToolkit.AI.slnx'
       - 'global.json'
-      - 'eng/Test-DocumentExtractionPackages.ps1'
-      - 'eng/DocumentExtraction.PackageSmoke/**'
       - 'eng/LegacySupport.props'
       - 'Shared/**'
-      - 'nuget.config'
       - '.github/workflows/document-extraction.yml'
   pull_request:
     branches: [main, 'copilot/*']
@@ -25,13 +25,9 @@ on:
       - 'Directory.Packages.props'
       - 'CommunityToolkit.AI.slnx'
       - 'global.json'
-      - 'eng/Test-DocumentExtractionPackages.ps1'
-      - 'eng/DocumentExtraction.PackageSmoke/**'
       - 'eng/LegacySupport.props'
       - 'Shared/**'
-      - 'nuget.config'
       - '.github/workflows/document-extraction.yml'
-  workflow_dispatch:
 
 permissions:
   contents: read
@@ -41,8 +37,8 @@ env:
   DOTNET_CLI_TELEMETRY_OPTOUT: true
 
 jobs:
-  build-test-pack:
-    name: Build, test and pack (${{ matrix.os }})
+  build-and-test:
+    name: Build and test (${{ matrix.os }})
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -59,17 +55,67 @@ jobs:
         run: dotnet build DocumentExtraction/DocumentExtraction.slnf --no-restore -c Release
       - name: Test
         run: dotnet test DocumentExtraction/DocumentExtraction.slnf --no-build --no-restore -c Release
-      - name: Restore portable test dependency closure
-        run: dotnet restore DocumentExtraction/DocumentExtraction.slnf --configfile DocumentExtraction/nuget.config -p:DocumentExtractionTestAsset=netstandard2.0
-      - name: Test using actual netstandard assets
-        run: dotnet test DocumentExtraction/DocumentExtraction.slnf --no-restore -c Release -p:DocumentExtractionTestAsset=netstandard2.0
-      - name: Pack preview artifacts
-        run: dotnet pack DocumentExtraction/DocumentExtraction.slnf --no-build --no-restore -c Release --output artifacts/document-extraction
-      - name: Verify packaged assets and experimental opt-in
-        shell: pwsh
-        run: ./eng/Test-DocumentExtractionPackages.ps1 -PackageDirectory artifacts/document-extraction -OutputDirectory "${{ runner.temp }}/document-extraction-package-checks"
-      - uses: actions/upload-artifact@v6
+
+  package:
+    name: Package
+    if: startsWith(github.ref, 'refs/tags/')
+    needs: build-and-test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-dotnet@v5
         with:
-          name: document-extraction-preview-packages-${{ matrix.os }}
-          path: artifacts/document-extraction/*.nupkg
-          if-no-files-found: error
+          global-json-file: global.json
+
+      - name: Parse release tag
+        id: tag
+        run: |
+          rest="${GITHUB_REF_NAME#document-extraction/}"
+          if [[ "$rest" == v* ]]; then
+            version="${rest#v}"
+            echo "pack_target=DocumentExtraction/DocumentExtraction.slnf" >> "$GITHUB_OUTPUT"
+          else
+            provider="${rest%%/v*}"
+            version="${rest#*/v}"
+            echo "pack_target=DocumentExtraction/src/$provider/$provider.csproj" >> "$GITHUB_OUTPUT"
+          fi
+          echo "version=$version" >> "$GITHUB_OUTPUT"
+
+      - name: Restore
+        run: dotnet restore DocumentExtraction/DocumentExtraction.slnf --configfile DocumentExtraction/nuget.config
+
+      - name: Pack NuGet packages
+        env:
+          PACK_TARGET: ${{ steps.tag.outputs.pack_target }}
+          PACKAGE_VERSION: ${{ steps.tag.outputs.version }}
+        run: >
+          dotnet pack "$PACK_TARGET"
+          --no-restore
+          -c Release
+          -o ./nuget
+          -p:PackageVersion="$PACKAGE_VERSION"
+          -p:CI=true
+
+      - name: Upload NuGet packages
+        uses: actions/upload-artifact@v6
+        with:
+          name: nuget-packages
+          path: ./nuget
+          if-no-files-found: error
+
+      - name: Upload signing file list
+        uses: actions/upload-artifact@v6
+        with:
+          name: nuget-list
+          path: ${{ github.workspace }}/.github/workflows/SignClientFileList.txt
+          if-no-files-found: error
+
+  release:
+    name: Release
+    if: startsWith(github.ref, 'refs/tags/')
+    needs: package
+    permissions:
+      contents: write
+      id-token: write
+    uses: ./.github/workflows/release.yml
+    secrets: inherit

DocumentExtraction/README.md

@@ -39,9 +39,7 @@ dependencies come from the public `dotnet-libraries` feed, not NuGet.org.
 
 The .NET Standard 2.0 asset makes the managed provider APIs available to compatible
 applications; it is not a new .NET Framework 4.6.2 target or a certification of
-every legacy host, backend SDK or native dependency. Tests run on .NET 10,
-including the full offline suites explicitly loading the netstandard provider
-assemblies. A separate package consumer verifies the actual packed assets.
+every legacy host, backend SDK or native dependency. Tests run on .NET 10.
 Use a supported application runtime and check each backend's platform policy.
 .NET 8 is not a supported runtime and no dedicated `net8.0` asset is shipped.
 An existing .NET 8 app may resolve the portable asset, but that is technical
@@ -233,23 +231,21 @@ or retries unsupported structured results through another inference request.
 
 ## Building from source
 
-From the repository root with PowerShell 7 and the SDK from `global.json`:
+From the repository root with the SDK from `global.json`:
 
 ```powershell
 dotnet restore DocumentExtraction/DocumentExtraction.slnf --configfile DocumentExtraction/nuget.config
 dotnet build DocumentExtraction/DocumentExtraction.slnf -c Release --no-restore
 dotnet test DocumentExtraction/DocumentExtraction.slnf -c Release --no-build --no-restore
-dotnet restore DocumentExtraction/DocumentExtraction.slnf --configfile DocumentExtraction/nuget.config -p:DocumentExtractionTestAsset=netstandard2.0
-dotnet test DocumentExtraction/DocumentExtraction.slnf -c Release --no-restore -p:DocumentExtractionTestAsset=netstandard2.0
-dotnet pack DocumentExtraction/DocumentExtraction.slnf -c Release --no-build --no-restore --output artifacts/document-extraction
-./eng/Test-DocumentExtractionPackages.ps1 -PackageDirectory artifacts/document-extraction -OutputDirectory artifacts/document-extraction-package-checks
 ```
 
-The second test run loads the actual `netstandard2.0` provider assemblies; restore
-its profile before running it. The package check requires a new output directory
-and restores into its own package cache. It verifies consumer diagnostics without
-opt-in, then compiles and executes opted-in consumers against the packed modern
-and `netstandard2.0` assets.
+## Publishing
+
+Like MEVD, releases are triggered by tags: `document-extraction/<Provider>/v<version>`
+for one provider or `document-extraction/v<version>` for all providers. Tag versions
+must match the corresponding project versions. The workflow builds and tests,
+packs the selected providers, then uses the shared signing and publishing workflow.
+Publishing to NuGet.org requires approval of `nuget-release-gate`.
 
 ## Work with the result
 

DocumentExtraction/test/Directory.Build.props

@@ -5,19 +5,11 @@
     <TargetFrameworks>net10.0</TargetFrameworks>
     <OutputType>Exe</OutputType>
     <NoWarn>$(NoWarn);CS1591;xUnit1051;EXTEXP0019;MEDE0001;CTDE0001</NoWarn>
-    <DefineConstants Condition="'$(DocumentExtractionTestAsset)' == 'netstandard2.0'">$(DefineConstants);STANDARD_PROVIDER_ASSET</DefineConstants>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="Microsoft.NET.Test.Sdk" />
     <PackageReference Include="xunit.v3" />
     <PackageReference Include="xunit.runner.visualstudio" PrivateAssets="all" />
     <Using Include="Xunit" />
-    <Compile Include="$(MSBuildThisFileDirectory)ProviderAssetTests.cs" Link="ProviderAssetTests.cs" />
   </ItemGroup>
-  <ItemDefinitionGroup Condition="'$(DocumentExtractionTestAsset)' == 'netstandard2.0'">
-    <ProjectReference>
-      <SetTargetFramework>TargetFramework=netstandard2.0</SetTargetFramework>
-      <SkipGetTargetFrameworkProperties>true</SkipGetTargetFrameworkProperties>
-    </ProjectReference>
-  </ItemDefinitionGroup>
 </Project>

DocumentExtraction/test/Mistral.UnitTests/MistralDocumentExtractionClientTests.cs

@@ -588,11 +588,7 @@ public async Task ExtractAsync_Five503Responses_StopsAndDisposesAllAttempts()
         }
 
         HttpRequestException error = await Assert.ThrowsAsync<HttpRequestException>(() => Extract(session, RequestedOptions()));
-#if STANDARD_PROVIDER_ASSET
-        Assert.Null(error.StatusCode);
-#else
         Assert.Equal(HttpStatusCode.ServiceUnavailable, error.StatusCode);
-#endif
         Assert.Contains("503", error.Message);
         Assert.Equal([3d, 6d, 9d, 12d], session.Delays.Select(d => d.TotalSeconds));
         Assert.All(session.Handler.Requests, AssertRequest);
@@ -610,11 +606,7 @@ public async Task ExtractAsync_Non503Failure_DoesNotRetry(HttpStatusCode status)
     {
         using var session = new OfflineSession(Encoding.UTF8.GetBytes("failure"), status);
         HttpRequestException error = await Assert.ThrowsAsync<HttpRequestException>(() => Extract(session));
-#if STANDARD_PROVIDER_ASSET
-        Assert.Null(error.StatusCode);
-#else
         Assert.Equal(status, error.StatusCode);
-#endif
         Assert.Contains(((int)status).ToString(System.Globalization.CultureInfo.InvariantCulture), error.Message);
         Assert.Empty(session.Delays);
         await session.AssertCleanAndBorrowedAsync(1, 1);

DocumentExtraction/test/Mistral.UnitTests/MistralDualHostTests.cs

@@ -144,11 +144,7 @@ public async Task ExtractAsync_AuthenticationFailure_DoesNotTryAnotherHostOrCred
         using var session = new OfflineSession(status: HttpStatusCode.Unauthorized,
             ocrEndpoint: endpoint, useApiKey: useApiKey);
         HttpRequestException error = await Assert.ThrowsAsync<HttpRequestException>(() => Extract(session));
-#if STANDARD_PROVIDER_ASSET
-        Assert.Null(error.StatusCode);
-#else
         Assert.Equal(HttpStatusCode.Unauthorized, error.StatusCode);
-#endif
         Assert.Contains("401", error.Message);
         Assert.Equal(endpoint, Assert.Single(session.Handler.Requests).Message.RequestUri);
         Assert.Empty(session.Delays);