Add Unit and Functional test CI pipeline - #4506
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new Azure DevOps pipeline (sqlclient-ci-unit) that runs the SqlClient Unit and Functional test suites in Package reference mode, consuming the exact NuGet artifacts produced by the triggering sqlclient-ci-package run. This follows the same downstream-consume-upstream-artifacts pattern introduced by the stacked PR (#4499) and extends it to broad Unit/Functional coverage across OS/TFM/SNI combinations.
Changes:
- Introduces a package-triggered pipeline definition that runs only on completion of
sqlclient-ci-package(no PR/CI triggers). - Defines a 3-stage OS matrix (Windows/Linux/macOS) with the intended TFM and SNI coverage.
- Adds a reusable job template that aligns source to the upstream commit, downloads/stages driver packages, runs Unit + Functional suites, and publishes results/artifacts.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| eng/pipelines/ci/unit/sqlclient-ci-unit-stages.yml | Defines the Windows/Linux/macOS runtime/SNI job matrix for the unit+functional test pipeline. |
| eng/pipelines/ci/unit/sqlclient-ci-unit-pipeline.yml | New pipeline entrypoint that is triggered by sqlclient-ci-package completion and invokes the stage matrix. |
| eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml | Job template to align to upstream commit, consume produced packages, run Unit/Functional tests, and publish results. |
paulmedynski
added
the
Area\Engineering
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:100
- The comment says this step downloads the exact driver packages, but
sqlServerVersionOverride: 1.0.0means$(sqlServerPackageVersion)will be pinned instead of using the version produced by the triggeringsqlclient-ci-packagerun. That makes the current comment misleading and obscures why the override exists.
Update the comment to reflect the pin (or remove the override if the intent is to validate the exact SqlServer package from the upstream run).
# Download the exact driver packages produced by the triggering pipeline.
- template: /eng/pipelines/common/steps/download-driver-packages-step.yml@self
parameters:
sqlServerVersionOverride: 1.0.0
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:120
update-config-file-step.ymlis being used only to setUseManagedSNIOnWindows, but because most parameters are omitted (so they default to empty/false), this step also overwrites the non-empty defaults inconfig.default.jsonc(e.g., TCP/NP connection strings andSupportsIntegratedSecurity=true). That unintentionally changes the baseline test config for this pipeline.
Pass through the defaults from config.default.jsonc so this step doesn’t clobber them while toggling SNI.
# Configure the test suite's Windows SNI implementation.
- template: /eng/pipelines/common/templates/steps/update-config-file-step.yml@self
parameters:
debug: ${{ parameters.debug }}
saPassword: ''
UseManagedSNIOnWindows: ${{ parameters.useManagedSNI }}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (1)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:8
- The header comment claims this job runs against the exact packages from the triggering
sqlclient-ci-packagerun, but the job forcessqlServerPackageVersionviasqlServerVersionOverride: 1.0.0, soMicrosoft.SqlServer.Serveris not necessarily taken from the upstream artifact (NU1605 workaround). Please adjust the comment to reflect this exception to avoid misleading future maintainers.
# Builds and runs the SqlClient Unit and Functional test suites in Package reference mode against
# the exact packages produced by the triggering sqlclient-ci-package run.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:107
- This job sets sqlServerVersionOverride: 1.0.0 while the adjacent comment says it downloads the “exact driver packages”. Since the override intentionally diverges from the artifact-resolved version (to avoid NU1605 downgrade warnings), add an in-file explanation (similar to managed-instance/stress jobs) so the behavior is clear.
# Download the exact driver packages produced by the triggering pipeline.
- template: /eng/pipelines/common/steps/download-driver-packages-step.yml@self
parameters:
sqlServerVersionOverride: 1.0.0
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:7
- The header comment says this job runs against the “exact packages produced” by the upstream run, but this job pins Microsoft.SqlServer.Server via sqlServerVersionOverride (so at least that package version is not taken from the upstream artifact). Update the comment to reflect the exception so future readers aren’t misled.
This issue also appears on line 104 of the same file.
# Builds and runs the SqlClient Unit and Functional test suites in Package reference mode against
# the exact packages produced by the triggering sqlclient-ci-package run.
| -p:PackageVersionSqlServer=$(sqlServerPackageVersion) | ||
| -p:TestResultsFolderPath=TestResults | ||
|
|
||
| - task: PublishTestResults@2 |
There was a problem hiding this comment.
Using the common template for publishing test results.
|
|
||
| steps: | ||
|
|
||
| # Align the checkout with the commit that produced the upstream packages. |
There was a problem hiding this comment.
I'm starting to see some common boilerplate in these jobs, so we will keep an eye out for some common steps to extract, but not yet.
| ${{ else }}: | ||
| testResultsFiles: | | ||
| TestResults/*.trx | ||
| TestResults/**/*.coverage |
There was a problem hiding this comment.
This was causing PublishTestResults@2 to silently fail when it encountered non-TRX files. Code coverage publishing is an entirely different subject we will tackle later.
| - Windows | ||
|
|
||
| # Optional test run title. When empty, preserve the historical OS-based title. | ||
| - name: testRunTitle |
There was a problem hiding this comment.
The old way would publish multiple test runs with the same titles like "Windows Tests". Now, the new CI pipelines use the job name (like "Win : Native SNI : net9.0" and "macos : net10.0") which is much more descriptive.
paulmedynski
assigned
mdaigle and
apoorvdeshmukh and unassigned
benrr101 and
cheenamalhotra
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev/paul/sqlclient-ci-kerberos #4506 +/- ##
==================================================================
+ Coverage 62.72% 62.81% +0.08%
==================================================================
Files 283 283
Lines 66979 67041 +62
==================================================================
+ Hits 42015 42114 +99
+ Misses 24964 24927 -37
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
3e381ab to
afe29fe
Compare
Description
Adds a package-triggered
sqlclient-ci-unitpipeline for the SqlClient Unit and Functional test suites.I decided not to bother adding separate pipelines for unit, functional, and simulated tests right now. We can decide if such a distinction is worth it later.
net462on Windows with native SNI.net8.0,net9.0, andnet10.0on Windows with native and managed SNI.net8.0,net9.0, andnet10.0on Linux and macOS with managed SNI.ADO-Win25,ADO-UB24, and the Microsoft-hostedmacos-latestimage.sqlclient-ci-packagerun.build.projdefault filter.Testing