Skip to content

Bundle evaluations and dataset extensions in the Foundry meta-package - #10085

Open
Mohamed Hessien (m7md7sien) wants to merge 2 commits into
Azure:mainfrom
m7md7sien:feat/foundry-pack-eval-dataset
Open

Mohamed Hessien (m7md7sien) wants to merge 2 commits into
Azure:mainfrom
m7md7sien:feat/foundry-pack-eval-dataset

Conversation

@m7md7sien

@m7md7sien Mohamed Hessien (m7md7sien) commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #10086

Adds azure.ai.evaluations and azure.ai.dataset to the microsoft.foundry meta-package so a single azd extension install microsoft.foundry brings the evaluation authoring and dataset versioning commands along with the rest of the Foundry surface.

Both extensions are already on main at 1.0.0-beta.1 (#9500 and #9499), and both ship their own release pipeline (eng/pipelines/release-ext-azure-ai-evaluations.yml, eng/pipelines/release-ext-azure-ai-dataset.yml).

Changes

File Change
extension.yaml Two new dependencies, inserted in the existing alphabetical order; package version remains 1.0.0-beta.3

Package version bumps and changelog updates are deferred to a separate consolidated release PR. version.txt and the historical CHANGELOG.md are unchanged from this PR's base.

dependencies:
  - id: azure.ai.agents
    version: "~1.0.0-beta.16"
  - id: azure.ai.connections
    version: "~1.0.0-beta.7"
  - id: azure.ai.dataset          # new
    version: "~1.0.0-beta.1"
  - id: azure.ai.evaluations      # new
    version: "~1.0.0-beta.1"
  - id: azure.ai.inspector
    version: "~1.0.0-beta.7"
  ...

The pins use the same ~<version> style as every existing dependency, so the pack picks up patch releases of both extensions without another manifest change.

⚠️ Release sequencing

Feature merge remains on hold. This metadata-only cleanup does not lift that hold. The PR only changes bundling dependencies in extension.yaml, not registry.json.

But the meta-package must not be released until azure.ai.evaluations and azure.ai.dataset are published to the registry. Neither is in registry.json yet (#10099 is still open).

Correction. An earlier revision of this description claimed ext-registry-ci would reject the pack's registry entry for the unpublished pins. That is wrong, and thanks to the Copilot review for catching it. Per validateDependencies, a dependency whose id is absent from the registry is reported as a warning, and TestRegistryFileIsValid fails only on errors — so registry validation would pass. An error is raised only when the id is present but the version constraint matches no published version.

So the gate is not CI, it is install-time resolution: azd resolves an extension's dependencies from the parent's own source when installing, so releasing the pack now would produce a microsoft.foundry that fails to resolve two of its dependencies for anyone installing it.

Correct order:

  1. Merge this PR only after the feature merge hold is lifted.
  2. Release azure.ai.evaluations and azure.ai.dataset (their release pipelines → registry update PRs).
  3. Prepare package version bumps and changelog updates in a separate consolidated release PR, then release microsoft.foundry only after both dependencies are available in the registry.

Worth noting for step 3: because the pins are ~1.0.0-beta.1 and the published versions will be exactly 1.0.0-beta.1, the constraint resolves cleanly. If the pack were ever released with a pin ahead of what is published (say ~1.0.0-beta.2), that would be a hard registry validation error rather than a warning.

Validation

  • Manifest parses; all nine dependency ids resolve to extension directories under cli/azd/extensions/, and every pinned version matches the version that extension declares.
  • No Go code touched.

Add `azure.ai.evaluations` and `azure.ai.dataset` to the `microsoft.foundry`
dependency list so installing the meta-package brings the evaluation
authoring and dataset versioning commands along with the rest of the
Foundry surface. Both extensions landed on main at 1.0.0-beta.1 in Azure#9500
and Azure#9499.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
3 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Correct the release-sequencing note to distinguish registry validation from install-time resolution.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates the microsoft.foundry meta-package to bundle dataset and evaluation extensions.

Changes:

  • Adds azure.ai.dataset and azure.ai.evaluations dependencies.
  • Bumps the package to 1.0.0-beta.4.
  • Documents the bundled extensions in the changelog.
File summaries
File Description
cli/azd/extensions/microsoft.foundry/extension.yaml Adds dataset and evaluations dependencies.
cli/azd/extensions/microsoft.foundry/version.txt Updates the package version.
cli/azd/extensions/microsoft.foundry/CHANGELOG.md Records the new release.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +15 to +18
- id: azure.ai.dataset
version: "~1.0.0-beta.1"
- id: azure.ai.evaluations
version: "~1.0.0-beta.1"
@@ -1,5 +1,11 @@
# Release History

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking comment untill the release of azure.ai.evaluations and azure.ai.dataset.
Feel free to resolve the comment when the two packages are released.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 23, 2026 23:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The dependency IDs, versions, ordering, and compatibility requirements are correct.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@m7md7sien

Copy link
Copy Markdown
Contributor Author

/check-enforcer evaluate

@microsoft-github-policy-service microsoft-github-policy-service Bot added the no-recent-activity identity issues with no activity label Oct 1, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Hi @Mohamed Hessien (@m7md7sien). Thank you for your interest in helping to improve the Azure Developer CLI experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment. Otherwise, we'll close this out in 7 days.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-recent-activity identity issues with no activity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bundle the evaluations and dataset extensions in the Foundry meta-package

2 participants