Modernize Python tooling (uv, pyproject.toml, src layout, semantic release) by salman2013 · Pull Request #246 · openedx/code-annotations

salman2013 · 2026-08-06T11:18:56Z

Summary

Phase 1 — Migrate package metadata to pyproject.toml (PEP 621/639): setuptools-scm dynamic versioning, importlib.metadata for docs version, coverage + isort config, semantic-release changelog config. Delete setup.py, setup.cfg, .coveragerc.
Phase 2 — Switch from pip-tools to uv: dependency-groups (test-base/test/django42/quality/doc/ci/dev), uv.lock, rewrite tox.ini with tox-uv>=1/uv-venv-lock-runner, rewrite Makefile with direct uv run --group commands, update CI to astral-sh/setup-uv with workflow_call trigger. Delete requirements/.
Phase 3 — Add python-semantic-release workflow: release.yml runs CI, cuts releases via PSR v10.5.3, publishes to PyPI via OIDC trusted publisher. Delete pypi-publish.yml.
Phase 4 — Migrate to src/ layout: git mv code_annotations/ src/code_annotations/, update all filesystem paths in tox.ini, docs/conf.py, pyproject.toml, Makefile.

Test plan

All CI matrix jobs pass (quality, docs, django42, django52)
Configure OIDC trusted publisher on PyPI for release.yml before merging

🤖 Generated with Claude Code

- Add pyproject.toml with PEP 621 metadata, setuptools-scm dynamic versioning, coverage config, isort config, and semantic-release changelog config - Add .. changelog-insertion-marker to CHANGELOG.rst for PSR - Remove __version__ from __init__.py (unused at runtime) - Update docs/conf.py to use importlib.metadata for version - Delete setup.py, setup.cfg, and .coveragerc Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Add dependency-groups (test-base/test/django42/quality/doc/ci/dev) - Add uv.lock (94 packages resolved) - Add constraint-dependencies via edx_lint write_uv_constraints - Rewrite tox.ini to use tox-uv>=1 and uv-venv-lock-runner - Rewrite Makefile to use uv run tox and uv sync - Rewrite ci.yml: setup-uv, uv sync --group ci, uv run tox, workflow_call trigger, matrix job name - Delete requirements/ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Add release.yml: runs CI via workflow_call, PSR v10.5.3, uploads dist artifact, publishes to PyPI via OIDC trusted publisher - Delete pypi-publish.yml (replaced by release.yml) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Move code_annotations/ to src/code_annotations/ - Add where = ["src"] to [tool.setuptools.packages.find] - Update tox.ini quality commands to src/code_annotations paths - Update docs/conf.py sphinx-apidoc path to src/code_annotations - Update coverage omit path in pyproject.toml - Rewrite Makefile to use direct uv run --group commands (no tox) - Regenerate uv.lock Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

openedx-webhooks · 2026-08-06T11:19:03Z

Thanks for the pull request, @salman2013!

This repository is currently maintained by @bmtcril.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

requirements/doc.txt was removed in the uv migration, so RTD builds were failing. Switch to installing via uv export and pip install. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

irfanuddinahmad

Verified against actual current file content, live CI, and by actually running commands locally against this branch -- not just reading the diff. All CI checks are green, which is expected since the bugs below aren't caught by CI.

Same changelog: "false" bug as edx-lint#563 and api-doc-tools#409 -- identical release.yml pattern across all three, looks template-derived rather than repo-specific.

What's done correctly here and worth not re-litigating: license/SPDX handling (confirmed via an actual local build that it doesn't raise InvalidConfigError), the Django version-matrix uses the correct [tool.uv].conflicts pattern, isort is both declared and actually wired into the quality targets (unlike a sibling repo), no ruff scope-creep, and the OIDC publish + pre-flight tag/PyPI version check both check out.

irfanuddinahmad · 2026-08-06T13:40:55Z

+        with:
+          git_committer_name: "github-actions"
+          git_committer_email: "github-actions@github.com"
+          changelog: "false"


changelog: "false" overrides the [tool.semantic_release.changelog] config this same PR carefully builds in pyproject.toml (mode = "update", matching insertion_flag) -- the action-level flag wins, silently disabling changelog generation entirely despite the config looking correct. This is the exact incident class that already caused real CHANGELOG.rst content loss in a sibling repo in this effort.

Fixed in 4b04932 — removed changelog: "false" so the [tool.semantic_release.changelog] config in pyproject.toml now applies (the .. changelog-insertion-marker is present in CHANGELOG.rst).

irfanuddinahmad · 2026-08-06T13:40:55Z

+    {include-group = "ci"},
+]
+
+[tool.uv]


Missing default-groups = []. I reproduced this directly against the PR branch:

$ uv run --group django42 python -Wd -m pytest --collect-only -q error: Groups `dev` (enabled by default) and `django42` are incompatible with the conflicts: {`code-annotations:dev`, `code-annotations:django42`}

This is make test-all's first line (Makefile), so that target is broken today for anyone running it locally. CI stays green only because tox-uv happens to pass --no-default-groups internally when driving tests through tox -- it masks the gap rather than avoiding it. uv.lock's own [manifest].conflicts list independently corroborates this: it enumerates django42 vs. test, doc, quality, and dev.

Fixed in 4b04932 — added default-groups = [] under [tool.uv]. Every uv invocation in the Makefile and CI already names its group explicitly, so nothing depended on the default dev group. Verified uv run --group django42 python -Wd -m pytest now runs (94 passed).

irfanuddinahmad · 2026-08-06T13:40:55Z

  pull_request:
    branches:
      - "**"
+  workflow_call:


This new workflow_call: trigger, combined with the pre-existing push: [master] trigger a few lines up (unchanged by this PR), means every push to master now runs this full matrix twice concurrently: once via the direct push trigger, once via release.yml calling this workflow as its run_tests job. Recommend dropping the push: [master] trigger now that release.yml owns that path.

Fixed in 4b04932 — dropped the push: [master] trigger; release.yml now owns the master path via workflow_call, and pull_request still triggers CI directly.

irfanuddinahmad · 2026-08-06T13:42:14Z

One more, on a file this PR doesn't touch so I can't anchor it inline: MANIFEST.in is untouched by this PR and still has:

include requirements/base.in
recursive-include code_annotations *.tpl *.html *.png *.gif *js *.css *jpg *jpeg *svg *py *.yaml *.yml

Line 1 references a file this PR deletes (requirements/base.in). Line 2 still hardcodes the pre-src/-layout path (code_annotations/ instead of src/code_annotations/). I built both a real sdist and wheel from this branch to check actual impact -- setuptools-scm's git-based file-finder pulls the right files into the sdist regardless of the stale line, and the wheel is governed by [tool.setuptools.package-data] (which was correctly updated), so this isn't a build breaker. Still worth fixing for correctness -- same "stale MANIFEST.in reference" pattern confirmed in a sibling repo (ccx-keys#190).

…tooling # Conflicts: # requirements/base.txt # requirements/ci.txt # requirements/common_constraints.txt # requirements/dev.txt # requirements/django.txt # requirements/doc.txt # requirements/pip-tools.txt # requirements/pip.txt # requirements/quality.txt # requirements/test.txt

…tooling # Conflicts: # .github/workflows/ci.yml # .github/workflows/pypi-publish.yml # requirements/ci.txt # requirements/common_constraints.txt # requirements/dev.txt # requirements/django.txt # requirements/doc.txt # requirements/pip.txt # requirements/quality.txt # requirements/test.txt

- release.yml: drop `changelog: "false"` so the semantic-release changelog config in pyproject.toml takes effect - pyproject.toml: set `default-groups = []` so `uv run --group django42` no longer conflicts with the default `dev` group (fixes `make test-all`) - ci.yml: drop the push-to-master trigger; release.yml already runs CI via workflow_call, so master pushes ran the matrix twice - MANIFEST.in: remove stale requirements/base.in reference and point recursive-include at src/code_annotations Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

salman2013 · 2026-11-04T09:23:19Z

@irfanuddinahmad re: MANIFEST.in — fixed in 4b04932: removed the stale include requirements/base.in line and updated recursive-include to src/code_annotations. Verified uv build produces both sdist and wheel cleanly.

irfanuddinahmad

Sorry, my August comments here are out of date: this PR missed the round edx-lint#563 and api-doc-tools#409 got in October, so some of today's fixes need to be reversed. This replaces my earlier changelog comment.

Please put back changelog: "false" and remove the [tool.semantic_release.changelog] and [tool.semantic_release.changelog.default_templates] tables (and the CHANGELOG insertion marker).
Please re-sync release.yml with sample-plugin's current file, the same as edx-lint#563: PSR v10.7.0, gh release create with the dists attached, a distribution-artifacts artifact with path: dist and id: artifact-download, every action SHA-pinned (pypi-publish too, no @release/v1), git_committer_email: "actions@users.noreply.github.com", and the comment blocks.
Please reseed uv.lock from master's requirements/*.txt pins instead of a fresh resolve: run edx_lint write_uv_constraints pyproject.toml, add master's pins as name==ver to constraint-dependencies, uv lock, restore pyproject, uv lock again, then diff uv export --only-group <g> against each requirements/<g>.txt. The current lock is behind master, e.g. edx-lint 6.2.0 → 5.2.0, python-slugify 9.1.3 → 8.0.4, pylint 4.1.3 → 4.0.6. Please also drop the edx-lint<6 and setuptools<81 caps from quality.
ci.yml: please use uv sync --locked --group ci and uv run --locked tox ..., and add permissions: contents: read to the job.
Makefile: please drop the uv run --group ... prefix from every target and call the tools bare (requirements keeps uv sync); upgrade should be uv run --with edx-lint edx_lint write_uv_constraints pyproject.toml then uv lock --upgrade. Please also keep quality, docs and test-all calling tox as on master.
Please keep __version__ in src/code_annotations/__init__.py: from importlib.metadata import version and __version__ = version("code-annotations").
Please make dependencies match PyPI 3.0.0: plain click, pyyaml, Jinja2, stevedore, python-slugify.
Release settings, inline.

nits: rerun edx_lint write_uv_constraints (the social-auth pins are missing), and point the README CI badge at actions/workflows/release.yml/badge.svg?branch=master, since ci.yml no longer runs on master.

irfanuddinahmad · 2026-11-04T13:38:32Z

+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+]
+dependencies = [


Please restore PyPI 3.0.0's requirements: no version floors.

irfanuddinahmad · 2026-11-04T13:38:32Z

+[tool.setuptools_scm]
+version_scheme = "only-version"
+local_scheme = "no-local-version"
+fallback_version = "0.0.0.dev0"


Please use fallback_version = "0.0.0".

irfanuddinahmad · 2026-11-04T13:38:32Z

+    "contrib/config/*.yaml",
+]
+
+[tool.semantic_release]


Please add tag_format = "v{version}" here.

irfanuddinahmad · 2026-11-04T13:38:32Z

+major_on_zero = false
+allow_zero_version = true
+
+[tool.semantic_release.changelog]


Please remove this and the default_templates table; release.yml should run PSR with changelog: "false".

irfanuddinahmad · 2026-11-04T13:38:32Z

+django42 = [{include-group = "test-base"}, "Django>=4.2,<4.3"]
+quality = [
+    {include-group = "test"},
+    "edx-lint<6",


Please drop <6; master is on edx-lint 6.2.0.

irfanuddinahmad · 2026-11-04T13:38:32Z

+    "isort",
+    "pycodestyle",
+    "pydocstyle",
+    "setuptools<81",


Please drop this cap; it isn't on master.

irfanuddinahmad · 2026-11-04T13:38:32Z

-      - name: Install Dependencies
-        run: pip install -r requirements/ci.txt
+      - name: Install CI dependencies
+        run: uv sync --group ci


uv sync --locked --group ci

irfanuddinahmad · 2026-11-04T13:38:32Z

-        env:
-          TOXENV: ${{ matrix.toxenv }}
-        run: tox
+        run: uv run tox -e ${{ matrix.toxenv }}


uv run --locked tox -e ${{ matrix.toxenv }}

irfanuddinahmad · 2026-11-04T13:38:32Z

@@ -0,0 +1,55 @@
+name: Semantic Release


Please replace this with sample-plugin's current release.yml (see the review summary).

irfanuddinahmad · 2026-11-04T13:38:32Z

@@ -1,5 +1,3 @@
 """
 Extensible tools for parsing annotations in codebases.
 """


Please add from importlib.metadata import version and __version__ = version("code-annotations").

- release.yml: re-sync with sample-plugin (PSR v10.7.0, changelog off, gh release create with dists attached, SHA-pinned actions) - ci.yml: use --locked for sync/run and add contents: read permission - Makefile: call tools bare; quality/docs/test-all go through tox - pyproject: unpinned runtime deps matching 3.0.0, fallback_version 0.0.0, tag_format v{version}, drop PSR changelog tables and the edx-lint/setuptools caps, regenerate uv constraints - uv.lock: reseed from master's requirements/*.txt pins - restore __version__ via importlib.metadata - drop the CHANGELOG insertion marker Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

irfanuddinahmad

Thanks, all addressed.

nit: dev, doc and quality lock Django 5.2.19, master pins 5.2.18; uv lock --upgrade-package django==5.2.18 brings it back.

salman2013 and others added 4 commits August 6, 2026 15:45

build: add python-semantic-release workflow

94dd984

- Add release.yml: runs CI via workflow_call, PSR v10.5.3, uploads dist artifact, publishes to PyPI via OIDC trusted publisher - Delete pypi-publish.yml (replaced by release.yml) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Aug 6, 2026

openedx-webhooks added this to Contributions Aug 6, 2026

github-project-automation Bot moved this to Needs Triage in Contributions Aug 6, 2026

salman2013 changed the title ~~build: modernize Python tooling (uv, pyproject.toml, src layout, semantic release)~~ Modernize Python tooling (uv, pyproject.toml, src layout, semantic release) Aug 6, 2026

fix: update hardcoded template paths in test configs for src layout

7ed923e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

salman2013 mentioned this pull request Aug 6, 2026

Modernize Python repo | Developer & CI Tools openedx/public-engineering#519

Open

3 of 8 tasks

build: fix RTD config to use uv dependency groups for docs

020273e

requirements/doc.txt was removed in the uv migration, so RTD builds were failing. Switch to installing via uv export and pip install. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

irfanuddinahmad reviewed Aug 6, 2026

View reviewed changes

mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Aug 7, 2026

mphilbrick211 moved this from Ready for Review to In Eng Review in Contributions Aug 7, 2026

salman2013 and others added 3 commits October 14, 2026 16:32

salman2013 requested a review from irfanuddinahmad November 4, 2026 09:24

irfanuddinahmad suggested changes Nov 4, 2026

View reviewed changes

salman2013 requested a review from irfanuddinahmad November 5, 2026 11:16

irfanuddinahmad approved these changes Nov 5, 2026

View reviewed changes

Conversation

salman2013 commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

openedx-webhooks commented Aug 6, 2026

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad commented Aug 6, 2026

Uh oh!

salman2013 commented Nov 4, 2026

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

salman2013 commented Aug 6, 2026 •

edited

Loading

irfanuddinahmad Aug 6, 2026 •

edited

Loading