Repository navigation
skpkg: final work required for scikit-packaging libdiffpy #63
Description
All migration work is done on skpkg-migration
CI
- Shrink the matrix. Python is only used to run SCons, so testing 3.12, 3.13 and 3.14 triples the jobs without testing anything new. Test one Python version on each of the 4 operating systems.
- Decide when the matrix runs. Running on push to
mainis commented out inmatrix-and-codecov.yml; the scikit-package default runs on merge tomain. - Release occurs before matrix test passes. Today the matrix runs after the GitHub release is published and does not block it. This test should pass before a release occurs.
- Use the same setup for PR tests and the matrix.
tests-on-pr.ymlwrites its ownsconscript.localand installscxxtestfrom conda.matrix-and-codecov.ymluses.github/scripts/configure-matrix.pyand a git checkout of CxxTest. Use one setup for both. - The docs workflow only works by accident. The reusable
_release-docs.ymlworkflow inscikit-package/release-scriptsrunspython -m pip install -e . --no-deps. libdiffpy has no Python project, so pip installs an empty placeholder package calledUNKNOWN 0.0.0. The docs build passes anyway, but only by accident. Decide how we should handle it or if we can just ignore it for now. - enable pre-commit bot for this repo.
Downstream packages
- diffpy.srreal will fail to build against migrated libdiffpy. srreal compiles libdiffpy's sources from the commit pinned in
cmake/LibdiffpyVersion.cmake. ItsCMakeLists.txtinstalls${LIBDIFFPY_SOURCE_DIR}/LICENSE.txt, which this migration renamed toLICENSE.rst. Once the pin moves to a migrated commit, the srreal build fails. Open an issue in diffpy.srreal to update that path. Check this when chore: use external libdiffpy diffpy.srreal#98 is merged. - Test diffpy.srreal against migrated libdiffpy before merging to
main(requested in v1.4.1 release tag is wrong #40). - Update the conda-forge
libdiffpy-feedstock. Write issues for the following and fix them in the feedstock:- Change the source URL from
archive/v{{ version }}.tar.gztoarchive/{{ version }}.tar.gz, because release tags no longer start withv. - Change
license_filefromLICENSE.txttoLICENSE.rst. - Simplify
build.shandbld.bat. SConstruct now installs to$PREFIX(or$CONDA_PREFIX) by default and uses theLibrary/layout on Windows. - Add
run_exportswithpin_subpackage(name, max_pin='x.x')so that packages built against libdiffpy pin a compatible version (see bugs: ABI incompatibility after1.5.0release #47).
- Change the source URL from
Optional cleanup
- Rename
LICENSE_DANSE.txttoLICENSE_DANSE.rstto match diffpy.srreal. srreal's build copies this file by name too, so do it together with the srreal license-path fix above. #67 - Replace the hand-formatted license text in
docs/source/license.rstwith.. include:: ../../LICENSE.rst, so there is only one copy of the license to maintain. #68
Release
- Merge
skpkg-migrationintomain. - Before tagging, set
FALLBACK_VERSIONinsite_scons/fallback_version.pyto the release version. - Make the 1.5.0 release issue from the release checklist template (make a release issue after skpkging is done. #54).
Activity
cadenmyers13 commentedon Oct 29, 2026
@SYMSCAE See above for what needs to be done.
SYMSCAE commentedon Oct 29, 2026
@cadenmyers13 For entry
diffpy.srreal will fail to build against migrated libdiffpythere is an existing PR diffpy/diffpy.srreal#98 that reverts this change and makes srreal depend on released libdiffpy.cadenmyers13 commentedon Oct 29, 2026
@SYMSCAE okay great. I updated above
SYMSCAE commentedon Oct 29, 2026
@cadenmyers13 Please add one more entry to enable pre-commit on this repo, I've setup clang-format so the hooks now work for cpp files as well.
SYMSCAE commentedon Oct 30, 2026
@cadenmyers13 I think we are good to merge and release an rc? There's only one
_release-docs.ymlleft and it seems the cleanest way to fix is to add a flag to upstreamrelease-scripts(something like install_pkg = true).cadenmyers13 commentedon Oct 30, 2026
@SYMSCAE Have all the above items been checked then? I think its quick and easy to make that change to
release-scriptsbecause it doesn't need a release.SYMSCAE commentedon Oct 30, 2026
ci checks are all set except for docs workflow and pre-commit bot. I'll add a PR to
release-scriptsto bypass python installation with a flag. @sbillinge Could you please enable pre-commit bot for this repo?cadenmyers13 commentedon Oct 30, 2026
@SYMSCAE great, thanks. That will also require a PR here too, right?
SYMSCAE commentedon Oct 30, 2026
@cadenmyers13 Indeed, should be a one-line change.
sbillinge commentedon Oct 31, 2026
@SYMSCAE @cadenmyers13 I believe that I activated pre-commit.ci successfully, but please check. Are we nearly there with this?
SYMSCAE commentedon Oct 31, 2026
@cadenmyers13 I think we're good for rc0 for
libdiffpy,srrealhas been manually checked (tests all passed and linked properly). But I'm not quite sure about the approach to handle compatibility forsrreal.I've opened up a PR conda-forge/diffpy.srreal-feedstock#5 in srreal's feedstock to trigger a new release for 1.4.0 srreal (so that new installations of
srrealwon't fetch 1.5.0rclibdiffpyif we release rclibdiffpyin main channel). Or should we create a prerelease channel for both so that there's no such workaround and we only need to do one repodata patch afterwards. @sbillinge Please see which is more preferrable.sbillinge commentedon Nov 1, 2026
Thanks @SYMSCAE this is great. As a general matter, we don't really want releases to depend on the release state of other packages because weird time-dependent pins are just deferred maintenance. It means that when we push things out we have to do them in the right order. For example, we usually start with
diffpy.utilsandbg-mpl-stylesheetswhich are used by just about everything and don't depend on anything, and then kind of roll things out from there. What that means is that my preference here is to getlibdiffpypushed out all the way to Conda-Forge (and tested in between) and then push out srreal but without any weird pins. It slows things down a bit, but keeps things a bit cleaner and minimizes code entropy. In this case, it is a bit trickier because we are usingsrrealto test that we didn't breaklibdiffpy, so thanks for that. In general, it would be good if the things that srreal is testing could be included inlibdiffpytest suite (but without asrrealdependency), which would allow us to completely separate the releaseses, but let's not worry about that now.sbillinge commentedon Nov 1, 2026
@SYMSCAE @cadenmyers13 please can you check off all the todo items in the top comment. I will use that, and any open issues and PRs as the things that need to be close to move to release.
cadenmyers13 commentedon Nov 2, 2026
@SYMSCAE @sbillinge I checked everything that is completed. I tried to bump the fallback version but it fails test because it compares to the current git tags. Next we need to merge
skpkg-migrationintomain. Some merge conflicts will need to be fixedsbillinge commentedon Nov 2, 2026
I created this PR.
cadenmyers13 commentedon Nov 2, 2026
@SYMSCAE Can you handle the merge conflicts between
mainandskpkg-migration? C++ code has been modified and I'm not sure which one is the right choice. Something must've been merged intomainwhile we were doing edits on the migration branch.sbillinge commentedon Nov 2, 2026
We merged the windows branch into main. We should have done the skpkg work off of the windows branch but we didn't realize it wasn't fully merged as it was the production version so should have been. Could this explain everything? It may be better to close this and redo the skpkg work?
cadenmyers13 commentedon Nov 2, 2026
@sbillinge I would rather not redo the skpkg work because it isn't super trivial. I think @SYMSCAE can handle the merge conflicts since he did the merge of windows to main. Its only a handful of files and a few lines that have the conflicts.
SYMSCAE commentedon Nov 2, 2026
@cadenmyers13 @sbillinge PR #60 merges
windowsintoskpkgrather thanmainand preserves the original commit history. The reason there was tons of merge conflicts before and file changes because the windows branch introduces code formatter for C++, which is also preserved and wired into ourpre-commitworkflow.