Repository navigation
python 3.15.0rc3: align python-feedstock with conda-forge - #256
skupr-anaconda wants to merge 13 commits into
Conversation
Replay #256 topology on origin/master without merge 2a73320. Keep master patch numbering (0028 site.pth, 0029 zlib-ng, 0030 xxlimited). Wrap win FT patches in freethreading jinja; add CF conda-forge#928 libpython unix weak run_exports. Refs: PKG-15601
2a73320 to
6e3e952
Compare
|
Interesting, there is a new vendored |
skupr-anaconda
changed the title
PKG-2624 spike stacked on #256. Drop release variants so this graph uploads only to ad-testing/label/py315-debug, and gate the d ABI with unconditional tests instead of PY_INTERP_DEBUG getenv. Windows skipped. Do not merge to pkgs/main.
Artifact compare vs conda-forge
|
| What | Staging | conda-forge | Action |
|---|---|---|---|
| Linux linker-compat dir | compiler_compat/{ld,README} |
share/python_compiler_compat/ |
It's historical. Optional: move to CF path |
GIL conda-site.pth |
missing | present | Keep missing. Do not restore CFEP-65 lib/python/site-packages (numba failure) |
FT conda-site.pth |
1 line | 2 lines | Keep 1 line |
platform.py / sys.version |
Anaconda branding | conda-forge branding | Keep 0023-branding.patch |
_tkinter |
libtcl9tk9.0 |
libtk8.6 |
Keep tk 9 |
osx-arm64 libpython*.dylib |
11.3 MB + __llvm_prf_* |
6.6 MB, no PGO sections | Look — leftover PGO on the shared dylib |
Shebangs / _sysconfig* / conda-meta differ by prefix (croot/ vs conda-bld/). Cannot match across CIs.
Why binaries differ (do not “fix” to CF)
- tk 9 vs 8.6, glibc 2.28 vs 2.17,
libgcc14 vs 15,libffi3.4 vs 3.7,sqlitevslibsqlite,xzvsliblzma, no_python_rc, licensePSF-2.0vsPython-2.0. - linux-64
CFLAGS:-march=x86-64-v2(aggregate) vs CF-march=nocona..comment: gcc 14.3 vs 15.3. - linux
pyconfig.h: extraHAVE_*(statx, getrandom, copy_file_range, …) from el8 vs cos7. osx: onlyHAVE_READLINE_READLINE_H. - linux
python3.15~40.4 vs 40.3 MB (DWARF, not a file-list bug). Win.pyd~8 KB larger (MSVC + tk 9).
osx-arm64 libpython (the one real size bug)
python3.15 is fine (~6.57 MB both). libpython3.15.dylib is not:
- Staging
__TEXT,__text6.46 MB vs CF 3.86 MB. - Staging has
__llvm_prf_cnts/data/names/vnds(~2.0 MB). CF has none.
Cause: both configures get --enable-optimizations, but only the static tree runs make profile-opt. Shared make leaves PGO instrumentation in the dylib that install_shared.sh ships. Same story for libpython3.15.nolto.a (21.5 vs 14.6 MB).
Fix (follow-up, not required for file-list parity): do not pass --enable-optimizations into the shared configure, or run/strip PGO on that tree before cp libpython*.dylib.
Bottom line
File names: aligned except linux compiler_compat/ and GIL conda-site.pth.
File bytes: will never match CF while we keep tk 9, glibc 2.28, x86-64-v2, branding, and PBP prefixes. That is the frozen AR-delta, not a regression.
Optional in this PR: relocate compiler_compat. Do not chase CF bytes. Worth a small follow-up for the osx shared-lib PGO leftover.
Artifact compare vs pkgs/main
|
| What | Staging 3.15 | pkgs/main 3.14.7 | Action |
|---|---|---|---|
Shared libpython |
new output; python has no .so/.dylib |
so/dylib inside python; no libpython pkg |
Keep CF-align split |
GIL conda-site.pth |
missing | present | Keep missing (numba / CFEP-65) |
bin/python3.1 |
absent | present | 3.14 conda#10969 workaround; optional |
_sysconfigdata_*_conda_cos6_* |
absent | present | Keep dropped CentOS clones |
sre_{compile,parse,constants}.py |
absent | present | Expected — removed upstream in 3.15 |
_asyncio*.so/.pyd |
absent | present | Expected — folded into interpreter in 3.15 |
Lib/profiling/sampling/ (~81 files) |
present | absent | Expected — new in 3.15 |
_math_integer, xxlimited_3_13 |
present | absent | Expected — new 3.15 modules |
| Win layout | lib/python/ + lib-dynload/ |
Lib/ + DLLs/ |
Keep CFEP-27 |
cpython noarch |
present | absent | Keep CF-align metapackage |
| win-arm64 | no artifacts | GIL + FT on main | PBP pool, not recipe |
Shebangs / _sysconfig* / conda-meta differ by prefix and by version. Cannot match across CIs or minors.
Why binaries differ (do not “fix” to 3.14)
- Different CPython (3.15.0rc2 ≠ 3.14.7). Interpreter /
.sobytes will never match. - Pin floors only:
libexpat2.8.5 vs 2.8.2,sqlite3.53.4 vs 3.53.2,ncurses6.6 vs 6.5,openssl3.5.8 vs 3.5.7. Same family as 3.14.7, not CF 8.6/glibc 2.17. - C-API headers renamed (
lock.h→pylock.h,typeslots.h→slots.h). - pkgconfig filenames carry the version (
python-3.15*.pcvspython-3.14*.pc).
win-arm64 (recipe yes, this graph no)
build_base.bat already sets PLATFORM=ARM64 / BUILD_PATH=arm64 (arm64t for FT) — the 3.14 #240 port. No extra LLVM override (3.15 upstream is 21).
PBP /check-queue lists win-amd64 only. Perseverance-skills: win-arm64 “(soon)”. 3.14.7 on main proves host deps work (ucrt, vc14, tk 9, openssl 3.5). This SHA never scheduled that subdir.
Bottom line
File names (Unix, version-normalized): aligned except CF-align split (libpython out of python), dropped cos6 sysconfig / python3.1, and 3.15 stdlib adds (profiling/, _math_integer).
File bytes: will never match 3.14.7 — different interpreter. That is expected, not a #256 regression.
Win is a layout rewrite (CFEP-27), not missing modules. Optional later: win-arm64 once PBP has a worker. Do not chase 3.14 paths or CF bytes.
Replay #256 topology on origin/master without merge 2a73320. Keep master patch numbering (0028 site.pth, 0029 zlib-ng, 0030 xxlimited). Wrap win FT patches in freethreading jinja; add CF conda-forge#928 libpython unix weak run_exports. Refs: PKG-15601
Keep build_type: release only in conda_build_config.yaml. Debug interpreter remains a PKG-2624 spike, not this CF-align PR. Refs: PKG-15601
Auto-bump must track every output from the CF-align split. Refs: PKG-15601
Debug variants stay on ad-testing/label/py315. Disable before any pkgs/main / stable 3.15.0+ release (PKG-2624). Refs: PKG-15601
Debug builds are exclusive to python-feedstock#258 / PKG-2624. Refs: PKG-15601
pyexpat overlinks libexpat 2.8.5; host expat has no run_exports, so the python output never got a run req. Same fix as #258. Refs: PKG-15601
This reverts commit 226ab8c.
Refs: PKG-15601
cffd5c1 to
ed13f25
Compare
|
Review #259 first (C16+C17 libpython split vs This PR (#256) is now based on Dest remains |
Keep #256 CF-align remainder (cpython, freethreading CBC, no from_source_control/gil_type). Take master 3.15.0rc3 sha256, macOS shared-PGO dylib symlink, prefix-replacement NEEDED checks, and explicit run: on noarch outputs. Keep build_number 1 so rc3 CF-align artifacts do not collide with master's _0 on ad-testing/label/py315.
skupr-anaconda
changed the title
skupr-anaconda
added
automated package review
and removed
automated package review
labels
🟠 Mergeability confidence: 65/100 (low) — 2 reviewers recommended135 checks · 104 passed · 6 warnings · 2 critical Categorization
Needs attentionlibpython
python
unknown
Passedno-new-dependencies Could not determine
Analysed channel: Automated signals — not an approval. Verify flagged items before merging. |
I think we were leaning into NOT introducing a new Here it isn't doing anything useful (unlike So I would remove the ( I guess there's a possibility that someone will now want a |
Ian: no cpython metapackage; python-{gil,freethreading,jit} depend on
python. Hold libpython-{gil,freethreading}. Charles: restore linux-only
xorg, drop unused xorg-libx11 test dep, restore doc/dev URLs, keep
.vscode/, comment-only debug CBC, delete dead CF comments and the
commented MSBuild cross block.
|
Addressing automated package review (
Charles recipe cleanups in the same commit: linux-only xorg, drop extra Linter-only red on graph |
Try the python output without \$RPATH libc/libcrypt/libdl/libm/libnsl/ libpthread/librt/libutil. Win MSVCR entries stay. Restore any soname conda-build still flags as missing DSO.
Remove the commented \$RPATH libc/libcrypt/libdl/libm/libnsl/libpthread/ librt/libutil entries. Win MSVCR whitelist stays; selector is win-only.
cbouss
left a comment
cbouss
left a comment
There was a problem hiding this comment.
Can you squash all commits into one and remove most comments of this PR?
I ran some comparison tests between this build and our previous build and found one issue regarding jit that need to be addressed. See inline comments.
Also:
- install_static.sh should be aligned. There are places where VERABI_NO_DBG should be used over VERABI
- fix_staged_recipe should be updated following conda-forge#944
| _common_configure_args+=(--with-system-libmpdec=yes) | ||
|
|
||
| if [[ "${DEBUG_PY}" == "yes" || "${target_platform}" != *"64" || ${PY_GIL_DISABLED} == yes ]]; then | ||
| if [[ "${PY_INTERP_DEBUG}" == "yes" || "${target_platform}" != *"-64" || ${PY_FREETHREADING} == yes ]]; then |
There was a problem hiding this comment.
| if [[ "${PY_INTERP_DEBUG}" == "yes" || "${target_platform}" != *"-64" || ${PY_FREETHREADING} == yes ]]; then | |
| if [[ "${PY_INTERP_DEBUG}" == "yes" || "${target_platform}" != *"64" || ${PY_FREETHREADING} == yes ]]; then |
With "${target_platform}" != *"-64", jit is disabled on osx-arm64 or linux-aarch64. But jit is supported on this platforms.
run_test_jit.py should be updated to fail when python is detected to not be built with jit.
| # python-jit is noarch and PBP tests it against every python variant. | ||
| # configure disables JIT for debug, freethreading, and some arches; | ||
| # the metapackage is a no-op there. | ||
| print("JIT not compiled into this interpreter; skip") | ||
| sys.exit(0) |
There was a problem hiding this comment.
| # python-jit is noarch and PBP tests it against every python variant. | |
| # configure disables JIT for debug, freethreading, and some arches; | |
| # the metapackage is a no-op there. | |
| print("JIT not compiled into this interpreter; skip") | |
| sys.exit(0) | |
| print("ERROR: JIT not compiled into this interpreter", file=sys.stderr) | |
| sys.exit(1) |
3 participants
Destination channel: ad-testing/label/py315
Links
main@ 3.15.0rc3 (#885, #928, #943)Explanation of changes:
Onto current
master(feb5133b: #259 then #267). Mergecb9904b1, then review cleanups throughfa2cc0c0. CF-align remainder only — do not re-review the libpython split or the rc3 bump.Taken from master
dev=rc3, sha256506630aa…(Python-3.15.0rc3.tar.xz). Shared patches already refreshed in python 3.15.0rc3 #267.build_base.sh(20d8a034).otool -Lchecks +exit 1(4ccdc624).run:on noarch outputs (b9667f6c).Kept from this PR (#256 remainder)
gil_type→freethreading: yes/noandPY_FREETHREADING. FT on all platforms.build_type: releaseonly (# - debugcommented; unix debug is #258 / PKG-2624).source:is url + sha256 + patches only (nofrom_source_control/ git_url). Dropgil_type,PY_GIL_DISABLED,bootstrap,linkage_nature.build_number1 so rc3 CF-align artifacts do not collide with master's_0onad-testing/label/py315.{% if freethreading == "yes" %}. 0023-branding.patch always applied.Python_FIND_ABI(cmake_test_args). Win import-lib tests cover bothlib\andlibs\(CFEP-27 / CF keep import libraries in libs to help meson and other build systems conda-forge/python-feedstock#918).channel_targets.python_site_packages_pathset (GILlib/python3.15/site-packages, FTlib/python3.15t/site-packages, winlib/python/site-packages). Changesindex.jsonvs 3.14.7; payload dirs already matched.build_base.sh: SYSCONFIG /build-details.jsonfrom shared build (CF LDLIBRARY is set to libpython3.9a although no static lib is included in mambaforge for Linux conda-forge/python-feedstock#565),VERABI_NO_DBGpic.a,llvm-tools-21via$HOST-llvm-ar. No CentOSOLD_HOST/conda_cos6clones.build_base.bat:PY_FREETHREADING. Win tkTclVersion=%tk%/TkVersion=%tk%(tk 9).run_test_jit.py: skip (exit 0) when this interpreter was built without JIT.python-jitstill skipped for FT.xorg-libx11in pythontest.requires.doc_url/dev_urlrestored to python.org docs + devguide.Review follow-ups (Ian / Charles,
03cfb43d…fa2cc0c0)cpythonoutput.python-freethreading/python-gil/python-jitrun: python, notcpython. Nolibpython-{gil,freethreading}until a named consumer exists..vscode/in.gitignore.$RPATHsysrootmissing_dso_whitelist(libc/libcrypt/libdl/libm/libnsl/libpthread/librt/libutil). Win MSVCR /api-ms-win-*whitelist stays (# [win]).Already on master (from #259 — not new here)
libpython(install_shared.sh) /libpython-static(install_static.sh).pythondoes not shiplibpython*.so/.dylib.libpython3.dylibreexport; CF add run-export to libpython conda-forge/python-feedstock#928 unix weakrun_exportsonlibpython.Frozen AR-delta vs conda-forge (intentionally not copied)
llvm-tools-21(not unversionedllvm-tools)._python_rc; nochannel_sources: python_rc/python_debugzip.msys2-gcc-libs; nobinutils_impl/patch/m2-patch.downstreamscommented;cmake-no-system/ninja-base/make.Notes:
ad-testing/label/py315only. Do not merge to pkgs/main.parsing_jinja_failureis skipped ({% if freethreading is not defined %}). Graph generate must succeed; the linter skip may stay red.libpython3.sois empty upstream (cpython#104612); we follow CF and only testlibpython3${SHLIB_EXT}on unix release.