From 2636e20e04750ccf8282eaa871d1c7ec605bedad Mon Sep 17 00:00:00 2001 From: Dianjin Wang Date: Wed, 9 Sep 2026 14:30:45 +0800 Subject: [PATCH] Upgrade gpMgmt's bundled Python dependencies PyYAML 5.4.1 no longer builds its C extension on Rocky 10. Its pre-generated _yaml.c declares the libyaml read/write handlers with char * where libyaml wants unsigned char *, and GCC 14 turned that from a warning into an error: yaml/_yaml.c:3598:52: error: passing argument 2 of 'yaml_parser_set_input' from incompatible pointer type [-Wincompatible-pointer-types] /usr/include/yaml.h:1374:30: note: expected 'int (*)(void *, unsigned char *, size_t, size_t *)' but argument is of type 'int (*)(void *, char *, size_t, size_t *)' Error compiling module, falling back to pure Python The build then completes with the slow pure-Python parser, so gpload still works; this is quality rot rather than a hard failure. Note that the CFLAGS="-w" already passed here cannot suppress it, since -w only silences warnings and GCC 14 makes this an error. While in here, the other two bundled packages are equally old: PyGreSQL 5.2 is from 2020-06, psutil 5.7.0 from 2020-02. PyGreSQL 5.2 -> 5.2.5 psutil 5.7.0 -> 7.2.2 PyYAML 5.4.1 -> 6.0.1 The ceilings are set by Python 3.6, which Rocky 8 ships as its system python3 and which is still a supported build platform: * PyYAML 6.0.1 is the newest release declaring >=3.6; 6.0.2 moved to >=3.8. * PyGreSQL stays on the 5.x line on purpose. 6.x requires >=3.7 and its setup.py refuses older interpreters outright ("Sorry, PyGreSQL 6.1.0 does not support this Python version"). 5.2.5 is also a drop-in, which matters because gpMgmt imports the top-level pg and pgdb modules in eleven files. * psutil is the exception: even 7.2.2 still declares >=3.6, and the only API gpMgmt uses is virtual_memory(), Process(), Process.children(), Process.memory_info(), wait_procs(), pid_exists() and NoSuchProcess, none of which changed in 6.x or 7.x. Verified on all five build platforms (Rocky 8/9/10, Ubuntu 22.04/24.04) by running the real download-python-deps, pyyaml, psutil and pygresql targets: all succeed with no errors and no pure-Python fallback, and importing yaml, psutil, pg and pgdb out of the resulting gpMgmt/bin/ext exercises the calls listed above. Rocky 8 covers the Python 3.6 end and Rocky 10 the GCC 14 end. This is not a security fix: OSV reports no known advisories against either the old or the new versions. Two things worth knowing before the next bump. From PyYAML 6.0.2 and PyGreSQL 6.0.1 onwards the sdist filename is lowercase, which the hardcoded PyYAML-$(PYYAML_VERSION).tar.gz style URLs and directory names here do not handle; since curl runs without -f the 404 body lands in the tarball and the failure surfaces at tar time. And the "cython<3.0.0" pin in download-python-deps is load-bearing: PyYAML re-cythonizes when Cython is importable, and both 5.4.1 and 6.0.1 die with "AttributeError: cython_sources" under Cython 3. Assisted-by: Claude Code Backpatch-through: REL_2_STABLE --- gpAux/client/install/src/windows/CreatePackage.bat | 2 +- gpMgmt/bin/Makefile | 6 +++--- python-dependencies.txt | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/gpAux/client/install/src/windows/CreatePackage.bat b/gpAux/client/install/src/windows/CreatePackage.bat index 04bd76b52b7..a21458e9260 100644 --- a/gpAux/client/install/src/windows/CreatePackage.bat +++ b/gpAux/client/install/src/windows/CreatePackage.bat @@ -12,7 +12,7 @@ copy ..\..\..\..\..\gpMgmt\bin\gppylib\gpversion.py %GPDB_INSTALL_PATH%\bin\gppy perl -pi.bak -e "s,\$Revision\$,%VERSION%," %GPDB_INSTALL_PATH%\bin\gpload.py copy ..\..\..\..\..\gpMgmt\bin\gpload.bat %GPDB_INSTALL_PATH%\bin REM Install PyYAML using pip instead of extracting from tarball -pip3 install --target=%GPDB_INSTALL_PATH%\lib\python PyYAML==5.4.1 +pip3 install --target=%GPDB_INSTALL_PATH%\lib\python PyYAML==6.0.1 perl -p -e "s,__VERSION_PLACEHOLDER__,%VERSION%," greenplum-clients.wxs > greenplum-clients-%VERSION%.wxs candle.exe -nologo greenplum-clients-%VERSION%.wxs -out greenplum-clients-%VERSION%.wixobj -dSRCDIR=%GPDB_INSTALL_PATH% -dVERSION=%VERSION% light.exe -nologo -sval greenplum-clients-%VERSION%.wixobj -out greenplum-clients-x86_64.msi diff --git a/gpMgmt/bin/Makefile b/gpMgmt/bin/Makefile index 7092700b784..3c7a6780dcd 100644 --- a/gpMgmt/bin/Makefile +++ b/gpMgmt/bin/Makefile @@ -83,9 +83,9 @@ endif # This replaces the previously bundled tar.gz files to comply with Apache Release policy # MOCK_VERSION=1.0.1 -PYGRESQL_VERSION=5.2 -PSUTIL_VERSION=5.7.0 -PYYAML_VERSION=5.4.1 +PYGRESQL_VERSION=5.2.5 +PSUTIL_VERSION=7.2.2 +PYYAML_VERSION=6.0.1 download-python-deps: @echo "--- Downloading Python dependencies for gpMgmt modules" diff --git a/python-dependencies.txt b/python-dependencies.txt index 7ca6d4dcf65..5a9bb76cc02 100644 --- a/python-dependencies.txt +++ b/python-dependencies.txt @@ -1,3 +1,3 @@ -psutil==5.7.0 -pygresql==5.2 -pyyaml==5.4.1 +psutil==7.2.2 +pygresql==5.2.5 +pyyaml==6.0.1