Skip to content

Add networkit 11.2.1 riscv64 wheels - #3089

Draft
luhenry wants to merge 2 commits into
mainfrom
networkit
Draft

luhenry wants to merge 2 commits into
mainfrom
networkit

Conversation

@luhenry

@luhenry luhenry commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Builds networkit's CMake/Cython C++ graph-analysis extension modules. Upstream publishes no riscv64 wheel.

Mirrors the Linux job of upstream's build_wheels.yml.

Differs from upstream

  • Adds a gpl_sources job - the build links OpenMP, so auditwheel vendors the manylinux image's libgomp.

Matrix: cp310-cp313, matching the ABIs upstream actually released for 11.2.1 (its CI matrix on master also builds cp314/cp315, but the PyPI release doesn't).

Testing

  • same as upstream

License: Wheel vendors the manylinux image's libgomp (GPL-3.0 with the GCC Runtime Library Exception); gpl_sources collects its sources.

Builds cp310-cp313 wheels for networkit, a C++ graph-analysis library
with Cython bindings and OpenMP parallelism. CMake build is portable:
the only SIMD (AVX2) and x86 inline-asm (ttmath) paths are each gated
behind proper __ifdef__ fallbacks to scalar/noasm code, and the vendored
tlx/dhb submodules add no arch-specific requirements. Validated locally
by cythonizing + cmake/ninja building the full tree and running a smoke
import/graph/centrality test against the resulting module.

A gpl_sources job collects the manylinux_riscv64 image's gcc, since the
build links OpenMP (libgomp).
luhenry added a commit that referenced this pull request Oct 13, 2026
@luhenry
luhenry marked this pull request as draft October 14, 2026 11:33

luhenry commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

Build networkit 11.2.1 cp310-manylinux_riscv64 (job 115420476819) failed compiling distance.cpp with GCC reporting several "redefinition of ..." errors for the same Cython-generated symbol, each pointing at two near-duplicate copies of the same function body inside that one generated file (e.g. lines 5425 and 6419 both define __pyx_convert_vector_to_py___pyx_t_9networkit_10structures_node). That's the signature of a parallel-build race corrupting a shared generated file, not a real code defect: this port's own PR description says it was validated locally by cythonizing + building with ninja, while cibuildwheel's isolated build here invokes CMake's default Unix Makefiles generator, whose recursive sub-make parallelism is the classic source of this exact race (ninja tracks one global dependency graph and doesn't have it).

Not fixing anything in the port for this — the matrix's other legs (cp312, cp313 still running at comment time, cp311 queued) aren't affected the same way, consistent with a race rather than a deterministic break. I'll re-run just the failed cp310 job (the one re-run this process allows for a suspected flake) once the rest of this run finishes — GitHub refuses a rerun while sibling jobs in the same run are still in progress. Watching for the result.


Generated by Claude Code

luhenry commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

Correction to my last comment: this is not a flake. Build networkit 11.2.1 cp312-manylinux_riscv64 (job 115420476885) just failed with the exact same distance.cpp redefinitions at the exact same line numbers (6419/5425, 6574/5580, 6884/6109) as the cp310 job — reproduced byte-identically across two independent runners. That rules out a parallel-build race; it's deterministic.

Working theory: cibuildwheel's isolated build uses CMake's default "Unix Makefiles" generator, and the PR's own description says local validation used ninja. A classic cause of this exact symptom (a Cython-generated file's contents duplicated wholesale) is CMake declaring the same cythonize-generated output from two different targets' custom commands — recursive Make's per-target sub-make invocations can each re-run that command without a shared build graph to dedupe against, while Ninja's single global graph wouldn't. I'm dispatching a subagent to confirm this against networkit's actual CMakeLists.txt and, if confirmed, switch this build to Ninja (matching the already-validated local config) rather than guessing blind. No re-run in the meantime — it would just reproduce the same deterministic failure.


Generated by Claude Code

networkit 11.2.1's build-system.requires leaves Cython unbounded, so the
isolated build env resolves Cython 3.3.0, released after 11.2.1. setup.py
cythonizes all modules once, serially, before CMake runs; CMake only
compiles the pre-generated networkit/<mod>.cpp files (no custom commands),
so the Makefiles-vs-Ninja generator choice does not affect the result.

Cython 3.3.0 itself emits the vector->list converter helpers for
structures.node, double and vector[node] twice into distance.cpp, which
GCC rejects as "redefinition of __pyx_convert_vector_to_py_...". A plain
`cython -3 --cplus networkit/distance.pyx` reproduces it on x86_64; the
other 30 modules are unaffected, and Cython 3.2.9 emits each helper once.

Write a Cython<3.3.0 constraint in CIBW_BEFORE_ALL and pass it through both
PIP_CONSTRAINT and PIP_BUILD_CONSTRAINT, since the manylinux image's pip
26 only honours the latter inside the isolated build environment.

luhenry commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

Root cause confirmed, and my Make-vs-Ninja theory was wrong: CMake here only compiles pre-generated .cpp files (no custom codegen commands), so the generator choice is irrelevant. The real cause is Cython 3.3.0, which networkit 11.2.1's unbounded build-system.requires lets the isolated build env resolve (3.3.0 was released after 11.2.1). That version emits the vector→list converter helpers for structures.node, double and vector[node] twice into distance.cpp; a bare cython -3 --cplus networkit/distance.pyx reproduces it standalone, and 3.2.x emits each helper once. All three legs (cp310/cp312/cp313) failed identically because it's fully deterministic, not a race.

Fix pushed: commit 60a61c3, pinning Cython<3.3.0 via CIBW_BEFORE_ALL + both PIP_CONSTRAINT/PIP_BUILD_CONSTRAINT (the image's pip only honors the latter inside the isolated build env). Validated by resolving cython==3.2.9 and building the wheel clean on x86_64, and by reproducing the exact CI error compiling distance.cpp with 3.3.0 vs. clean with 3.2.9 under the real manylinux_2_39_riscv64 image. Watching for the new CI run.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant