Repository navigation
Documentation doesn't generate with docutils >= 0.22 #139257
Description
Documentation
The html doc is not building with the new docutils >= 0.22. It's not a problem yet, because the current Sphinx used (sphinx~=8.2.0) doesn't install the latest one, but it's something that should be fixed for the future.
The problem can be reproduced:
$ cd Doc
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
$ pip install -U docutils
$ sphinx-build --pdb -b html . build/html
[...]
File "env/lib64/python3.13/site-packages/docutils/parsers/rst/states.py", line 1494, in parse_enumerator
ordinal = int(self.enum.converters[sequence](text))
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
The actual problem is directly related to this line in docutils, done as part as this commit.
They wrap the converters with int so the monkey patch done in pyspecific.py doesn't work anymore.
# monkey-patch reST parser to disable alphabetic and roman enumerated lists
from docutils.parsers.rst.states import Body
Body.enum.converters['loweralpha'] = \
Body.enum.converters['upperalpha'] = \
Body.enum.converters['lowerroman'] = \
Body.enum.converters['upperroman'] = lambda x: None
Activity
pythongh-139257: Support docutils >= 0.22
pythongh-139257: Add news entry
picnixz commentedon Sep 23, 2025
I don't think we should actually worry about a docutils version that is not installed by the environment itself. I would rather pin the docutils version in
constraints.txtinstead of re-patchingconf.pydirectly. It might not be the only issue we'll have if we upgrade to newest Sphinx.Is there a reason why the docutils version is actually being updated after installing the requirements? maybe we should pin the docutils version anyway so that it's compatible with our
conf.py(having version-guardedifs always complicates maintenance).danigm commentedon Sep 23, 2025
This problem appears in openSUSE where we are trying to update
docutils, and we can patch downstream to keep it working. But as I said, this is not a problem right now, but will be a problem in the future, so I tried to work on a fix for it.picnixz commentedon Sep 23, 2025
Ah I see in the logs
So the venv you're using for building the docs on openSUSE is actually using the latest docutils and the latest Sphinx (for some reasons) instead of those given in
Doc/requirements.txt. How easy is it to actually use those instead? I would prefer pinning the versions inDoc/requirements.txtandDoc/constraints.txtinstead as other users might benefit from this locally. If this is not easy, I can indeed accept the docutils patch (other distros might have the same issue)danigm commentedon Sep 23, 2025
That is the main issue. We build cpython in openSUSE with the current packages, everything installed from the rpm repository so there's no venv during build. That's why if we want to update docutils we need to patch cpython to work correctly with the new version.
For documentation usually the Sphinx version is the problem and we are more conservative when updating Sphinx, but in this case, it's docutils and this specific monkey-patch that doesn't work with the current version.
The patch is not very complex and it's compatible with the previous versions, so I though that it could be interesting to report and fix upstream.
picnixz commentedon Sep 23, 2025
Mmh. Are you aware of other distributors that could have a similar issue? I don't know much about their process. If all other major distributors also don't use our venv, then I think we could allow this patch (or we can also change upstream docutils, cc @AA-Turner).
Considering the issue, I think I'd be fine with the version-guarded patch.
AA-Turner commentedon Sep 23, 2025
We don't support building the documentation with exotic configurations/environments. When we update to a newer version of Sphinx, we'll address this. Thank you for raising the issue, though.
A
mcepl commentedon Sep 24, 2025
I think this will be the problem for all Linux distributions (and others like
*BSD, Homebrew on Mac, etc.), only some of them have so far olddocutils(e.g., Fedora is still on 0.21.2, Debian will hits this inexperiemental).picnixz commentedon Sep 24, 2025
@AA-Turner I think we could make the change here otherwise it could complicate redistribution. We're already somehow monkey-patching docutils so maybe it doesn't hurt to monkey-patch it better (the fix looks fine, and instead of an ImportError, we could use docutils' version guards instead).
Update python39 to version 3.9.23 / rev 80 via SR 1308251
3 remaining items
Update python312 to version 3.12.11 / rev 38 via SR 1308661
stefanor commentedon Oct 13, 2025
Coming here from Debian which is hitting the same issue:
I would expect all Linux distributions to run into this. We build self-contained systems. This typically requires things to work with the latest versions of each other. We can hold something at an older version for a while while we figure out an issue, but pinning anything to an ancient version is not possible.
pythongh-139257: docutils 0.22 support
pythongh-139257: docutils 0.22 support
pythongh-139257: docutils 0.22 support
picnixz commentedon Oct 13, 2025
I see. @AA-Turner The fix on our side is not really hard and I think this would help getting the latest python on mainstream distros. Would you be willing to reconsider your position here?
gpshead commentedon Oct 13, 2025
How is anyone maintaining documentation supposed to know what needs escaping? These escapes are unnatural.
What checks do we have in place that prevent docs from being checked in that conflict with unsupported-by-us future versions of docutils? Unless we have linting and CI checks that make these very-awkward escaping rules discoverable in our docs maintenance I expect such a PR will be temporary at best and regress in the future.
That doesn't mean we should not simply apply the PR for practical purposes - but nobody can expect our doc builds to stay working for the long term on arbitrary not-what-we-use docutils version without a CI test matrix in place.
stefanor commentedon Oct 14, 2025
My PR #139257 removed the monkeypatch, because things were escaped, but it could instead raise an exception, as I proposed on it.
From my PoV, I don't think that's something you really need to worry about. If things break loudly (like this) people will come and fix them. If they break silently, maybe you won't hear for a while, but you'll have the same silence problem when you make the docutils upgrade yourself.
Python could, of course, have a CI job that builds the docs with upstream unpinned versions of things, that'd probably be nice. But I'm not a Python Core developer at this point, so I'm not going to propose that you take on the burden of maintaining that.
befeleme commentedon Nov 24, 2025
Late to the party, but coming from Fedora with the same issue - also confirming that pinning is not really an option for us. I managed to fix or backport the fixes to all packages broken with docutils 0.22 in Fedora, the ecosystem seems to be pretty prepared for it. For me, it's just the Python docs left.
pythonGH-139257 Use an explicitly named link
picnixz commentedon Nov 29, 2025
Since this issue has been addressed in #121970, I'm going to close this it. We went for an alternative change that doesn't monkeypatch docutils.
picnixz commentedon Nov 29, 2025
We also fixed this in #142057.