Merge branch 'python:main' into doc-regrtest-unittest-loader

Author: bazhil

Doc/c-api/exceptions.rst

@@ -1034,6 +1034,10 @@ because the :ref:`call protocol <call>` takes care of recursion handling.
    case, a :exc:`RecursionError` is set and a nonzero value is returned.
    Otherwise, zero is returned.
 
+   The check is based on the remaining C stack space of the current thread,
+   not on a count of calls, so it is unaffected by
+   :c:func:`Py_SetRecursionLimit` and :func:`sys.setrecursionlimit`.
+
    *where* should be a UTF-8 encoded string such as ``" in instance check"`` to
    be concatenated to the :exc:`RecursionError` message caused by the recursion
    depth limit.
@@ -1044,6 +1048,10 @@ because the :ref:`call protocol <call>` takes care of recursion handling.
    .. versionchanged:: 3.9
       This function is now also available in the :ref:`limited API <limited-c-api>`.
 
+   .. versionchanged:: 3.14
+      The check is based on the remaining C stack space.  Previously, a
+      separate counter of C-level calls was used.
+
 .. c:function:: void Py_LeaveRecursiveCall(void)
 
    Ends a :c:func:`Py_EnterRecursiveCall`.  Must be called once for each

Doc/c-api/interp-lifecycle.rst

@@ -514,7 +514,7 @@ Process-wide parameters
    Return the version of this Python interpreter.  This is a string that looks
    something like ::
 
-      "3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]"
+      "3.15.0rc2 (3.15.0~rc2-1.fc44.x86_64, Sep  3 2026, 00:00:00) [GCC 16.2.1 20260819 (Red Hat 16.2.1-2)]"
 
    .. index:: single: version (in module sys)
 

Doc/howto/abi3t-migration.rst

@@ -217,9 +217,9 @@ Module export hook
 
 Unless you've done this step already, your extension module defines a
 :ref:`module initialization function <extension-pyinit>`
-named :samp:`PyInit_{<module_name>}`.
+named :samp:`PyInit_{<modname>}` (where ``modname`` is the name of your module).
 You will need to port it to a :ref:`module export hook <extension-export-hook>`,
-:samp:`PyModExport_{<module name>}`, a feature added in CPython 3.15 in
+:samp:`PyModExport_{<modname>}`, a feature added in CPython 3.15 in
 :pep:`793`.
 
 Your existing init function should look like this (with your own names
@@ -297,6 +297,34 @@ pointer to static data.
 If you cannot avoid additional code, refer to the
 :ref:`caveats in PyModExport documentation <pymodexport-api-caveats>`.
 
+.. note::
+
+   When building for Windows using the Setuptools_ build tool,
+   removing the :samp:`PyInit_{<modname>}` function may result in the linker error
+   :samp:`LINK : error LNK2001: unresolved external symbol PyInit_{<modname>}`.
+   This is caused by Setuptools passing an ``/EXPORT`` linker flag, which
+   is redundant since Python 3.15 (see :gh:`141671`).
+   A workaround is to add a dummy :samp:`PyInit_{<modname>}` function
+   to your code.
+   Python 3.15+ will never call this function if
+   :samp:`PyModExport_{<modname>}` is present, so it can always fail:
+
+   .. code-block:: c
+
+      // Workaround for https://github.com/pypa/distutils/issues/387
+      PyMODINIT_FUNC
+      PyInit_<modname>(void)
+      {
+          PyErr_SetString(PyExc_SystemError,
+                          "PyInit_* called for module with PyModExport_*");
+          return NULL;
+      }
+
+   (This issue is present in Setuptools 84.0.0; it might be fixed in newer
+   versions.)
+
+.. _Setuptools: https://setuptools.pypa.io/
+
 
 Existing slots
 --------------

Doc/library/codecs.rst

@@ -1395,15 +1395,6 @@ encodings.
 |                    |         | :mod:`encodings.idna`.    |
 |                    |         | Only ``errors='strict'``  |
 |                    |         | is supported.             |
-|                    |         |                           |
-|                    |         | .. warning::              |
-|                    |         |                           |
-|                    |         |    This codec builds on   |
-|                    |         |    ``punycode``, whose    |
-|                    |         |    algorithms scale       |
-|                    |         |    poorly, so limit the   |
-|                    |         |    length of untrusted    |
-|                    |         |    input.                 |
 +--------------------+---------+---------------------------+
 | mbcs               | ansi,   | Windows only: Encode the  |
 |                    | dbcs    | operand according to the  |
@@ -1655,11 +1646,6 @@ Applications) and :rfc:`3492` (Nameprep: A Stringprep Profile for
 Internationalized Domain Names (IDN)). It builds upon the ``punycode`` encoding
 and :mod:`stringprep`.
 
-.. warning::
-
-   This module builds on ``punycode``, whose algorithms scale poorly, so limit
-   the length of untrusted input.
-
 If you need the IDNA 2008 standard from :rfc:`5891` and :rfc:`5895`, use the
 third-party :pypi:`idna` module.
 
@@ -1697,11 +1683,31 @@ international domain names, and to unify similar characters. The nameprep
 functions can be used directly if desired.
 
 
-.. function:: nameprep(label)
+.. function:: nameprep(label, *, limit=None)
 
    Return the nameprepped version of *label*. The implementation currently assumes
    query strings, so ``AllowUnassigned`` is true.
 
+   Raise :exc:`UnicodeEncodeError` if the nameprep algorithm emits an error.
+
+   If the *limit* argument is given, it should be set to the maximum size
+   of an encoded A-label (that is, 63 for IDNA).
+   :func:`!nameprep` will raise :exc:`UnicodeEncodeError` if the label is
+   **much** larger than *limit*.
+   Note that this is only a rough check meant to skip expensive processing
+   of extremely large input; the caller should check any exact
+   limits separately.
+
+   .. warning::
+
+      For backwards compatibility, label size is unlimited by default.
+      This may cause issues when processing the result with the
+      ``punycode`` encoding, whose algorithms scale poorly.
+
+   .. versionchanged:: next
+
+      Added the *limit* parameter.
+
 
 .. function:: ToASCII(label)
 

Doc/whatsnew/3.15.rst

@@ -188,10 +188,10 @@ name:
    lazy import json
    lazy from pathlib import Path
 
-   print("Starting up...")  # json and pathlib not loaded yet
+   print("Starting up...")  # json and pathlib are not fully imported yet
 
-   data = json.loads('{"key": "value"}')  # json loads here
-   p = Path(".")  # pathlib loads here
+   data = json.loads('{"key": "value"}')  # json gets fully imported here
+   p = Path(".")  # pathlib gets fully imported here
 
 This mechanism is particularly useful for applications that import many
 modules at the top level but may only use a subset of them in any given run.

Doc/whatsnew/3.16.rst

@@ -397,6 +397,10 @@ encodings
   used for international IMAP4 mailbox names (:rfc:`3501`).
   (Contributed by Serhiy Storchaka in :gh:`66788`.)
 
+* :func:`encodings.idna.nameprep` now takes a *limit* argument that allows
+  rejecting extremely large input early.
+  (Contributed by Petr Viktorin in :gh:`157675`.)
+
 
 gzip
 ----

Include/internal/pycore_dict.h

@@ -23,6 +23,11 @@ PyAPI_FUNC(int) _PyDict_DelItemIf(PyObject *mp, PyObject *key,
                                   int (*predicate)(PyObject *value, void *arg),
                                   void *arg);
 
+// Atomically replace an existing value if it is expected (by identity).
+// Return 1 if replaced, 0 if absent or different, or -1 on error.
+extern int _PyDict_ReplaceItemIf(PyObject *dict, PyObject *key,
+                               PyObject *expected, PyObject *replacement);
+
 // "KnownHash" variants
 // Export for '_asyncio' shared extension
 PyAPI_FUNC(int) _PyDict_SetItem_KnownHash(PyObject *mp, PyObject *key,
@@ -147,8 +152,8 @@ PyAPI_FUNC(Py_ssize_t) _PyDictKeys_StringLookup(PyDictKeysObject* dictkeys, PyOb
  */
 PyAPI_FUNC(Py_ssize_t) _PyDictKeys_StringLookupAndVersion(PyDictKeysObject* dictkeys, PyObject *key, uint32_t *version);
 PyAPI_FUNC(Py_ssize_t) _PyDictKeys_StringLookupSplit(PyDictKeysObject* dictkeys, PyObject *key);
-PyAPI_FUNC(PyObject *)_PyDict_LoadGlobal(PyDictObject *, PyDictObject *, PyObject *);
-PyAPI_FUNC(void) _PyDict_LoadGlobalStackRef(PyDictObject *, PyDictObject *, PyObject *, _PyStackRef *);
+// Return the borrowed source dictionary, or NULL if absent or on error.
+PyAPI_FUNC(PyObject *) _PyDict_LoadGlobalStackRef(PyDictObject *, PyDictObject *, PyObject *, _PyStackRef *);
 
 // Loads the __builtins__ object from the globals dict. Returns a new reference.
 extern PyObject *_PyDict_LoadBuiltinsFromGlobals(PyObject *globals);

Include/internal/pycore_getopt.h

@@ -5,18 +5,21 @@
 #  error "this header requires Py_BUILD_CORE define"
 #endif
 
-extern int _PyOS_opterr;
-extern Py_ssize_t _PyOS_optind;
-extern const wchar_t *_PyOS_optarg;
+struct _PyOS_GetOpt {
+    int error;                    // generate error messages
+    Py_ssize_t index;             // index into argv array
+    const wchar_t *arg;           // optional argument
+    const wchar_t *ptr;
+    Py_ssize_t argc;
+    wchar_t * const *argv;
+};
 
-extern void _PyOS_ResetGetOpt(void);
+extern void _PyOS_GetOpt_Init(
+    struct _PyOS_GetOpt *getopt,
+    Py_ssize_t argc,
+    wchar_t * const *argv);
 
-typedef struct {
-    const wchar_t *name;
-    int has_arg;
-    int val;
-} _PyOS_LongOption;
-
-extern int _PyOS_GetOpt(Py_ssize_t argc, wchar_t * const *argv, int *longindex);
+extern int _PyOS_GetOpt(
+    struct _PyOS_GetOpt *getopt);
 
 #endif /* !Py_INTERNAL_PYGETOPT_H */

Include/internal/pycore_import.h

@@ -32,20 +32,18 @@ extern int _PyImport_FixupBuiltin(
     PyObject *modules
     );
 
-extern PyObject * _PyImport_ResolveName(
-    PyThreadState *tstate, PyObject *name, PyObject *globals, int level);
 extern PyObject * _PyImport_GetAbsName(
     PyThreadState *tstate, PyObject *name, PyObject *globals, int level);
 // Symbol is exported for the JIT on Windows builds.
 PyAPI_FUNC(PyObject *) _PyImport_LoadLazyImportTstate(
     PyThreadState *tstate, PyObject *lazy_import);
-typedef enum {
-    _Py_LAZY_SUBMODULE_ERROR = -1,
-    _Py_LAZY_SUBMODULE_NOT_FOUND = 0,
-    _Py_LAZY_SUBMODULE_LOADED = 1,
-} _PyLazySubmoduleImportResult;
-extern _PyLazySubmoduleImportResult _PyImport_TryLoadLazySubmodule(
-    PyObject *mod_name, PyObject *attr_name, PyObject **result);
+// Bind a pending child on module before removing its registration. Return a
+// new reference, NULL without an exception if absent, or NULL on error.
+// Set recheck_dict if a concurrent lookup may have already bound the child.
+extern PyObject * _PyImport_TryLoadLazySubmodule(
+    PyObject *module, PyObject *attr_name, int suppress, int *recheck_dict);
+extern int _PyImport_ClearLazySubmodule(
+    PyThreadState *tstate, PyObject *name, int bind);
 extern PyObject * _PyImport_LazyImportModuleLevelObject(
     PyThreadState *tstate, PyObject *name, PyObject *builtins,
     PyObject *globals, PyObject *locals, PyObject *fromlist, int level);
@@ -91,6 +89,8 @@ extern void _PyImport_ClearModulesByIndex(PyInterpreterState *interp);
 extern PyObject * _PyImport_InitLazyModules(
     PyInterpreterState *interp);
 extern void _PyImport_ClearLazyModules(PyInterpreterState *interp);
+extern int _PyImport_DiscardLazyModule(
+    PyInterpreterState *interp, PyObject *name);
 
 extern int _PyImport_InitDefaultImportFunc(PyInterpreterState *interp);
 extern int _PyImport_IsDefaultImportFunc(

Include/internal/pycore_interp_structs.h

@@ -352,16 +352,15 @@ struct _import_state {
     PyObject *lazy_import_func;
     int lazy_imports_mode;
     PyObject *lazy_imports_filter;
-    PyObject *lazy_importing_modules;
     // The set stored in sys.lazy_modules if values that have been
     // lazily imported. This value is only for debugging/introspection
     // purposes and is not used by the runtime.
     PyObject *lazy_modules;
-    // A dict mapping package names to a set of submodule names that
-    // have been imported lazily from packages which have been imported
-    // lazily. When the package is reified we need to add a
-    // LazyImportObject which refers to the submodule on the module.
+    // Package names map to pending children: declarations for plain imports,
+    // or None for from-import names that may be ordinary attributes.
     PyObject *lazy_pending_submodules;
+    // Avoid pending-child work for ordinary cached imports.
+    int has_lazy_submodules;
 #ifdef Py_GIL_DISABLED
     PyMutex lazy_mutex;
 #endif