Conversation
…routines The sampling profiler's exception mode decided whether a thread was handling an exception by reading the embedded PyThreadState.exc_state. Generators, coroutines and async generators repoint tstate->exc_info at their own _PyErr_StackItem while they run, so an except block running in one of them (or in a function they call) stored the exception in that item instead, and was never sampled. Follow tstate->exc_info and its previous_item chain, mirroring _PyErr_GetTopmostException(), and export the two debug offsets needed to walk the chain from remote memory. The common case where exc_info points at the embedded exc_state keeps the existing zero-extra-read fast path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #158539.
Problem
Exception mode (
--mode=exception) decides whether a thread is handling anexception from
THREAD_STATUS_HAS_EXCEPTION, which was computed by reading theexc_stateembedded inPyThreadState. Generators, coroutines and asyncgenerators repoint
tstate->exc_infoat their own_PyErr_StackItemwhilethey run (
gen_send_ex2()), so anexceptblock running in one of them — orin a function they call — stores the exception in that item and was never
sampled.
The practical effect is that exception mode reports almost nothing for asyncio
programs: every
exceptruns in a coroutine, or in something called from one.The run still reports a 0.00 error rate, so the missing samples are silent.
Fix
Compute the flag the way the interpreter does, by following
tstate->exc_infoand its
previous_itemchain, like_PyErr_GetTopmostException()(the helperbehind
sys.exception()):thread_state.exc_infoanderr_stackitem.previous_itemin_Py_DebugOffsets(and register them with the offsets validation in_remote_debugging);exc_infofrom the thread state buffer. When it points at the embeddedexc_state— the common case, e.g. a thread-level handler — keep theexisting zero-extra-read fast path. Otherwise treat it as a remote
_PyErr_StackItemand walk the chain until a non-NULLexc_valueis found.A failed remote read is best-effort: it clears the error and reports "no
exception", rather than turning one unreadable stack item into a failed
get_stack_trace()call.Tests
Lib/test/test_external_inspection.py:TestExceptionDetectionScenarioscovering a handler ina generator, in a function called from a generator expression, in a
coroutine, and in a function called from a coroutine;
TestExceptionDetectionInProcessclass that inspects the currentprocess, so it does not need subprocess debugging permissions (macOS) and can
run on the buildbots. Besides the positive cases it covers the exception-mode
thread filter and two negative cases (a generator with no exception, and a
generator's
finallyafter a handled exception).All four positive cases fail without the fix and pass with it;
test_profilingand
test_external_inspectionpass with it.Notes
AI tooling was used while preparing this change; I have reviewed it in detail
and can explain all of it.