Some problem with recursion handling

[gregorlingl]

BPO	6717
Nosy	@loewis, @terryjreedy, @kbkaiser, @pitrou, @vstinner, @DinoV, @bitdancer, @asvetlov, @serhiy-storchaka, @srinivasreddy, @willingc, @LionKimbro
Files	dragbug.py: script to reproduce FatalError with python 3.1 turtlecrash.py tkinter_recursionbug_31.py: tkinter script producing the crash described here drag_bug_is_nesting_events.py: Demo program showing nesting of event callbacks... test.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2009-08-17.14:15:59.337>
labels = ['expert-tkinter', 'library', 'type-crash']
title = 'Some problem with recursion handling'
updated_at = <Date 2018-11-02.11:09:53.131>
user = 'https://bugs.python.org/gregorlingl'

bugs.python.org fields:

activity = <Date 2018-11-02.11:09:53.131>
actor = 'thatiparthy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Tkinter']
creation = <Date 2009-08-17.14:15:59.337>
creator = 'gregorlingl'
dependencies = []
files = ['14738', '15063', '15149', '22073', '22365']
hgrepos = []
issue_num = 6717
keywords = []
message_count = 21.0
messages = ['91664', '91665', '93676', '94148', '136587', '136590', '136592', '137089', '138347', '158144', '158145', '158146', '158148', '158149', '158150', '158151', '158157', '174892', '319741', '319764', '328663']
nosy_count = 17.0
nosy_names = ['loewis', 'terry.reedy', 'kbk', 'pitrou', 'vstinner', 'gregorlingl', 'dino.viehland', 'r.david.murray', 'asvetlov', 'flupke', 'python-dev', 'Peter.Wentworth', 'Emmanuel.Decitre', 'serhiy.storchaka', 'thatiparthy', 'willingc', 'LionKimbro']
pr_nums = []
priority = 'high'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue6717'
versions = ['Python 3.6']

Linked PRs

• gh-50966: Fix unbounded recursion in turtle drag handlers #152626
• [3.15] gh-50966: Fix unbounded recursion in turtle drag handlers (GH-152626) #152657
• [3.14] gh-50966: Fix unbounded recursion in turtle drag handlers (GH-152626) #152658
• [3.13] gh-50966: Fix unbounded recursion in turtle drag handlers (GH-152626) #152667

[gregorlingl]

I suspect that there is some deeper (more severe) issue behind the
problem I describe below. I've observed the following on Windows only.
Didn't try it with different OSs

running dragbug.py shows different behaviour with Python 3.1 compared to
Python 2.6:

Running it with Python 3.1 and performing heavy turtle dragging with the
mouse results in:

Fatal Python error: Cannot recover from stack overflow
This application has requested the Runtime to terminate it in an unusual
way. Please contact the application's support team
-- As I do at the moment ;-)

Running the same script from Python 2.6: The error is much harder to
reproduce (only with very excessive dragging). It's a bit easier to
reproduce when running the program from a console and if it occurs one
gets a conventional Python error message:

First a long sequence of:
Exception in Tkinter callback
Traceback (most recent call last):
Exception RuntimeError: 'maximum recursion depth exceeded while .... 
ignored

Followed by:

Traceback (most recent call last):
  File "dragbug.py", line 10, in <module>
    mainloop()
  ....
  File "c:\Python26\lib\traceback.py", line 57, in print_tb
    if hasattr(sys, 'tracebacklimit'):
AttributeError: 'module' object has no attribute 'tracebacklimit'

This problem can be overcome by setting a higher recurson limit. But I
think one should require that it doesn't produce a Fatal Error like in
Python 3.1

Regards,
Gregor

[bitdancer]

With py3k trunk on Gentoo Linux after less than a second of flailing the
cursor around on the screen I got:

rdmurray@partner:~/python/py3k>./python dragbug.py
Fatal Python error: Cannot recover from stack overflow.
zsh: abort ./python dragbug.py

[terryjreedy]

For anyone trying this on another system: you must specifically click on
and drag the mouse/turtle graphic.

Running from IDLE edit, I get no traceback in the Shell Window, unlike
with the script I attached to bpo-7074 and now here. While I am closing the
latter as probably due to the same cause, it does act differently:
adding the 'speed(0)' command present in dragbug.py appears to have
fixed the problem with turtlecrash.py.

Rewriting dragbug to use tkinter canvas directly would determine whether
the turtle module is a factor. The change of behavior with turtlecrash
with the speed command suggests to me that it might be.

[gregorlingl]

I've written and appended with this message a small script using
tkinter, that produces the same crash as described earlier with turtle.py:

tkinter_recursion_31.py

It occurs when heavily and fast dragging the red square. So the bug
doesn't depent on turtle.py

I'd like to mention again, that the corresponding script under Python26
does crash also, but with a regular Python Error-Message. It does not
kill IDLE when stated from within IDLE.

So I'd suggest to give this one a higher priority. Should be fixed with 3.2

Regards,
Gregor

[PeterWentworth]

I can confirm the crash persists as of Python 3.1.3 on Windows, and would like to add my vote to prioritizing it.

Without having delved into the code, it seems strange that the rapid stream of events is causing stack overflow / recursion limit problems.

It suggests that tkinter is allowing new event arrivals to interrupt older event handling, so that the older ones remain incomplete as the newer ones pile up on the stack. If this is the case, it is going to lead to long-term instability, and needs attention.

The usual technique is to use a queue to decouple "occurs now" from "handle immediately". (The OS puts the mouse and keyboard events onto window's event queue). So the mainloop that services the event queue should not start handling a new queued event until the previous handling has completed.

[PeterWentworth]

Attached is a crashing program that shows that the event handler is called again before activation of the prior instance has completed.

I also have a second turtle that queues the nested events. It doesn't crash the system, at least.

Perhaps someone can also explain why the handler doesn't work if I leave out the global declaration.

[PeterWentworth]

Oops, I wish I hadn't asked that silly question about the global declaration!

Here is the tweaked file...

[EmmanuelDecitre]

Issue reproduceable on 3.2 (r32:88445) with drag_bug_is_nesting_events.py

[DinoV]

I ran into a similar issue and believe I have a simpler repro. I think the issue here might be that when you take a stack overflow inside of a sys.settrace handler that you can then later violate the stack depth. The attached test.py has no dependencies and crashes 3.x but works on 2.x.

[pitrou]

Yes, this is all by design. The interpreter *has* to stop: either it stops in a controlled way (the fatal error) or the stack is blown and it crashes. If you think the fatal error (basically a C abort() call) should be replaced with another way of exiting, please suggest so.

(actually, it's not the interpreter as a whole, only the current thread, but crashing a thread without affecting the others is probably impossible, due to reference leaks, resource cleanup, etc.)

[DinoV]

Antoine: If you're looking at my test.py then my expectation is that this doesn't crash because a RuntimeError should be raised when the maximum recursion limit is hit, and then the trace handler should be uninstalled because it leaks an exception. And that's exactly what seems to happens on Python 2.x. We shouldn't ever hit the OS stack limit because Python's recursion limit should be enforced even in the face of a sys.settrace handler.

[pitrou]

Antoine: If you're looking at my test.py then my expectation is that
this doesn't crash because a RuntimeError should be raised when the
maximum recursion limit is hit, and then the trace handler should be
uninstalled because it leaks an exception.

I don't understand why you say that "the trace handler leaks an
exception", since it silences it in the "try ... except" block.