fix: avoid global SQL stack guard mutation in unparser#23284
Conversation
|
@phillipleblanc do you have time to help us review this PR? |
|
@mdashti FYI |
alamb
left a comment
There was a problem hiding this comment.
Is there some test that fails without this PR and passes with it?
I updated the regression test so it now fails on Specifically, left: 262144 With this PR, the test passes because the unparser uses local I also ran:
|
|
I will review this later today |
phillipleblanc
left a comment
There was a problem hiding this comment.
I initially was looking at why we were depending on a new crate: stacker - but this is already an indirect dependency of recursive which is a dependency datafusion-sql already has when recursive_protection is enabled.
The old StackGuard struct was inherently racy across threads because recursive::set_minimum_stack_size is a process global (eww). Replacing the global mutation with scoped increases seems correct to me.
|
Thanks @xudong963 and @phillipleblanc and @ametel01 |
Which issue does this PR close?
StackGuardracesrecursive's process-global stack-size minimum across threads #23246.Rationale for this change
datafusion-sqlusedStackGuardto temporarily raiserecursive's process-global minimum stack size while planning or unparsing deep SQL structures. Because that global is shared across threads, one guard could restore a lower value while another thread was still relying on DataFusion's larger SQL red zone.What changes are included in this PR?
This PR replaces the scoped global mutation with local
stacker::maybe_growcheckpoints for the SQL planner/unparser recursion paths that need a 256 KiB red zone. The public planning and unparsing APIs are unchanged.It also adds regressions covering both non-mutating stack growth and deep unparsing while another thread repeatedly lowers
recursive's global minimum to the default value.Are these changes tested?
Yes.
cargo test -p datafusion-sql --features recursive_protectioncargo fmt --all --checkcargo clippy -p datafusion-sql --all-targets --all-features -- -D warningscargo clippy --all-targets --all-features -- -D warningsAre there any user-facing changes?
No.