Skip to content

fix: avoid global SQL stack guard mutation in unparser#23284

Merged
xudong963 merged 4 commits into
apache:mainfrom
ametel01:fix-stackguard-global-race
Jul 5, 2026
Merged

fix: avoid global SQL stack guard mutation in unparser#23284
xudong963 merged 4 commits into
apache:mainfrom
ametel01:fix-stackguard-global-race

Conversation

@ametel01

@ametel01 ametel01 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

datafusion-sql used StackGuard to temporarily raise recursive's process-global minimum stack size while planning or unparsing deep SQL structures. Because that global is shared across threads, one guard could restore a lower value while another thread was still relying on DataFusion's larger SQL red zone.

What changes are included in this PR?

This PR replaces the scoped global mutation with local stacker::maybe_grow checkpoints for the SQL planner/unparser recursion paths that need a 256 KiB red zone. The public planning and unparsing APIs are unchanged.

It also adds regressions covering both non-mutating stack growth and deep unparsing while another thread repeatedly lowers recursive's global minimum to the default value.

Are these changes tested?

Yes.

  • cargo test -p datafusion-sql --features recursive_protection
  • cargo fmt --all --check
  • cargo clippy -p datafusion-sql --all-targets --all-features -- -D warnings
  • cargo clippy --all-targets --all-features -- -D warnings

Are there any user-facing changes?

No.

@github-actions github-actions Bot added the sql SQL Planner label Jul 1, 2026
@ametel01 ametel01 marked this pull request as ready for review July 1, 2026 21:25
@alamb alamb changed the title fix: avoid global SQL stack guard mutation fix: avoid global SQL stack guard mutation in unparser Jul 3, 2026
@alamb

alamb commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

@phillipleblanc do you have time to help us review this PR?

@alamb

alamb commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

@mdashti FYI

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some test that fails without this PR and passes with it?

@ametel01

ametel01 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Is there some test that fails without this PR and passes with it?

I updated the regression test so it now fails on main and passes with this PR.

Specifically, test_expr_to_sql_does_not_mutate_recursive_minimum_stack_size observes recursive::get_minimum_stack_size() from inside Unparser::expr_to_sql. On main with only this test applied, it fails because the old StackGuard changes the process-global minimum from 128 KiB to 256 KiB:

left: 262144
right: 131072

With this PR, the test passes because the unparser uses local stacker::maybe_grow boundaries and leaves the global recursive minimum unchanged.

I also ran:

  • cargo test -p datafusion-sql
  • cargo test -p datafusion-sql --features recursive_protection
  • ./dev/rust_lint.sh

@phillipleblanc

Copy link
Copy Markdown
Contributor

I will review this later today

@phillipleblanc phillipleblanc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially was looking at why we were depending on a new crate: stacker - but this is already an indirect dependency of recursive which is a dependency datafusion-sql already has when recursive_protection is enabled.

The old StackGuard struct was inherently racy across threads because recursive::set_minimum_stack_size is a process global (eww). Replacing the global mutation with scoped increases seems correct to me.

@xudong963 xudong963 added this pull request to the merge queue Jul 5, 2026
Merged via the queue into apache:main with commit bba2f4b Jul 5, 2026
39 checks passed
@alamb

alamb commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Thanks @xudong963 and @phillipleblanc and @ametel01

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StackGuard races recursive's process-global stack-size minimum across threads

4 participants