docs: correct which operators run separate native plans in a task

[andygrove]

Which issue does this PR close?

No issue was filed. This is a small documentation correction found while working on #6191.

Rationale for this change

The memory management guide and the tuning guide both say that a native shuffle runs the pre-shuffle operators and the shuffle writer as two native execution contexts in the same task, and use that to explain why the memory pool is shared per task. That stopped being true with #4507, which plans the shuffle writer together with the native operators that feed it.

I checked this by counting the native plans each task created, using a temporary print where CometExecIterator creates its plan, on the default Spark 4.1 build:

Query	Native plans per task
Scan, filter, native shuffle	1
Union of two filtered scans, partial aggregate, native shuffle	2
Filter, coalesce(1), aggregate	6 in the single task: one per coalesced input partition, plus the aggregate
Filter, native Parquet write	2
Filter, then a typed map that Comet does not support, then an aggregate	1, because everything above the map stays in Spark

What changes are included in this PR?

• The memory management guide notes that a native shuffle is a single plan. It then lists the operators that do split a task's native work into separate plans:
  • union and coalesce, which read their children through the JVM
  • collect limit and take-ordered-and-project, which apply their limit in a plan of their own
  • native Parquet and Iceberg writes, which run the writer as a plan of its own
• The tuning guide's description of the shared pool now uses union and coalesce as its example instead of shuffle.

How are these changes tested?

This is a documentation-only change. The counts above come from a temporary instrumented run that is not part of this PR. The collect limit, take-ordered-and-project and Iceberg write cases were checked in the code rather than measured.

[sunchao]

Summary

• Prior state and problem: Both guides incorrectly used native shuffle as an example of separate native plans sharing a task’s memory pool.
• Design approach: Explain shuffle’s fused writer plan and identify operators that introduce separate native plans.
• Correctness / compatibility analysis: Traced shuffle, union, coalesce, limit/top-K, Parquet and Iceberg write paths and task-shared pool creation. Compared relevant Spark sources for versions 3.4.3, 3.5.9, 4.0.4, 4.1.3 and 4.2.0. The revised examples are supported by the implementation.
• Key design decisions: Preserve the explanation of task-wide pool sharing and distinguish native-plan boundaries from Spark-task boundaries.
• Implementation sketch: Expand the contributor guide’s examples and replace the tuning guide’s shuffle example with union and coalesce.
• Behavioral changes worth calling out: This is documentation-only. It adds no runtime overhead or implementation abstractions.
• Suggested improvements: None at P1/P2 severity. No introduced P1/P2 issues found within this review.

Reviewed the entire base-relative diff from 67803a7a422c44de07af1e5d25c1dbeae8df68d4 to full SHA 7cb0006d76923aa8f6d4b21b95f2a6ec97df4cee: one commit affecting two documentation files. The PR remains open and non-draft. Existing reviews, issue comments, inline comments and review threads were empty in both the supplied snapshot and live GitHub.

Routed skills: review-comet-pr, review-comet-memory-pr and review-comet-shuffle-pr.

Exact-head CI: 7 successful checks and 15 skipped, with no failed or unfinished check runs. Preflight, Required Checks, change detection, title/label checks and CodeQL passed. Runtime builds, Spark SQL/Iceberg suites, benchmarks and site deployment were skipped.

Validation limits: Source inspection and git diff --check passed. No local JVM/native tests or documentation build were run. The author’s temporary instrumented measurements were not independently reproduced.