fix(amber): skip statistics queries to workers of a terminating region

[Neilk1021]

What changes were proposed in this PR?

Closes #7031 by keeping the coordinator from sending queryStatistics to workers of a region whose teardown it has already started.

• RegionExecutionManager sets an endWorkerSentRef flag as the first statement of terminateWorkers and exposes it as isTerminating, plus a containsPhysicalOp membership test. The flag is deliberately not derived from the existing terminationFutureRef. That reference is compared and swapped only after terminateWorkersWithRetry has been constructed, and constructing it already runs terminateWorkers.
• WorkflowExecutionManager.isRegionTerminating(opId) resolves an operator to a terminating region. Documented as must-be-called at the point of use rather than cached: a caller spanning several coordinator rounds would otherwise act on an answer from before a region started tearing down, which is exactly the window being
  closed.
• QueryWorkerStatisticsHandler consults it alongside the existing filters and skips those operators.

Any related issues, documentation, discussions?

Closes #7031.

Part of #6891. This is the request-side half. #6960 fixed the reply-side half; the two are independent and this PR does not change anything #6960 introduced.

How was this PR tested?

Three unit tests added to the existing harnesses, no new harness:

• WorkflowExecutionManagerSpec: drives a real two-region schedule to the state where region 1's EndWorker is on the wire and unanswered, then asserts isRegionTerminating is true for that region's operator, false for the not-yet-started region, and still true after termination completes. It also asserts the operator aggregates as RUNNING at that moment, which is what pins the point of the fix: the window is real, and the pre-existing COMPLETED skip could not have covered it.
• RegionExecutionManagerSpec: isTerminating is false while the region is merely running and observably true by the time EndWorker is emitted (checked from inside the RPC probe's callback, not merely afterwards); and a region recognises only its own operators.

sbt "WorkflowExecutionService/testOnly *RegionExecutionManagerSpec *WorkflowExecutionManagerSpec *PortCompletedHandlerSpec *EndHandlerSpec"                          

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

[github-actions]

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch

Analysis

⚠️ release/v1.2

Not labeled automatically — none of the files this PR modifies exist on this branch (amber/src/main/scala/org/apache/texera/amber/engine/architecture/coordinator/promisehandlers/QueryWorkerStatisticsHandler.scala, amber/src/main/scala/org/apache/texera/amber/engine/architecture/scheduling/RegionExecutionManager.scala, amber/src/main/scala/org/apache/texera/amber/engine/architecture/scheduling/WorkflowExecutionManager.scala, amber/src/test/scala/org/apache/texera/amber/engine/architecture/scheduling/RegionExecutionManagerSpec.scala, amber/src/test/scala/org/apache/texera/amber/engine/architecture/scheduling/WorkflowExecutionManagerSpec.scala). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. Please check and add release/v1.2 by hand if this fix should be backported here.

Auto-label run.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aglinxinyuan, @Yicong-Huang
  You can notify them by mentioning @aglinxinyuan, @Yicong-Huang in a comment.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 5 worse · ⚪ 10 noise (<±5%) · 0 without baseline

Compared against main 5e41efa benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	552	0.337	17,392/27,061/27,061 us	🔴 +16.8% / 🔴 +67.4%
⚪	bs=100 sw=10 sl=64	1,223	0.747	80,816/97,270/97,270 us	⚪ within ±5% / 🟢 +27.8%
⚪	bs=1000 sw=10 sl=64	1,439	0.878	683,820/825,140/825,140 us	⚪ within ±5% / 🟢 +45.8%

Baseline details

Latest main 5e41efa from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	552 tuples/sec	587 tuples/sec	755.31 tuples/sec	-6.0%	-26.9%
bs=10 sw=10 sl=64	MB/s	0.337 MB/s	0.359 MB/s	0.461 MB/s	-6.1%	-26.9%
bs=10 sw=10 sl=64	p50	17,392 us	16,131 us	12,952 us	+7.8%	+34.3%
bs=10 sw=10 sl=64	p95	27,061 us	23,176 us	16,161 us	+16.8%	+67.4%
bs=10 sw=10 sl=64	p99	27,061 us	23,176 us	19,292 us	+16.8%	+40.3%
bs=100 sw=10 sl=64	throughput	1,223 tuples/sec	1,274 tuples/sec	957.58 tuples/sec	-4.0%	+27.7%
bs=100 sw=10 sl=64	MB/s	0.747 MB/s	0.778 MB/s	0.584 MB/s	-4.0%	+27.8%
bs=100 sw=10 sl=64	p50	80,816 us	77,885 us	104,473 us	+3.8%	-22.6%
bs=100 sw=10 sl=64	p95	97,270 us	95,626 us	110,867 us	+1.7%	-12.3%
bs=100 sw=10 sl=64	p99	97,270 us	95,626 us	120,336 us	+1.7%	-19.2%
bs=1000 sw=10 sl=64	throughput	1,439 tuples/sec	1,427 tuples/sec	986.93 tuples/sec	+0.8%	+45.8%
bs=1000 sw=10 sl=64	MB/s	0.878 MB/s	0.871 MB/s	0.602 MB/s	+0.8%	+45.8%
bs=1000 sw=10 sl=64	p50	683,820 us	690,251 us	1,021,008 us	-0.9%	-33.0%
bs=1000 sw=10 sl=64	p95	825,140 us	786,502 us	1,059,187 us	+4.9%	-22.1%
bs=1000 sw=10 sl=64	p99	825,140 us	786,502 us	1,093,309 us	+4.9%	-24.5%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,362.06,200,128000,552,0.337,17392.35,27061.03,27061.03
1,100,10,64,20,1634.83,2000,1280000,1223,0.747,80816.22,97270.38,97270.38
2,1000,10,64,20,13902.60,20000,12800000,1439,0.878,683819.55,825139.75,825139.75

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.70%. Comparing base (5e41efa) to head (3f59df6).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7391      +/-   ##
============================================
- Coverage     84.70%   84.70%   -0.01%     
- Complexity     4153     4155       +2     
============================================
  Files          1169     1169              
  Lines         46740    46745       +5     
  Branches       5202     5202              
============================================
+ Hits          39592    39595       +3     
  Misses         5433     5433              
- Partials       1715     1717       +2     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)		Carriedforward from 5e41efa
agent-service	85.50% <ø> (ø)		Carriedforward from 5e41efa
amber	80.86% <100.00%> (-0.01%)	⬇️
computing-unit-managing-service	50.72% <ø> (ø)		Carriedforward from 5e41efa
config-service	65.97% <ø> (ø)		Carriedforward from 5e41efa
file-service	69.05% <ø> (ø)		Carriedforward from 5e41efa
frontend	86.45% <ø> (ø)		Carriedforward from 5e41efa
notebook-migration-service	78.89% <ø> (ø)		Carriedforward from 5e41efa
pyamber	97.55% <ø> (ø)		Carriedforward from 5e41efa
workflow-compiling-service	26.31% <ø> (ø)		Carriedforward from 5e41efa

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Neilk1021]

/request-review @aglinxinyuan