Skip to content

chore(amber): remove the deprecated ExpansionGreedyScheduleGenerator - #7446

Open
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:chore/remove-expansion-greedy-scheduler
Open

chore(amber): remove the deprecated ExpansionGreedyScheduleGenerator#7446
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:chore/remove-expansion-greedy-scheduler

Conversation

@aglinxinyuan

@aglinxinyuan aglinxinyuan commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Acts on the removal notice ExpansionGreedyScheduleGenerator has carried since #3542. Pure deletion of the class and its spec, no behaviour change: −832 lines.

#3144 ("Refactoring of Schedule Generation", 2024-12-14) made CostBasedScheduleGenerator the only generator the engine constructs. #3542 (2025-07-09) then annotated the greedy one with the notice it still carries today:

This greedy schedule generator will be removed in the future. Use CostBasedScheduleGenerator instead.

WorkflowScheduler.scala:46 -> new CostBasedScheduleGenerator(...)   (live, untouched)
                              ExpansionGreedyScheduleGenerator      (unreachable since #3144)

No configuration can bring it back: the schedule-generator block in application.conf holds only CostBased tuning parameters (max-concurrent-regions, use-global-search, use-top-down-search, search-timeout, read at ApplicationConfig.scala:87-90), with no generator-selection key.

Reviewer note: the abstract base ScheduleGenerator is not touched — CostBasedScheduleGenerator extends it and is unaffected. Only the greedy subclass and its spec are removed.

Any related issues, documentation, discussions?

Closes #7444

How was this PR tested?

Existing tests only — this PR adds none, since it removes code and the spec that covered it.

Locally, from the repo root with Java 17:

  • sbt "WorkflowExecutionService/Test/compile" — success (main and test sources).

Verification that nothing references the removed class, re-runnable by a reviewer:

git grep -n ExpansionGreedy          # only the two deleted files
git grep -in greedy -- '*.conf' '*.yml' '*.yaml' '*.json' '*.properties'   # no generator-selection key

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

Copilot AI lite review requested due to automatic review settings August 9, 2026 11:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main e03d971 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 373 0.228 24,280/37,959/37,959 us 🔴 +24.8% / 🔴 +135.5%
🔴 bs=100 sw=10 sl=64 870 0.531 108,338/171,711/171,711 us 🔴 +11.4% / 🔴 +54.4%
🔴 bs=1000 sw=10 sl=64 1,092 0.666 903,921/1,058,331/1,058,331 us 🔴 +8.2% / 🟢 -11.8%
Baseline details

Latest main e03d971 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 373 tuples/sec 437 tuples/sec 752.25 tuples/sec -14.6% -50.4%
bs=10 sw=10 sl=64 MB/s 0.228 MB/s 0.267 MB/s 0.459 MB/s -14.6% -50.3%
bs=10 sw=10 sl=64 p50 24,280 us 23,798 us 13,006 us +2.0% +86.7%
bs=10 sw=10 sl=64 p95 37,959 us 30,418 us 16,117 us +24.8% +135.5%
bs=10 sw=10 sl=64 p99 37,959 us 30,418 us 19,286 us +24.8% +96.8%
bs=100 sw=10 sl=64 throughput 870 tuples/sec 873 tuples/sec 953.66 tuples/sec -0.3% -8.8%
bs=100 sw=10 sl=64 MB/s 0.531 MB/s 0.533 MB/s 0.582 MB/s -0.4% -8.8%
bs=100 sw=10 sl=64 p50 108,338 us 108,556 us 105,130 us -0.2% +3.1%
bs=100 sw=10 sl=64 p95 171,711 us 154,206 us 111,226 us +11.4% +54.4%
bs=100 sw=10 sl=64 p99 171,711 us 154,206 us 121,065 us +11.4% +41.8%
bs=1000 sw=10 sl=64 throughput 1,092 tuples/sec 1,119 tuples/sec 984.25 tuples/sec -2.4% +10.9%
bs=1000 sw=10 sl=64 MB/s 0.666 MB/s 0.683 MB/s 0.601 MB/s -2.5% +10.9%
bs=1000 sw=10 sl=64 p50 903,921 us 893,768 us 1,024,295 us +1.1% -11.8%
bs=1000 sw=10 sl=64 p95 1,058,331 us 977,675 us 1,062,710 us +8.2% -0.4%
bs=1000 sw=10 sl=64 p99 1,058,331 us 977,675 us 1,095,400 us +8.2% -3.4%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,535.90,200,128000,373,0.228,24280.08,37959.02,37959.02
1,100,10,64,20,2299.68,2000,1280000,870,0.531,108338.34,171711.33,171711.33
2,1000,10,64,20,18316.44,20000,12800000,1092,0.666,903921.46,1058330.63,1058330.63

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.31%. Comparing base (fb1a5c4) to head (e884e48).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7446      +/-   ##
============================================
- Coverage     85.33%   85.31%   -0.03%     
+ Complexity     4153     4130      -23     
============================================
  Files          1169     1168       -1     
  Lines         46740    46657      -83     
  Branches       5202     5198       -4     
============================================
- Hits          39887    39806      -81     
+ Misses         5145     5143       -2     
  Partials       1708     1708              
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø) Carriedforward from fb1a5c4
agent-service 85.50% <ø> (ø) Carriedforward from fb1a5c4
amber 80.82% <ø> (-0.09%) ⬇️
computing-unit-managing-service 50.72% <ø> (ø) Carriedforward from fb1a5c4
config-service 65.97% <ø> (ø) Carriedforward from fb1a5c4
file-service 69.05% <ø> (ø) Carriedforward from fb1a5c4
frontend 87.85% <ø> (ø) Carriedforward from fb1a5c4
notebook-migration-service 78.89% <ø> (ø) Carriedforward from fb1a5c4
pyamber 97.55% <ø> (ø) Carriedforward from fb1a5c4
workflow-compiling-service 26.31% <ø> (ø) Carriedforward from fb1a5c4

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the deprecated ExpansionGreedyScheduleGenerator from the amber engine

3 participants