Skip to content

chore(amber): remove the deprecated project feature and its tables - #7464

Open
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:chore/remove-project-backend
Open

chore(amber): remove the deprecated project feature and its tables#7464
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:chore/remove-project-backend

Conversation

@aglinxinyuan

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Removes the deprecated project feature from the backend and drops its four tables, completing the removal agreed in #5172.

8 files deleted, 18 edited (+66 / −1955).

Area What went
REST ProjectResource, PublicProjectResource, ProjectAccessResource + their three jersey.register lines in TexeraWebApplication
Search ProjectSearchQueryBuilder, PROJECT_RESOURCE_TYPE, both project arms of DashboardResource, and 4 of the 24 UnifiedResourceSchema slots
Workflow coupling project association on create/duplicate/list, DashboardWorkflow.projectIDs, WorkflowIDs.pid, Record10Record9
Config gui.tabs.projects_enabled / GUI_TABS_PROJECTS_ENABLED
Schema 4 tables, plus sql/updates/32.sql and its changelog entry
public_project ──────┐
project_user_access ─┼──> project        all four dropped
workflow_of_project ─┘

privilege_enum stays — it is shared with workflow_user_access, dataset_user_access and computing_unit_user_access.

Two behaviour changes reviewers should look at. Both are non-project code paths that read the project tables today:

Site Before After
WorkflowAccessResource.getPrivilege falls back to PROJECT_USER_ACCESS when there is no direct grant only a direct WORKFLOW_USER_ACCESS grant counts
WorkflowSearchQueryBuilder private-access condition WORKFLOW_USER_ACCESS.UID = uid OR PROJECT_USER_ACCESS.UID IS NOT NULL WORKFLOW_USER_ACCESS.UID = uid
Before:  user ─▸ project share ─▸ workflow      (readable, appears in search)
After:   user ─X─ workflow                      (inaccessible, absent from search)

A workflow reachable only through a project share becomes inaccessible to that user. That is the intended consequence of dropping the feature — notebook-migration-service already documents exactly these post-removal semantics — but it is called out in 32.sql so operators can copy those grants into workflow_user_access before migrating.

jOOQ. Generated sources are not committed (common/dao/.gitignore); sbt jooqGenerate reflects the live database. So the DDL change and the Scala change have to land together, and the migration must be applied before regenerating — otherwise the old classes reappear and stale references still compile.

sql/updates/02.sql is left untouched: it is already applied, and editing it would break its liquibase checksum. bin/k8s/files/texera_ddl.sql is a symlink to sql/texera_ddl.sql, so the helm chart is covered automatically.

Any related issues, documentation, discussions?

Closes #7461

Removal agreed in #5172; the frontend half is #7460. The two are independent and can merge in either order.

How was this PR tested?

Existing tests. The /search API suite in WorkflowResourceSpec seeded projects and asserted counts that included them, so those eight tests were rewritten workflow-only with adjusted counts; UnifiedResourceSchemaSpec's pinned projection arity went 24 → 20.

sbt "WorkflowExecutionService/Test/compile" "ConfigService/Test/compile" "DAO/Test/compile" "NotebookMigrationService/Test/compile"
sbt scalafmtCheckAll "scalafixAll --check"
sbt "WorkflowExecutionService/testOnly *WorkflowResourceSpec *UnifiedResourceSchemaSpec *WorkflowSearchQueryBuilderSpec *HubResourceSpec *DashboardResourceSpec *WorkflowAccessResourceSpec" "ConfigService/testOnly *ConfigResourceSpec *DefaultsConfigSpec" "DAO/test"
Check Result
compile (amber, config-service, dao, notebook-migration) clean
scalafmtCheckAll + scalafixAll --check clean
specs 9 suites, 197 tests passed, 0 failed

The spec run is the meaningful check on the schema change: MockTexeraDB bootstraps embedded Postgres from the edited texera_ddl.sql, so those 197 tests all ran against a database with no project tables. Any surviving query against a dropped table would have failed there. MockTexeraDB's hard-coded fulltext-index block was updated in lockstep — it referenced ON project and would otherwise take out every DAO-backed suite in the repo.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

Copilot AI lite review requested due to automatic review settings August 9, 2026 13:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added engine ddl-change Changes to the TexeraDB DDL common platform Non-amber Scala service paths labels Aug 9, 2026
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @xuang7, @VuMartin, @Yicong-Huang
    You can notify them by mentioning @xuang7, @VuMartin, @Yicong-Huang in a comment.

@codecov-commenter

codecov-commenter commented Aug 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.30%. Comparing base (e03d971) to head (d8bf888).

Files with missing lines Patch % Lines
...rce/dashboard/user/workflow/WorkflowResource.scala 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7464      +/-   ##
============================================
- Coverage     85.32%   85.30%   -0.03%     
+ Complexity     4151     4112      -39     
============================================
  Files          1169     1165       -4     
  Lines         46740    46512     -228     
  Branches       5202     5188      -14     
============================================
- Hits          39882    39675     -207     
+ Misses         5148     5135      -13     
+ Partials       1710     1702       -8     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø)
agent-service 85.50% <ø> (ø) Carriedforward from e03d971
amber 80.75% <75.00%> (-0.14%) ⬇️
computing-unit-managing-service 50.72% <ø> (ø)
config-service 65.97% <ø> (ø)
file-service 69.05% <ø> (ø)
frontend 87.85% <ø> (ø) Carriedforward from e03d971
notebook-migration-service 78.89% <ø> (ø)
pyamber 97.51% <ø> (ø) Carriedforward from e03d971
workflow-compiling-service 26.31% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 15 worse · ⚪ 0 noise (<±5%) · 0 without baseline

CI benchmark results are noisy; treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 424 0.259 23,394/28,363/28,363 us 🔴 +90.9% / 🔴 +78.2%
🔴 bs=100 sw=10 sl=64 821 0.501 120,649/150,448/150,448 us 🔴 +50.4% / 🔴 +33.7%
🔴 bs=1000 sw=10 sl=64 950 0.58 1,055,127/1,087,449/1,087,449 us 🔴 +14.7% / ⚪ within ±5%
Baseline details

Latest main e03d971 from 2026-08-09T12:55:54.590Z

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 424 tuples/sec 800.61 tuples/sec 743.73 tuples/sec -47.0% -43.0%
bs=10 sw=10 sl=64 MB/s 0.259 MB/s 0.489 MB/s 0.454 MB/s -47.0% -42.9%
bs=10 sw=10 sl=64 p50 23,394 us 12,254 us 13,130 us +90.9% +78.2%
bs=10 sw=10 sl=64 p95 28,363 us 15,377 us 16,391 us +84.5% +73.0%
bs=10 sw=10 sl=64 p99 28,363 us 17,693 us 19,408 us +60.3% +46.1%
bs=100 sw=10 sl=64 throughput 821 tuples/sec 1,069 tuples/sec 944.04 tuples/sec -23.2% -13.0%
bs=100 sw=10 sl=64 MB/s 0.501 MB/s 0.652 MB/s 0.576 MB/s -23.2% -13.1%
bs=100 sw=10 sl=64 p50 120,649 us 92,789 us 105,853 us +30.0% +14.0%
bs=100 sw=10 sl=64 p95 150,448 us 100,036 us 112,493 us +50.4% +33.7%
bs=100 sw=10 sl=64 p99 150,448 us 106,858 us 122,200 us +40.8% +23.1%
bs=1000 sw=10 sl=64 throughput 950 tuples/sec 1,087 tuples/sec 972.22 tuples/sec -12.6% -2.3%
bs=1000 sw=10 sl=64 MB/s 0.58 MB/s 0.663 MB/s 0.593 MB/s -12.6% -2.3%
bs=1000 sw=10 sl=64 p50 1,055,127 us 919,503 us 1,033,856 us +14.7% +2.1%
bs=1000 sw=10 sl=64 p95 1,087,449 us 970,060 us 1,076,083 us +12.1% +1.1%
bs=1000 sw=10 sl=64 p99 1,087,449 us 1,014,468 us 1,107,701 us +7.2% -1.8%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,471.55,200,128000,424,0.259,23393.80,28363.47,28363.47
1,100,10,64,20,2435.12,2000,1280000,821,0.501,120649.21,150448.21,150448.21
2,1000,10,64,20,21060.56,20000,12800000,950,0.580,1055127.36,1087448.54,1087448.54

@Yicong-Huang
Yicong-Huang self-requested a review August 9, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common ddl-change Changes to the TexeraDB DDL engine platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the deprecated project feature from the backend and database schema

3 participants