feat: add subscription groups server contract - #76
Conversation
Priveetee
left a comment
There was a problem hiding this comment.
Hey, first thx a lot for taking the time to work on this!
I went through the database changes, routes, account isolation, feed filtering, restore behavior, OpenAPI and tests.
Honestly the base is really good. Groups are correctly isolated per account, one channel can be in multiple groups, the ungrouped feed is handled properly, and reusing the existing subscription feed snapshot makes sense.
I found a few things that need to be fixed before merge:
-
The important one is backups. Groups and their channel memberships are not exported right now. Restoring on a fresh instance would restore the subscriptions, but all the organization created by the user would be lost. Both the groups and their memberships need to be included in TypeType backups.
-
There is also an edge case with pagination. The cursor only remembers the group ID and filter, while the group membership is loaded again for every page. If someone edits the group between two pages, the next page can skip or repeat videos. The cursor needs to keep the same group membership snapshot for the full pagination session.
-
Small OpenAPI mismatch:
subscribedAtis required in the subscription creation schema, but Server actually generates it and ignores the submitted value. It would be cleaner to have a separate request schema where this field is not required.
Could u also split the first commit please? It currently contains 1,174 insertions, and I try to keep commits below 290 insertions so it stays easy to understand later why each part changed. Separating the database/model work, group API, feed filtering, backup support and tests would make the history much easier to follow.
I ran the full validation: 998 tests passed, including OpenAPI validation, coverage and the production jar build. I also checked it together with the current Server changes and all 1,036 tests passed there.
So yeah, no need to rebuild everything from scratch at all. The base is good, it just needs these points fixed and I will gladly review it again :)
Thx again, and have a great day!
Add account-scoped group and membership tables plus the API models used by the group service. Register both tables in production and test database setup.
Create, rename, and remove account-owned groups and assign subscribed channels to multiple groups. Keep memberships consistent when subscriptions are deleted or replaced by imports.
Add authenticated CRUD and membership endpoints backed by the account-scoped group service, and register the service with the application.
Parse group and ungrouped selectors for subscription reads, reject invalid or foreign group IDs, and separate subscription creation input from server-generated timestamps.
Project group and ungrouped feeds from the shared global snapshot while retaining each cursor's account-scoped membership selection in cache for the full pagination session.
Export named groups with their channel memberships and restore them transactionally with subscriptions. Validate names and membership references before replacing account-owned group data.
Describe group management and filtered list/feed operations, and make subscription creation use a request schema without the server-generated subscribedAt field.
Verify account isolation, normalized unique names, many-to-many membership, ungrouped selection, and cleanup after subscription replacement or deletion.
Exercise authenticated group CRUD, membership updates, filter validation, and cross-account access through the HTTP routing surface.
Verify shared-snapshot projection, source-channel attribution, filter-bound cursors, and unchanged membership snapshots across paginated group reads.
Filtered pagination needs a stable membership view without creating an unbounded cache entry for every initial request. Allocate a fixed set of account-scoped slots atomically, reuse content-derived tokens, and reject a new distinct session at capacity without evicting any issued cursor. Strengthen observable compatibility coverage for legacy backups and server timestamps. Constraint: Every issued cursor must retain its membership snapshot for the full cache TTL Rejected: Evict the oldest snapshot after eight sessions | invalidates a still-live cursor Rejected: Store all snapshots in one read-modify-write value | loses concurrent writes across server instances Rejected: Put channel URLs directly in the cursor | produces oversized client-controlled cursor payloads Confidence: high Scope-risk: moderate Reversibility: clean Directive: Never overwrite an occupied selection slot; reject new sessions before weakening issued cursors Tested: Focused feed, concurrent selection-store, subscription route, backup, and OpenAPI tests on JDK 25 Not-tested: Live Dragonfly slot saturation before the final runtime gate
a260906 to
abd1ae9
Compare
Subscription group assignment checked subscription ownership without coordinating with unsubscribe or replacement restores. Take an account-keyed transaction advisory lock across membership assignment and every subscription removal or replacement path so the check and insert cannot straddle a committed deletion. Constraint: Replacement imports must retain memberships whose subscriptions survive the import Rejected: Composite foreign key with cascading deletes | cascade semantics would discard memberships before retained subscriptions are reinserted Confidence: high Scope-risk: narrow Directive: Any new path that removes or replaces an account's subscriptions must acquire SubscriptionMutationLock in the same transaction Tested: Focused PostgreSQL concurrency regression on JDK 25 Not-tested: Full suite and live HTTP concurrency gate run after both fix commits
Subscription and group sections were read in separate transactions, so a membership committed between them could reference a subscription absent from the exported list. Capture subscriptions once and export only group memberships belonging to that captured set, preserving the restore validator's referential invariant. Constraint: Subscription groups remain coupled to the subscriptions backup category Rejected: Add a cross-service export transaction | existing service reads open their own transactions and captured-set filtering is the smaller accepted repair Confidence: high Scope-risk: narrow Directive: Exported group memberships must remain a subset of the subscriptions captured for the same backup Tested: Focused mixed-read export and restore regression on JDK 25 Not-tested: Full suite and live HTTP concurrency gate run after this commit
Summary
Adds the backend contract for named subscription groups as the first Server-only part of TypeType-Video/TypeType#172.
GET /subscriptionsandGET /subscriptions/feedbygroupIdorungrouped=true;subscribedAtgenerated by the server;The branch is rebased onto the current
devbase and the history is split into focused commits, each below the requested 290-insertion limit.Review feedback addressed
Backup round-trip
Backup exports now include subscription groups and their channel memberships. Restore recreates groups with fresh IDs, reconnects memberships by canonical channel URL, and performs the replacement in the same database transaction as subscription restore. Backups created before the new optional
subscriptionGroupsfield remain accepted.Export captures the subscription list once and filters group memberships against that canonical captured set. A concurrent subscription/group change therefore cannot produce a server-generated backup that the restore validator rejects.
Membership mutation serialization
Group membership assignment, direct unsubscribe, TypeType restore, and PipePipe replacement import now acquire the same account-scoped PostgreSQL transaction advisory lock. The subscription existence check and membership insert cannot straddle a committed deletion, while replacement imports still retain memberships for channel URLs that survive.
Stable feed pagination
A grouped or ungrouped first page derives an immutable membership snapshot used by every continuation cursor. Snapshots are stored in eight fixed, account-scoped Dragonfly slots:
SET NX EXclaims prevent cross-process lost updates;429 subscription_feed_cursor_capacitybefore a cursor is issued.Cursor resolution remains bound to the authenticated account, filter, visibility, page size, and feed generation.
Subscription creation contract
POST /subscriptionsnow acceptsSubscriptionCreateRequest, which does not exposesubscribedAt. The service always assigns the timestamp. An obsolete client-suppliedsubscribedAtfield is ignored by the production JSON configuration and cannot control stored state.API changes
GET/POST /subscriptions/groupsPUT/DELETE /subscriptions/groups/{groupId}PUT/DELETE /subscriptions/groups/{groupId}/channelsGET /subscriptions?groupId=...GET /subscriptions?ungrouped=trueGET /subscriptions/feed?groupId=...GET /subscriptions/feed?ungrouped=trueThe handwritten OpenAPI contract documents the subscription and group operations, filtered feed cursors, and the capacity response.
Commit structure
7c9c93b47ef9808761b01ade461269708665282d860d6869f756bfab8336b003939158abaf4cd5c1abd1ae9a93a0830a267fda9aVerification
Exact head:
267fda9a7cf7854655b4289f9cf7665c33f26c2fUsing the required JDK 25 toolchain:
This includes OpenAPI validation, compilation, coverage verification, route/service/database regressions, legacy-backup compatibility, cursor/filter isolation, and the shadow JAR build.
Live API QA ran two real shadow-JAR server processes against shared disposable PostgreSQL and Dragonfly instances. It verified concurrent cross-process cursor creation and continuation, identical snapshot reuse, eight-slot saturation, typed 429 for the ninth session, first-cursor survival, server-owned timestamps, backup round-trip, legacy backup acceptance, and cross-account 404 isolation.
Additional exact-head QA held the account advisory lock in real PostgreSQL and issued concurrent authenticated membership assignment and unsubscribe requests. Both requests waited; after release, unsubscribe returned
204, assignment re-checked state and returned404, and both HTTP and SQL read-back showed zero orphan memberships. A deliberately inconsistent membership fixture was omitted from HTTP backup export, and that server-generated backup restored to a fresh account with200.Component follow-up
A separate TypeType-Frontend PR is required to add group management, channel assignment, and filtering UI. No Token, Downloader, or Player change is required.