Skip to content

feat: add subscription groups server contract - #76

Draft
kapdon wants to merge 13 commits into
TypeType-Video:devfrom
kapdon:feat/subscription-groups-server
Draft

feat: add subscription groups server contract#76
kapdon wants to merge 13 commits into
TypeType-Video:devfrom
kapdon:feat/subscription-groups-server

Conversation

@kapdon

@kapdon kapdon commented Aug 7, 2026

Copy link
Copy Markdown

Summary

Adds the backend contract for named subscription groups as the first Server-only part of TypeType-Video/TypeType#172.

  • creates, renames, lists, and deletes account-scoped groups;
  • supports many-to-many membership for existing subscribed channels;
  • filters GET /subscriptions and GET /subscriptions/feed by groupId or ungrouped=true;
  • preserves the existing global subscription list/feed when no filter is supplied;
  • freezes each filtered feed's channel membership for its full cursor session;
  • includes groups and memberships in backup export and transactional restore;
  • uses a create-only subscription request schema, with subscribedAt generated by the server;
  • clears memberships safely when a subscription or replacement import/restore removes the underlying channel;
  • serializes membership assignment with unsubscribe and replacement restores so an orphan membership cannot commit.

The branch is rebased onto the current dev base and the history is split into focused commits, each below the requested 290-insertion limit.

Review feedback addressed

Backup round-trip

Backup exports now include subscription groups and their channel memberships. Restore recreates groups with fresh IDs, reconnects memberships by canonical channel URL, and performs the replacement in the same database transaction as subscription restore. Backups created before the new optional subscriptionGroups field remain accepted.

Export captures the subscription list once and filters group memberships against that canonical captured set. A concurrent subscription/group change therefore cannot produce a server-generated backup that the restore validator rejects.

Membership mutation serialization

Group membership assignment, direct unsubscribe, TypeType restore, and PipePipe replacement import now acquire the same account-scoped PostgreSQL transaction advisory lock. The subscription existence check and membership insert cannot straddle a committed deletion, while replacement imports still retain memberships for channel URLs that survive.

Stable feed pagination

A grouped or ungrouped first page derives an immutable membership snapshot used by every continuation cursor. Snapshots are stored in eight fixed, account-scoped Dragonfly slots:

  • terminal pages allocate no slot;
  • identical membership selections reuse and refresh the same slot;
  • atomic SET NX EX claims prevent cross-process lost updates;
  • occupied slots are never overwritten, so an issued cursor remains valid for the cache TTL;
  • a ninth distinct active session returns documented 429 subscription_feed_cursor_capacity before a cursor is issued.

Cursor resolution remains bound to the authenticated account, filter, visibility, page size, and feed generation.

Subscription creation contract

POST /subscriptions now accepts SubscriptionCreateRequest, which does not expose subscribedAt. The service always assigns the timestamp. An obsolete client-supplied subscribedAt field is ignored by the production JSON configuration and cannot control stored state.

API changes

  • GET/POST /subscriptions/groups
  • PUT/DELETE /subscriptions/groups/{groupId}
  • PUT/DELETE /subscriptions/groups/{groupId}/channels
  • GET /subscriptions?groupId=...
  • GET /subscriptions?ungrouped=true
  • GET /subscriptions/feed?groupId=...
  • GET /subscriptions/feed?ungrouped=true

The handwritten OpenAPI contract documents the subscription and group operations, filtered feed cursors, and the capacity response.

Commit structure

Commit Insertions Purpose
7c9c93b4 84 group persistence
7ef98087 255 membership persistence and service
61b01ade 116 group API
46126970 67 grouped subscription lists
8665282d 163 stable grouped feed pagination
860d6869 149 backup round-trip
f756bfab 214 OpenAPI contract
8336b003 120 persistence tests
939158ab 161 route tests
af4cd5c1 168 feed stability tests
abd1ae9a 285 bounded cross-process snapshot storage
93a0830a 107 subscription membership mutation serialization
267fda9a 90 consistent subscription/group backup export

Verification

Exact head: 267fda9a7cf7854655b4289f9cf7665c33f26c2f

Using the required JDK 25 toolchain:

./gradlew --no-daemon clean check shadowJar validateOpenApi --console=plain
BUILD SUCCESSFUL
1,068 tests, 0 failures, 0 errors, 0 skipped

This includes OpenAPI validation, compilation, coverage verification, route/service/database regressions, legacy-backup compatibility, cursor/filter isolation, and the shadow JAR build.

Live API QA ran two real shadow-JAR server processes against shared disposable PostgreSQL and Dragonfly instances. It verified concurrent cross-process cursor creation and continuation, identical snapshot reuse, eight-slot saturation, typed 429 for the ninth session, first-cursor survival, server-owned timestamps, backup round-trip, legacy backup acceptance, and cross-account 404 isolation.

Additional exact-head QA held the account advisory lock in real PostgreSQL and issued concurrent authenticated membership assignment and unsubscribe requests. Both requests waited; after release, unsubscribe returned 204, assignment re-checked state and returned 404, and both HTTP and SQL read-back showed zero orphan memberships. A deliberately inconsistent membership fixture was omitted from HTTP backup export, and that server-generated backup restored to a fresh account with 200.

Component follow-up

A separate TypeType-Frontend PR is required to add group management, channel assignment, and filtering UI. No Token, Downloader, or Player change is required.

@Priveetee Priveetee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, first thx a lot for taking the time to work on this!

I went through the database changes, routes, account isolation, feed filtering, restore behavior, OpenAPI and tests.

Honestly the base is really good. Groups are correctly isolated per account, one channel can be in multiple groups, the ungrouped feed is handled properly, and reusing the existing subscription feed snapshot makes sense.

I found a few things that need to be fixed before merge:

  • The important one is backups. Groups and their channel memberships are not exported right now. Restoring on a fresh instance would restore the subscriptions, but all the organization created by the user would be lost. Both the groups and their memberships need to be included in TypeType backups.

  • There is also an edge case with pagination. The cursor only remembers the group ID and filter, while the group membership is loaded again for every page. If someone edits the group between two pages, the next page can skip or repeat videos. The cursor needs to keep the same group membership snapshot for the full pagination session.

  • Small OpenAPI mismatch: subscribedAt is required in the subscription creation schema, but Server actually generates it and ignores the submitted value. It would be cleaner to have a separate request schema where this field is not required.

Could u also split the first commit please? It currently contains 1,174 insertions, and I try to keep commits below 290 insertions so it stays easy to understand later why each part changed. Separating the database/model work, group API, feed filtering, backup support and tests would make the history much easier to follow.

I ran the full validation: 998 tests passed, including OpenAPI validation, coverage and the production jar build. I also checked it together with the current Server changes and all 1,036 tests passed there.

So yeah, no need to rebuild everything from scratch at all. The base is good, it just needs these points fixed and I will gladly review it again :)

Thx again, and have a great day!

noreply added 11 commits August 17, 2026 15:34
Add account-scoped group and membership tables plus the API models used by the group service. Register both tables in production and test database setup.
Create, rename, and remove account-owned groups and assign subscribed channels to multiple groups. Keep memberships consistent when subscriptions are deleted or replaced by imports.
Add authenticated CRUD and membership endpoints backed by the account-scoped group service, and register the service with the application.
Parse group and ungrouped selectors for subscription reads, reject invalid or foreign group IDs, and separate subscription creation input from server-generated timestamps.
Project group and ungrouped feeds from the shared global snapshot while retaining each cursor's account-scoped membership selection in cache for the full pagination session.
Export named groups with their channel memberships and restore them transactionally with subscriptions. Validate names and membership references before replacing account-owned group data.
Describe group management and filtered list/feed operations, and make subscription creation use a request schema without the server-generated subscribedAt field.
Verify account isolation, normalized unique names, many-to-many membership, ungrouped selection, and cleanup after subscription replacement or deletion.
Exercise authenticated group CRUD, membership updates, filter validation, and cross-account access through the HTTP routing surface.
Verify shared-snapshot projection, source-channel attribution, filter-bound cursors, and unchanged membership snapshots across paginated group reads.
Filtered pagination needs a stable membership view without creating an
unbounded cache entry for every initial request. Allocate a fixed set of
account-scoped slots atomically, reuse content-derived tokens, and reject a
new distinct session at capacity without evicting any issued cursor. Strengthen
observable compatibility coverage for legacy backups and server timestamps.

Constraint: Every issued cursor must retain its membership snapshot for the full cache TTL
Rejected: Evict the oldest snapshot after eight sessions | invalidates a still-live cursor
Rejected: Store all snapshots in one read-modify-write value | loses concurrent writes across server instances
Rejected: Put channel URLs directly in the cursor | produces oversized client-controlled cursor payloads
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Never overwrite an occupied selection slot; reject new sessions before weakening issued cursors
Tested: Focused feed, concurrent selection-store, subscription route, backup, and OpenAPI tests on JDK 25
Not-tested: Live Dragonfly slot saturation before the final runtime gate
@kapdon
kapdon force-pushed the feat/subscription-groups-server branch from a260906 to abd1ae9 Compare August 17, 2026 23:48
@kapdon
kapdon marked this pull request as draft August 18, 2026 01:36
Subscription group assignment checked subscription ownership without
coordinating with unsubscribe or replacement restores. Take an
account-keyed transaction advisory lock across membership assignment and
every subscription removal or replacement path so the check and insert
cannot straddle a committed deletion.

Constraint: Replacement imports must retain memberships whose subscriptions survive the import
Rejected: Composite foreign key with cascading deletes | cascade semantics would discard memberships before retained subscriptions are reinserted
Confidence: high
Scope-risk: narrow
Directive: Any new path that removes or replaces an account's subscriptions must acquire SubscriptionMutationLock in the same transaction
Tested: Focused PostgreSQL concurrency regression on JDK 25
Not-tested: Full suite and live HTTP concurrency gate run after both fix commits
Subscription and group sections were read in separate transactions, so a
membership committed between them could reference a subscription absent
from the exported list. Capture subscriptions once and export only group
memberships belonging to that captured set, preserving the restore
validator's referential invariant.

Constraint: Subscription groups remain coupled to the subscriptions backup category
Rejected: Add a cross-service export transaction | existing service reads open their own transactions and captured-set filtering is the smaller accepted repair
Confidence: high
Scope-risk: narrow
Directive: Exported group memberships must remain a subset of the subscriptions captured for the same backup
Tested: Focused mixed-read export and restore regression on JDK 25
Not-tested: Full suite and live HTTP concurrency gate run after this commit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants