drafts: fold the archive format into hang as a Recording section - #2574
drafts: fold the archive format into hang as a Recording section#2574kixelated wants to merge 2 commits into
Conversation
moq-archive specified a generic, arrival-ordered chunk format at the moq-lite layer, with its own per-track index mapping group runs to byte ranges. It predated the timeline rework, which already publishes a per-segment index of group ranges per track. Addressing a recording by segment instead of by flush interval collapses the format: the segment number names the object, so the timeline is the only index and the separate index log, chunk numbering, byte offsets, partial spans, and overlap resolution all disappear. It also gives a reader one whole-object GET per segment per track, which byte-range splicing across chunks could not. That makes the format a hang concept rather than a moq-lite one, since segments are defined by the timeline and the timeline is hang. Specify it in the hang draft and delete the standalone draft, which was never submitted to the datatracker. Segment objects carry their own group boundaries so they parse without the timeline, and frames stay byte-identical to moq-lite FRAME so a recorded group is a FETCH response body unchanged. Known trade: a group arriving after its segment object is written is not recorded. Writers wait for completeness and bound that wait. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Cxx2ebpiGmWaD8AvJkRdi
The segmenter required every enrolled track to vote on boundaries and to report past a segment's end before its record could flush. That works for audio and video, whose groups arrive continuously, and stalls the timeline permanently for anything else: boundary() returns None the moment an enrolled track has no group past the threshold, and a catalog publishes a group only when the renditions change. So a broadcast could only segment tracks it could also pace, which left catalogs, metadata, and application tracks like a chat log with no way to be indexed at all. Add Producer::passive alongside Producer::track. A passive track's groups are recorded into whichever segment is open when they arrive, but it never votes on a boundary, never gates completeness, and never anchors the first segment (a catalog published while the encoder warms up would otherwise stretch segment 0 across the whole startup gap). It also does not create the timeline track, so segmentation stays opt-in by pacing track. Placement is by arrival rather than content time, since nothing waits for these groups: one that shows up after its segment flushed is recorded in the next. The frames still carry their own timestamps. A passive track whose group never closes is recorded once, in the segment its group opened in. Rolling the group at segment boundaries is the fix and is left to the publisher. Recording drops its special case for the catalog as a result: every track the timeline lists is stored the same way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Cxx2ebpiGmWaD8AvJkRdi
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ed2fe6e74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ~~~ | ||
| <prefix>/.timeline | ||
| <prefix>/<track>/<segment> | ||
| ~~~ |
There was a problem hiding this comment.
Persisting only .timeline and per-track segment objects makes a recording impossible to decode or replay after the live broadcast disappears. The stored FRAME timestamps are deltas in the track's negotiated timescale, and moq-lite requires TRACK_INFO before parsing them (drafts/draft-lcurley-moq-lite.md:1180-1182 and 1050-1054), but this layout stores neither the timescale nor the other immutable properties needed to reconstruct TRACK_INFO. Add per-track metadata to the recording layout so readers can interpret frames and serve the recorded tracks. (Written by GPT-5.6 Sol)
Useful? React with 👍 / 👎.
| pub fn passive(&self, name: &str) -> crate::Result<Recorder> { | ||
| self.enroll(name, true) |
There was a problem hiding this comment.
This only marks the track passive for boundary and completeness calculations. flush_segment still stops draining it when group_pts >= end, and the terminal duration still takes the maximum frontier across all tracks; the JS implementation mirrors both behaviors. When passive metadata uses a future or different timestamp basis, such as catalog snapshots stamped with Timestamp::now() alongside file media starting at PTS 0, an update that arrived during an open segment remains pending until the final flush and can inflate or fail the final segment. Drain all pending passive groups at each flush and exclude passive frontiers from terminal duration. (Written by GPT-5.6 Sol)
Useful? React with 👍 / 👎.
| Segment objects are immutable and SHOULD be served with long-lived caching. | ||
| `.timeline` and `.catalog` change while the recording is live and SHOULD be served with short lifetimes; all three become immutable once the recording has ended. |
There was a problem hiding this comment.
.catalog object
The layout defines only .timeline and immutable <track>/<segment> objects, with catalog generations stored as ordinary segment objects under the encoded catalog track name. Consequently there is no .catalog object that changes during a live recording, and catalog segment objects should receive the long-lived caching described immediately above. This instruction can make implementations request or publish an undefined resource and apply the wrong cache policy. (Written by GPT-5.6 Sol)
Useful? React with 👍 / 👎.
2 participants
Deletes
draft-lcurley-moq-archiveand specifies the storage format as a# Recordingsection indraft-lcurley-moq-hang.Why
moq-archive(#2504, closing #2456) specified a generic, arrival-ordered chunk format at the moq-lite layer: per-track chunks written on a flush interval, plus a per-trackindex.binlog mapping run-length group ranges to byte offsets within them. It was written before #2547 reworked the timeline into a single track of complete segments, which already publishes a per-segment index of group ranges per track. The two formats independently invented the same primitive (hang::timeline::Rangeand the archive index's run-lengthRun), a week apart.The archive's chunk boundaries were arbitrary, so it needed an index to say which groups landed where and at what byte offset. Addressing a recording by segment instead collapses that: the segment number names the object, so the timeline becomes the only index and the separate index log, chunk numbering, byte offsets, continuation spans, and later-chunk-wins overlap resolution all disappear.
It also buys the property the format exists for. Byte-range splicing across chunks meant a reader might need several requests to reconstruct one segment; segment-addressed objects mean one whole-object GET per segment per track, which is what an HLS/DASH origin and a VOD player both want.
Since segments are defined by the timeline and the timeline is hang, this makes the format a hang concept rather than a moq-lite one. The standalone draft was never submitted to the datatracker (no
number, nodate), so deleting it costs no published version.What the section specifies
Layout, with
<track>percent-encoded so it can never begin with.and collide with the reserved names:Sequence,Length, then frames), so an object parses without the timeline. Frames stay byte-identical to moq-lite FRAME, so a recorded group is a FETCH response body unchanged and a recording never re-encodes media..timelinestores the timeline track's frames verbatim; a follower ranged-GETs from its last offset and feeds the bytes to the decompressor it already holds, which the sync-flush framing makes decodable..catalog/<segment>is keyed by the first segment it applies to, so "the catalog in effect at segment N" is the highest object not greater than N. This resolves the catalog-generations question the archive draft left open.Trade-off
A group arriving after its segment object is written is not recorded. The archive's arrival-ordered design could append it to whatever chunk was open; a segment-addressed one cannot. Writers wait for segment completeness (which the timeline already gates on) and may bound that wait so a stalled track cannot hold up the recording. This is stated explicitly in the section rather than left implicit, since it is a real capability being traded for single-request retrieval.
Base branch
Targets
devrather thanmain, against the usual docs rule, because the section builds directly on text that only exists ondev: the# Timelinesection (#2547) and moq-lite's frame-bounded FETCH (#2537). Written againstmainit would reference sections that are not there.Validation
nixandkramdown-rfcare not available in this container, sojust drafts checkwas not run and should be run before merge. What was verified:bun run --cwd doc checkpasses (33 tests, VitePress build clean), which renders the drafts through the site translator and catches constructs it does not handle.{{...}}cross-reference in the file resolves to a defined anchor or a frontmatter reference. This caught one real problem:{{catalog}}had no target, so# Cataloggained a{#catalog}anchor./draft/moq-archivepage is gone and the draft index no longer links it. Both the drafts justfile anddrafts.tsglob the directory, so no registration needed updating, and nothing else in the repo referenced the draft.Not in this PR
rs/moq-archiveimplementation exists yet; DVR: rewind and time-shift a live broadcast (timeline + FETCH exist, durable storage doesn't) #2275 (DVR) and moq-cli: generic broadcast recording and paced replay (all tracks + catalog) #2281 (recording/replay) remain open.broadcast::Consumeronly takes a track by name, so a recorder discovers tracks through the hang catalog. A.tracksmeta track would make that generic, and is left as a separate moq-lite change.Generated by Claude Code