Skip to content

feat(cli): measure trail, box, rhythm, objects, track, flow and continuity - #7728

Draft
WaterrrForever wants to merge 1 commit into
mainfrom
miao/reverse-skill-measure
Draft

WaterrrForever wants to merge 1 commit into
mainfrom
miao/reverse-skill-measure

Conversation

@WaterrrForever

Copy link
Copy Markdown
Collaborator

What this carries

Seven hyperframes measure ops that were written for reverse-any-video and never reached main. They were staged on feat/reverse-video-skill (Oct 22–29, identical to the copies on miao/reverse-journey-test) and dropped from #6754 because the skill did not call them yet.

Op What it reports
trail A window's frames as one picture: onion skin, the moving part's path, a heat map of change
box One graphic found by its colour: box, ink, colours, text lines, cap height; over a range when it is on screen, and with --track each move's ease and the holds
rhythm Moving share, holds, update rate (ones, twos), cuts and each shot's camera; --compare gives ratios against a render
objects Separate elements at one time and their arrangement (ring, grid, row, column, stack)
track One textured thing (a cursor, a hand) followed by template matching until a cut or a lost match, its moves fitted to eases
flow One element's moves from optical flow as tweens, or compared with a render's on one clock
continuity Jumps, flicker and extra or missing shapes a render has and the original does not, with a sheet per event

Adapted to main as it is now:

  • Commands use the shared defineCommand factory, so the agent-mode contract test covers them.
  • They run on main's measure helpers (ffprobe reasons, odd-sized box reads). Window flags also take --from/--to (trail: --start/--end) and folder flags -o, like the other ops; rhythm uses compareWindowArgs.
  • curve.ts exports Row, EASES and fitMoves again (used by box, track, flow).
  • continuity's Gaussian and box blurs share one separable pass (fallow flagged the duplicate). The results are the same.
  • Five helpers used only inside their own module are no longer exported. The 11 complex numeric functions get fallow-ignore-next-line complexity, as the existing measure code does.
  • The OpenCV tests (track, flow) load opencv.js with loadOpenCvFirst, as diff's do, so CI runs them. The face test in the old opencv.test.ts is dropped because faceVerdict.test.ts already covers it.
  • Examples use ref.mp4, not .reverse/source.mp4.
  • docs/packages/cli.mdx lists every measure op, including ink, which was missing.

Skipped: main already has it (newer)

Most of the staged work landed through #6754, #6833 and #6891 and has changed since, so main's version is kept:

  • The 11 existing ops and their helpers and tests: commands/measure/{probe,cuts,events,frames,extract,curve,energy,ink,look,face,sync}.ts and measure/{command,cuts,curve,diff,drawn,energy,events,face,ink,look,media,opencv,person,segments,sync}.ts.
  • commands/diff.ts, commands/media-use.ts, commands/media-use/generate.ts, commands/snapshot.ts, commands/transcribe.ts, api/transcribe.ts, heygen/{generate,pricing,transcribe,v3}.ts with their tests, utils/checkBrowser.ts with its test, and media-use/lib/*.d.mts.
  • Skills: reverse-any-video (SKILL.md, references/match.md, references/people.md), media-use (SKILL.md, references/heygen.md, references/operations.md), and remotion-to-hyperframes (including the two deleted scripts). Also .prettierignore, scripts/check-core-skills.mjs and skills-manifest.json. Each staged handbook is identical to, or an older draft of, what is on main, and none of them uses the seven ops, so no skill changes.

Left out

  • analyze (commands/analyze.ts and its test, heygen/reverseDraft.ts, and its cli.ts/help.ts entries), plus diff --explain (heygen/explain.ts and its test). feat(skills): reverse any video on the HyperFrames plan #6754 dropped these on purpose because nothing called them. analyze calls runJob(route, body, key, statusPath, opts), but main's runJob now takes a JobRoute, so it would need rework.
  • Engine audioMixer first_pts fix and its test. This pads a sound read from the file's start to its first decoded timestamp, for HE-AAC edit lists that played about 0.1 s early. main doesn't have it, but the patch conflicts with main's rewritten seek path (ffmpegSeekSeconds, misstatedByteRate, the sidecar warm-up). It's an engine-wide audio change, so it belongs in its own PR.
  • bun.lock: the staged change only bumped version numbers.

Open question

No skill calls these ops yet, which is why #6754 dropped them. Should reverse-any-video (for example references/match.md) point agents at them, or should they stay CLI-only, documented in docs/packages/cli.mdx?

Tests

  • bun run build ✓
  • packages/cli typecheck (tsc --noEmit) ✓
  • New and changed measure tests: box, objects, continuity, rhythm, opencv (track, flow, similarity) and curve: 6 files, 11 tests ✓
  • cli.agentMode.test.ts -t measure: 76 tests ✓
  • scripts/ci/flake-gate.mjs --base origin/main: each of the 5 new test files passed 10/10 under CPU contention ✓
  • Full packages/cli suite: 8593 passed, 139 skipped, 16 failed in 9 files: checkSweep.chromium, cloud/offload.undecodableTrack, lambda/deploy, snapshotHdrFrame, server/fileWatcher.atomic, publishProject (the Unicode-form case), and whisper/{manager-resolution,manager.install,sherpa}. The same 16 fail on origin/main (58ef24c) on this machine. None of those files imports measure code, and the failures look tied to this machine's toolchain (whisper and sherpa runtimes, ffmpeg 6.0, Chrome WebGL, APFS file names).
  • bun run lint ✓; fallow audit --base origin/main --fail-on-issues ✓ (its only warnings are the 12 added suppressions); comment citations and ratchet ✓; check-docs-cli-commands ✓; test reachability ✓; check:cli-lazy-deps ✓
  • Smoke test of the CLI from source on a synthetic 3 s clip, with each op run alone and against a render where it supports one. track follows a 200 px slide over 0.5–1.5 s with no loss (mean score 0.997). flow --compare gives an energy ratio of 0.97 and motion correlation of 0.98. continuity flags the one red frame drawn into the render as a jump. box --track reads a 0.8 s linear x move. trail and track write their pictures.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NqeWUzwnWFt2hjCRT7p6ux

…nuity

Seven measure ops written for reverse-any-video were left out when the
skill landed (#6754), because the skill did not call them yet. They
answer what cuts, frames and faces do not: how a title enters, where a
graphic sits and how it eases, whether an element moves in the render as
it does in the original, and where a rebuild has seams the original
lacks.

- trail: a window's frames as one picture: onion skin, the moving
  part's path and a heat map of change
- box: one graphic found by its colour: box, ink, colours, text lines
  and cap height; over a range, when it is on screen, and with --track
  each move's ease and the holds
- rhythm: moving share, holds, update rate, cuts and each shot's
  camera, alone or as ratios against a render (--compare)
- objects: the separate elements at one time and their arrangement
  (ring, grid, row, column, stack)
- track: one textured thing followed by template matching until a cut
  or a lost match, its moves fitted to eases
- flow: one element's moves from optical flow, or compared with a
  render's on one clock
- continuity: jumps, flicker and extra or missing shapes a render has
  and the original does not, with a sheet per event

They run on main's shared measure helpers, so they get its ffprobe
reasons and odd-sized box reads; window flags also take --from/--to
(trail's --start/--end) and folder flags -o, as the other ops do. curve
exports Row, EASES and fitMoves again for box, track and flow.
continuity's Gaussian and box blurs share one separable pass. The
optical-flow tests load opencv.js with loadOpenCvFirst, as diff's do.
docs/packages/cli.mdx lists every measure op, ink included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NqeWUzwnWFt2hjCRT7p6ux
@mintlify

mintlify Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
hyperframes 🟢 Ready View Preview Nov 14, 2026, 8:26 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

This branch was successfully deployed

1 active deployment
staging - docs — 72d879f2 Deployed Nov 14, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant