Edit videos by transcript

[richiemcilroy]

Adds word-level transcript editing in the web video editor. Transcription now stores an immutable, encrypted word transcript; the sidebar lets you cut fillers/words into timeline deletes, and edits remap captions from that transcript instead of re-transcribing when possible.

Greptile Summary

Adds transcript-based video editing across the web and media-processing pipeline.

• Stores encrypted, word-level transcripts and remaps captions after timeline edits.
• Adds transcript selection and filler-removal controls to the web editor.
• Reworks media rendering to process retained ranges in bounded, sequential FFmpeg batches using input-side seeking.
• Adds integration and unit coverage for transcript storage, backfills, editing, rendering, and UI helpers.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported decode-amplification issue is addressed by input-side seeking, batches capped at four ranges, sequential batch execution, and a timeout shared across rendering and concatenation; no blocking failure remains.

Important Files Changed

Filename	Overview
apps/media-server/src/lib/media-edit.ts	Replaces the previously reported full-input filter branches with bounded groups of independently seeked inputs, sequential execution, and a shared operation timeout.
apps/web/lib/edit-transcript.ts	Implements word-transcript validation, cut planning, timeline remapping, filler detection, and caption generation.
apps/web/lib/edit-transcript-storage.ts	Adds authenticated encryption for transcript sidecars with explicit IV and authentication-tag length validation.
apps/web/actions/videos/get-edit-transcript.ts	Loads owner-scoped transcript sidecars, remaps them through existing edits, and coordinates serialized backfill requests.
apps/web/workflows/transcribe.ts	Persists immutable word transcripts during transcription and adds guarded backfill processing for existing videos.
apps/web/workflows/edit-video.ts	Reuses the original word transcript to regenerate captions after edits while preserving original-timeline transcript data.
apps/web/app/s/[videoId]/edit/TranscriptSidebar.tsx	Adds transcript-driven word selection and filler-cut controls to the video editor.

_{Reviews (2): Last reviewed commit: "fix: bound video edit transcoding" | Re-trigger Greptile}

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[tembo]

disfluencies: true is already set inside getAssemblyAITranscriptionOptions, so this is redundant. The same duplicate appears in transcribeEditTranscriptWithAssemblyAI below.

Suggested change

	...getAssemblyAITranscriptionOptions(language),
	disfluencies: true,
	});
	...getAssemblyAITranscriptionOptions(language),
	});

[tembo]

The original transcription pass uses the org's configured aiGenerationLanguage (via validateVideo), but the backfill hardcodes "auto". For orgs that pinned a specific language, this can produce a transcript with different word segmentation or even a mismatched detected language, and the remapped captions would then diverge from the original ones. Consider threading the org's aiGenerationLanguage into the backfill payload (or loading it in validateEditTranscriptBackfill) so both passes stay consistent.

[tembo]

Observing document.body with subtree: true fires the callback on every DOM mutation across the whole page (tooltips, menus, virtual list updates, etc.). Although attachVideoListeners early-returns once the video is bound, the observer churn itself adds overhead on a view that already runs a virtualizer and rAF loops. Scoping the observer to the video element's nearest container, or replacing it with a lighter check (e.g. a short polling retry on mount/words change), would avoid the app-wide mutation tax.

[richiemcilroy]

hey @greptileai, please re-review the PR