Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 53 additions & 18 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1329,9 +1329,23 @@ the **single** builder of both decoders' arguments (encode + preview):
mid-file: ffprobe counts the last segment only, `-ss` can't reach a later
segment, and `-copyts` stops at the reset. `source_continuity.rs` is the
one decision: a reset is ffmpeg's own (DTS > 0.1 s back or > 10 s forward,
per A/V stream), and the fix is a **same-container stream copy without
`-copyts`**, checked (no reset left, every video packet) before use and
made only when a reset is found — a file without one keeps every argument.
per A/V stream), and the fix is a **same-container stream copy**, checked
(no reset left, every video packet) before use and made only when a reset
is found — a file without one keeps every argument.
- **The picture decides the join (task #58).** Each segment moves by one
offset, every stream alike, so its first picture follows the last
segment's last by one frame (`JoinPlan`, made by the scan from each
packet's DTS, duration and position): `-copyts` and a `setts` filter per
stream (by packet index `N` for a stream that resets, by byte position
`POS` for one that doesn't: subtitles; MPEG audio split from a PES has
no position). A sound packet overhanging the next segment by more than
half of itself is dropped (`noise`), and the log says so. ffmpeg's own
correction (no `-copyts`) took the offset from whichever stream showed
the reset first: a part whose sound ran 16 ms past its last picture put
every later picture off the grid (125 fps, 493 pictures for 100). It is
still the fallback when no plan can be made (no durations, streams
resetting a different number of times) or the planned copy fails its
check.
- **The app makes it after analysis, in the background (task #55).**
The scan reads every byte (77 s for a 2 h DVD title on a network
share), so analysis probes with `--probe-source --defer-scan` (the model
Expand Down Expand Up @@ -1420,12 +1434,18 @@ the **single** builder of both decoders' arguments (encode + preview):
transcription, `OutputWindow`, `SoundCut` and the notes keep the true
start. The sidecar decision (`needs_sound_sidecar`) asks with the
encoder's start, so `main.rs` measures the priming before the sound copy.
- **A trimmed one stops at the window's length (`-t`), not `-shortest`**
(`encode_end_args`, task #58): read late by P, a sound that ends with the
window (the restored track; a source's sound at a trim to its last
picture) ends P before the picture on ffmpeg's timeline, and
`-shortest` dropped the last picture (49 stored for 50).
- The cost is the first P (21–23 ms) of sound, replaced by the encoder's
near-silent start-up; a failed probe is a warning, never a refusal; every
other job keeps its arguments byte for byte
(`a_priming_shift_changes_nothing_outside_convert_into_avi`).
`integration_video_start_offset_test.dart` (heavy): AAC/MP3/AC-3 into AVI
store 100 frames and are in sync within 1 ms, restored and trimmed too,
store 100 frames and are in sync within 1 ms, restored and trimmed too
(the matrix counts trimmed pictures since task #58),
and `small_clip.mp4` 0..49 stores 50 (docs/reports/2026-11-avi-convert-priming).
- **An AAC stream in AVI declares a block size no packet exceeds (task
#41).** avienc writes AAC's `nBlockAlign` as 768 × channels, and ffmpeg,
Expand Down Expand Up @@ -1783,11 +1803,18 @@ Invariants that are easy to break:
`PitchFactorFile.sidecarCandidates` or the factor's meaning (`invertFactors`).
- **Pitch on a still-interlaced clip clears `_FieldBased`** (a vertical resample
would mix the fields; film scans are progressive). Weave alone leaves it.
- **`pipe_source` evicts silently.** It keeps `cache_frames` decoded frames and
returns the *last* frame read for anything already evicted. The cache is
sized from the pipeline's declared radius (`pipe_cache_frames`, never below
the old 40), so a pass with a large window must report it through its
`FrameMap` radius.
- **`pipe_source` raises on a frame it has dropped (task #58).** A pipe reads
forward only, and VapourSynth asks out of order: vspipe requests a frame
whenever any completes, so one late callback let the reader read the whole
clip past it (frames 6–7 served after 99 was read, on a plain job under
load). It keeps every frame until it is served and `cache_frames` behind
the oldest frame not yet served, with a memory cap (512 MB beyond the
window) for frames never asked for; a frame asked for after it was dropped
raises. It used to return the *last* frame read instead, silently. The
window is sized from the pipeline's declared radius (`pipe_cache_frames`,
never below 40), so a pass with a large window must report it through its
`FrameMap` radius, or its job now fails rather than writing wrong frames.
`pipe_source_order_test.dart` (push gate) drives the reader in that order.

Tests: `test_211`…`test_218` (script generation, legacy block, offsets, errors,
cache size, field marking) and `app/test/integration_stabilize_test.dart`
Expand Down Expand Up @@ -2509,18 +2536,20 @@ table. Rules:
codec the container takes, Restore, Restore + Convert AAC) × source class
(plain MP4, late-video TS, delayed AVI, reset PS through the continuity
copy) × trim (none, from frame 2, from frame 40), plus ProRes/MOV and
AV1/MP4 rows, from `generate_output_conformance.sh`'s flash/click sources:
~440 cases, generated from the enums and the table
AV1/MP4 rows and trims to the last picture for the primed Convert codecs
into AVI (`tailCases`), from `generate_output_conformance.sh`'s
flash/click sources: 448 cases, generated from the enums and the table
(`test/support/output_matrix.dart`, which also holds the oracle; the push
gate's `output_matrix_cases_test.dart` checks the cases). Each checks the
header, picture count and first picture, no null frames, every flash's
click within one audio packet, packet timing, and lossless paths sample-
or byte-exact. **A new container fact or source rule gets a case here, not
another one-cell heavy test.** Cases that fail today are `knownFailures`
(id → task, today all task #58); they still run and fail when they pass.
One fault can't be listed: `pipe_source` intermittently serves an evicted
frame as the last one (#58), so the matrix fails a different case or two
per run until that is fixed. One case:
click within one audio packet **of where the source has it** (each
source's own lag, measured: `conf_delayed.avi`'s MP3 is 23 ms late by
libmp3lame's priming, which AVI cannot mark), packet timing, and lossless
paths sample- or byte-exact (a restore against the render's decoder).
**A new container fact or source rule gets a case here, not another
one-cell heavy test.** Cases that fail today are `knownFailures` (id →
task; empty since task #58); they still run and fail when they pass. One
case:
`flutter test --tags heavy test/integration_output_conformance_test.dart
--plain-name 'avi/convert-aac/delayed-avi/near-start'`. The shared
measuring helpers are `test/support/av_sync.dart`.
Expand Down Expand Up @@ -2549,6 +2578,12 @@ Rules, from docs/reports/2026-11-audio-restoration:
lead `adelay` → [DeepFilterNet] → `loudnorm` + `aresample` → `alimiter` →
`apad`+`atrim` to the picture's length. Each ffmpeg run gets one `-af`
(`every_combination_keeps_the_order_and_one_af_per_run`, 512 × 5 cases).
- **MP1/MP2 are read with the float decoder** (`render_decoder`:
`mp2float`, task #58). FFmpeg's default for them is fixed-point, which
rounds with an error carried from frame to frame, so a render that seeked
differed from the source's decode by one 16-bit step in ~40% of samples;
the float decode is exact from any seek. MP3 and AC-3 default to float
already; Convert and the analysis keep the default.
- **Measured traps, each now handled in the plan:** `afftdn` outputs 25 ms
late at every rate (`atrim=start_sample=rate/40`); its default `nf=-50` took
0.7 dB off −39 dB hiss — it gets the source's measured floor (quietest tenth
Expand Down
97 changes: 67 additions & 30 deletions app/test/integration_output_conformance_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,12 @@ import 'support/worker_harness.dart';
String _resource(String name) =>
p.join(WorkerHarness.repoRoot, 'Tests', 'TestResources', name);

/// The task that owns the off-grid join (task #58, filed by task #49).
const _offGridJoin = 'off-grid reset join, task #58';

final _outDir = p.join(Directory.systemTemp.path, 'vapourbox_output_conformance');

/// One source as the app analyses it.
class _Source {
_Source(this.decodePath, this.model, this.info, this.videoStart, this.payload, this.samples,
this.packetSeconds);
this.renderSamples, this.packetSeconds, this.clickLags);

/// What the encode reads: the file, or the app's joined copy of a reset.
final String decodePath;
Expand All @@ -70,6 +67,18 @@ class _Source {
/// The decoded sound, f32 mono.
final Float32List samples;

/// The decoded sound as Audio Restoration's render decodes it
/// ([MatrixCase.referenceDecoder]): the float decoder for MP2. Null when
/// that is FFmpeg's default ([samples]).
final Float32List? renderSamples;

/// How far each white frame's click sits after its picture in the source
/// itself, seconds, in [matrixFlashes] order. The delayed AVI's is 23 ms:
/// libmp3lame's priming, which AVI cannot mark, so every player plays it
/// that late. The output keeps the source's timing, so its clicks are
/// expected this far after their flashes (task #58).
final List<double> clickLags;

/// The longest audio packet, seconds.
final double packetSeconds;
}
Expand All @@ -88,10 +97,11 @@ Future<Uint8List> _payload(String path) async {
return Uint8List.fromList(r.stdout as List<int>);
}

Future<Float32List> _samples(String path) async {
Future<Float32List> _samples(String path, {String? decoder}) async {
final r = await Process.run(
WorkerHarness.ffmpegPath,
['-v', 'error', '-i', path, '-map', '0:a:0', '-ac', '1', '-f', 'f32le', '-'],
['-v', 'error', if (decoder != null) ...['-c:a', decoder], '-i', path, '-map', '0:a:0', '-ac', '1',
'-f', 'f32le', '-'],
stdoutEncoding: null,
environment: WorkerHarness.ffmpegEnv,
);
Expand Down Expand Up @@ -132,8 +142,26 @@ Future<_Source> _prepare(SourceClass s) async {
expect(info!.frameCount, matrixPictures, reason: '${s.file}: the app counts every picture');
expect(model.videoStartOffset, closeTo(s.nominalVideoStart, 0.03),
reason: '${s.file}: where the picture starts after the sound');
// Each click against its picture, on the source's own timeline: the
// container's start plus the picture's start plus n frames (not the
// decoded frame's PTS, which a delayed AVI stamps a frame late).
final fmt = await WorkerHarness.ffprobeJson(
['-v', 'error', '-show_entries', 'format=start_time', '-of', 'json', decode]);
final start = double.tryParse('${(fmt['format'] as Map)['start_time']}') ?? 0;
final beeps = await beepOnsets(decode);
final lags = [
for (final n in matrixFlashes)
beeps
.map((b) => b - (start + model.videoStartOffset + n / matrixFps))
.reduce((a, b) => a.abs() < b.abs() ? a : b),
];
for (final l in lags) {
expect(l.abs(), lessThan(0.05), reason: '${s.file}: each click near its flash (lags $lags)');
}
final render = s.renderDecoder;
return _Source(decode, model, info, model.videoStartOffset, await _payload(decode),
await _samples(decode), _longest(await _audioTimes(decode)));
await _samples(decode), render == null ? null : await _samples(decode, decoder: render),
_longest(await _audioTimes(decode)), lags);
}

VideoJob _job(MatrixCase c, _Source s) {
Expand Down Expand Up @@ -288,24 +316,27 @@ Future<void> _check(MatrixCase c, _Source s) async {
reason: '3. pictures ${i - 1}..$i are not one frame apart');
}

// 4. Every white frame has its click, within one audio packet.
// 4. Every white frame has its click, within one audio packet of where the
// source has it (its own lag after the picture, [_Source.clickLags]).
final times = await _audioTimes(out);
final tolerance = [_longest(times), s.packetSeconds].reduce((a, b) => a > b ? a : b) + 0.001;
final e = await events(out);
final want = c.expectedFlashTimes(firstPts);
expect(e.flashes, hasLength(want.length), reason: '4. white frames at ${e.flashes}');
for (var i = 0; i < want.length; i++) {
expect(e.flashes[i], closeTo(want[i], 0.5 / matrixFps), reason: '4. white frame $i');
final lag = s.clickLags[matrixFlashes.indexOf(c.trim.flashes[i])];
final at = e.flashes[i] + lag;
final near = e.beeps.isEmpty
? double.infinity
: e.beeps.map((b) => (b - e.flashes[i]).abs()).reduce((a, b) => a < b ? a : b);
: e.beeps.map((b) => (b - at).abs()).reduce((a, b) => a < b ? a : b);
final nearest = e.beeps.isEmpty
? null
: e.beeps.reduce((a, b) => (a - e.flashes[i]).abs() < (b - e.flashes[i]).abs() ? a : b);
: e.beeps.reduce((a, b) => (a - at).abs() < (b - at).abs() ? a : b);
expect(near, lessThan(tolerance),
reason: '4. the click for the flash at ${e.flashes[i].toStringAsFixed(4)} s is at '
'${nearest?.toStringAsFixed(4)} (${((nearest ?? 0) - e.flashes[i]).toStringAsFixed(4)} s; '
'one packet ${tolerance.toStringAsFixed(4)} s)');
'${nearest?.toStringAsFixed(4)} (${((nearest ?? 0) - e.flashes[i]).toStringAsFixed(4)} s '
'after it, the source\'s ${lag.toStringAsFixed(4)}; one packet ${tolerance.toStringAsFixed(4)} s)');
}

// 5. The audio packets follow on. AVI stores no duration for its first
Expand Down Expand Up @@ -335,7 +366,8 @@ Future<void> _check(MatrixCase c, _Source s) async {
final fmt = '${a['sample_fmt']}';
final raw = int.tryParse('${a['bits_per_raw_sample']}') ?? 0;
final depth = raw > 0 ? raw : (fmt.startsWith('s16') ? 16 : fmt.startsWith('s32') ? 32 : 24);
final why = _sampleExact(await _samples(out), s.samples, depth);
final ref = c.referenceDecoder == null ? s.samples : s.renderSamples!;
final why = _sampleExact(await _samples(out), ref, depth);
expect(why, isNull, reason: '6. the samples ($depth-bit): $why');
case LosslessCheck.none:
break;
Expand Down Expand Up @@ -384,25 +416,30 @@ void main() {
// picture's grid (2.0 s of MP2 is 83.3 frames) is joined with its second
// part's picture 16 ms late, so ffprobe reads 125 fps and the app analyses
// 493 pictures. Known failure: fails while the fault is there.
// The first part's sound runs 16 ms past its last picture (a DVD cell's
// may). ffmpeg's own correction joined at the sound's end, so every later
// picture sat 16 ms off the grid and ffprobe read 125 fps; the picture now
// decides the join, and the overhanging sound packet goes (task #58).
test('a reset whose sound ends off the picture grid joins on the grid', () async {
Object? failure;
try {
final service = TimestampContinuityService(
workerPath: WorkerHarness.workerPath, environment: WorkerHarness.workerEnv);
final r = await service.prepare(_resource('conf_reset_offgrid.mpg'), const Uuid().v4());
expect(r.error, isNull);
expect(r.scan!.videoResetPictures, [50]);
addTearDown(r.proxy!.delete);
final info = await FieldOrderDetector()
.getVideoInfo(r.proxy!.path, await WorkerHarness.sourceModel(r.proxy!.path));
expect(info!.frameRate, 25, reason: 'the copy\'s rate');
expect(info.frameCount, matrixPictures, reason: 'the copy\'s pictures');
} catch (e) {
failure = e;
final service = TimestampContinuityService(
workerPath: WorkerHarness.workerPath, environment: WorkerHarness.workerEnv);
final r = await service.prepare(_resource('conf_reset_offgrid.mpg'), const Uuid().v4());
expect(r.error, isNull);
expect(r.scan!.videoResetPictures, [50]);
addTearDown(r.proxy!.delete);
final info = await FieldOrderDetector()
.getVideoInfo(r.proxy!.path, await WorkerHarness.sourceModel(r.proxy!.path));
expect(info!.frameRate, 25, reason: 'the copy\'s rate');
expect(info.frameCount, matrixPictures, reason: 'the copy\'s pictures');
// Every picture one frame after the last, and each click on its flash.
final e = await events(r.proxy!.path);
expect(e.flashes, hasLength(matrixFlashes.length), reason: 'white frames at ${e.flashes}');
for (var i = 0; i < matrixFlashes.length; i++) {
expect(e.flashes[i] - e.flashes[0], closeTo((matrixFlashes[i] - matrixFlashes[0]) / matrixFps, 0.002),
reason: 'flash $i on the grid');
final near = e.beeps.map((b) => (b - e.flashes[i]).abs()).reduce((a, b) => a < b ? a : b);
expect(near, lessThan(0.013), reason: 'the click for flash $i within half a sound packet');
}
expect(failure, isNotNull,
reason: 'a known failure ($_offGridJoin) now passes: make this an ordinary test');
print(' known failure ($_offGridJoin): $failure');
});

test('a copy the container cannot hold is refused before any work (task #49)', () async {
Expand Down
24 changes: 22 additions & 2 deletions app/test/output_matrix_cases_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,13 @@ void main() {
final rows = 1 + AudioCodec.values.where((c) => c.supportsContainer(k)).length + 2;
final sources = SourceClass.values.length;
final copied = SourceClass.values.where((s) => k.copies(s.audioCodec)).length;
expected += ((rows - 1) * sources + copied) * MatrixTrim.values.length;
expected += ((rows - 1) * sources + copied) * MatrixTrim.everyCase.length;
}
for (final (k, _) in extraVideoRows) {
final copied = SourceClass.values.where((s) => k.copies(s.audioCodec)).length;
expected += (SourceClass.values.length + copied) * MatrixTrim.values.length;
expected += (SourceClass.values.length + copied) * MatrixTrim.everyCase.length;
}
expected += tailCases.length;
expect(cases, hasLength(expected));
print('${cases.length} cases');
});
Expand All @@ -50,6 +51,25 @@ void main() {
}
});

test('the tail trims are the primed Convert codecs into AVI (task #58)', () {
final tail = cases.where((c) => c.trim == MatrixTrim.tail).toList();
expect(tail.map((c) => c.id), [
'avi/convert-aac/plain/tail',
'avi/convert-libmp3lame/plain/tail',
'avi/convert-ac3/plain/tail',
'avi/restore-convert-aac/plain/tail',
]);
expect(tail.first.expectedPictures, 50);
expect(tail.first.trim.flashes, [55, 80]);
});

test('a restore of MP2 is checked against the float decode (task #58)', () {
MatrixCase find(String id) => cases.singleWhere((c) => c.id == id);
expect(find('mkv/restore/late-video/mid').referenceDecoder, 'mp2float');
expect(find('mkv/convert-flac/late-video/mid').referenceDecoder, isNull);
expect(find('mkv/restore/plain/mid').referenceDecoder, isNull);
});

test('every known failure names a real case', () {
final ids = cases.map((c) => c.id).toSet();
for (final id in knownFailures.keys) {
Expand Down
Loading
Loading