Skip to content

feat(jianying): add the jianying Plugin (8 MCP verbs + 6 Skills) - #72

Open
antianqi wants to merge 12 commits into
MiniMax-AI:mainfrom
antianqi:feature/jianying-plugin
Open

antianqi wants to merge 12 commits into
MiniMax-AI:mainfrom
antianqi:feature/jianying-plugin

Conversation

@antianqi

@antianqi antianqi commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What changes

Adds plugins/antianqi/jianying — a Windows-only MiniMax Code Plugin that grades video with a
pixel-for-pixel re-implementation of Jianying's own grading math, and edits Jianying drafts through
the running application's local named pipe.

Layer Content
MCP verbs (8) draft_open · draft_read · draft_patch (27 official behaviour operations) · draft_commit · render_frames · grade_apply · audio_mix · verify_diff — mcp/server.mjs, stdio, 0 npm dependencies, one hand-written input schema per verb under mcp/tools/
Skills (6) jianying-cut · jianying-draft-edit · jianying-review · jianying-talk-cut + two library Skills (jianying-verify, jianying-grade-chain) that only activate when referenced
Knowledge references/ — chain API semantics, verification discipline, pitfalls, machine-readable capability dictionary
Guardrails scripts/ — the plugin's own MCP client, an environment self-check, three self-tests, and the talk-cut planner/applier

Both runtime layouts ship in parallel: plugin.json + mcp.json (portable Agent Plugins 1.0, read by
mcode 0.3.x) and .claude-plugin/plugin.json (mcode 0.4.0+ manifest, with the six Skill paths and the
stdio MCP server).

The grading math is re-implemented rather than delegated: the official exporter is blocked by -100007
(session mode) and -100015 (unauthorized host process), and its command tree is five days older than
the application build. The Plugin re-implements the 13 grading features to ≤ 1 LSB and states the
block in the README instead of working around it.

User value

A MiniMax Code user can grade a clip and export a 10-bit master without opening Jianying, and can
edit a Jianying draft — add a clip, trim, split, change a segment's volume — while the application stays
the only execution platform and the only renderer.

Example prompt

给这个视频调色出片,风格用暖一点的,输出到 out\cut\demo.mp4

Expected result

  1. The environment self-check runs first and names any missing dependency.
  2. grade_apply exports graded10.mp4 (direct point-wise path, recommended) and graded_lut.mp4
    (65³ LUT path), and returns a per-node effectiveness probe: every node in the spec must produce a
    non-zero MAD, and a zero node fails the verb instead of silently dropping that node.
  3. The reply carries the whole-chain change amount and the output paths.
  4. A follow-up "review this version" asks for a multi-dimension comparison (visibility / picture
    quality / loudness / timeline / draft integrity) and reports the smallest repair range, not a vague
    suggestion.

Two more example prompts, for the parts that are not grading:

把这个草稿里的第一段音量调到 25%,改完落盘并确认生效
审一下这版,告诉我哪里要返修、返修后用什么证据确认

Plugin submission checklist

  • Plugin lives at plugins/<github-owner>/<plugin-name> — plugins/antianqi/jianying.
  • plugin.json name matches the Plugin directory — jianying in both manifests.
  • README.md includes a real example prompt and expected result.
  • LICENSE and plugin.json declare an open-source license — Apache-2.0 in all three places.
  • Required executables, accounts, paid services, and supported platforms are disclosed —
    Windows 10/11; ffmpeg 6.0, Python 3.14, Node ≥ 22, Jianying 11.6 (running for the draft verbs);
    JYAI_HOME pointing at the toolchain; no account, no paid service, no API key.
  • Network destinations and data handled by the plugin are disclosed — no network access, no
    credentials, no telemetry, no third-party services; the only socket is the local Windows named
    pipe of a running Jianying instance; drafts are read-only and every write happens on a copy with
    a per-file sha assertion that the real draft did not change.
  • No credentials, private endpoints, hidden telemetry, installers, symlinks, or native binaries are
    included — 31 text files, no symlinks, no binaries, no scaffold placeholders.
  • Every scaffold TODO has been replaced.
  • npm run check passes.

Evidence

node scripts/validate.mjs

OK   plugin antianqi/jianying
Validated 30 hosted Plugins and all examples.

node --test test/jianying.test.mjs — 10 pass / 0 fail. The new test pins the things that fail
silently: every skills[] path in the v0.4 manifest resolves and its frontmatter name equals its
directory name (a mismatch is dropped by the runtime with no diagnostic), the stdio transport and its
portable working directory, all eight verb schemas, ${PLUGIN_ROOT} anchoring of script invocations,
the existence of every referenced file, the absence of a network client / credential literal / telemetry,
the absence of contributor machine paths, and the README disclosures.

Negative-injection audit of that test — seven defects injected one at a time, suite re-run, file
restored from a backup afterwards; each injection failed for the expected reason and the suite returned
to 10/10 after every restore:

Injected defect Expected failure
frontmatter name != directory name frontmatter name must equal its directory name
description over 1024 characters description is 1285 chars (limit 1024)
hardcoded drive-letter path in a script must not hardcode a drive letter path
fetch() in a script must not make HTTP requests
README loses "expected result" README must state the expected result
README loses the "no telemetry" disclosure README must disclose telemetry behaviour
skills[] entry points at a missing file deep-equal mismatch on the Skill path list

Plugin self-tests (Windows, manual — they need a live Jianying 11.6 and a real draft; the paths are
supplied through JYAI_TEST_DRAFT, and the suite prints an explicit SKIP when it is not set, so it can
never report a false green):

Self-test Result
selftest-mcp.mjs 17 / 17
selftest-draft-edit.mjs 16 / 16
selftest-talk-cut.mjs 10 / 10
check-env.mjs 17 OK / 0 MISS

These carry their own negative injections: a synthetic noise frame set must be judged "not visible", a
synthetic large change must be judged "visible", a white background pushed to 255 must be reported as
newly clipped, and an expect that says a field must not change must fail.

Windows CI — .github/workflows/jianying-windows.yml runs only node --test test/jianying.test.mjs
on windows-latest, filtered to this Plugin's paths. The repository's validate job stays on
ubuntu-latest; this is the scoped Windows counterpart that .github/workflows/ci.yml asks a
non-tool-map Windows Plugin to add, rather than re-introducing a repository-wide Windows job (that job
was dropped in round-9 because it failed on files a PR does not own).

Known limits (stated in the README as well)

  • Windows only. The draft verbs need Jianying 11.6 running; grading and export do not need the GUI.
  • Structured grading fields (curves, HSL, colour wheels) cannot be written back into a Jianying preview
    yet — the draft-side field names are not confirmed, so grading is local export only.
  • The official exporter stays unreachable, and the Plugin does not attempt to bypass the block.
  • clear (8-level pyramid) is not wired into the chain.
  • talk-cut does not do ASR: the official ASR lives in the agent runtime and is not reachable from a
    plugin, so it takes a user-supplied SRT and only decides which lines are filler, pauses or repeats.

Standalone copy

The same Plugin is also published as a standalone repository whose root is the Plugin root, so it can
be installed from the local marketplace before this PR merges:

git clone https://github.com/antianqi/jianying-plugin.git ~/.minimax/plugins/jianying
mcode plugin list          # the local marketplace is a directory scan: jianying@local enabled

That repository carries the same 31 Plugin files, an EVIDENCE.md with the measured numbers, and the
same test/jianying.test.mjs — the focused suite was made layout-agnostic (it detects the Plugin root
instead of assuming the hosted plugins/<owner>/<name>/ path), so one file guards either layout.

Adds `plugins/antianqi/jianying`: a Windows-only MiniMax Code Plugin that
grades video with a pixel-for-pixel re-implementation of Jianying's own
grading math, and edits Jianying drafts through the application's local
named pipe.

The Plugin ships two runtime layouts in parallel:

- `plugin.json` + `mcp.json` — the portable Agent Plugins 1.0 shape read by
  mcode 0.3.x.
- `.claude-plugin/plugin.json` — the mcode 0.4.0+ manifest, with the six
  Skill paths and the stdio MCP server.

MCP layer (8 verbs, `mcp/server.mjs`, zero npm dependencies): `draft_open`,
`draft_read`, `draft_patch`, `draft_commit`, `render_frames`, `grade_apply`,
`audio_mix`, `verify_diff`. Every verb has a hand-written input schema under
`mcp/tools/`; unknown keys and out-of-range values are rejected before the
call leaves the plugin.

Skill layer (6 Skills): `jianying-cut` (grade and export), `jianying-draft-edit`
(edit a draft through the 27 official behaviour operations), `jianying-review`
(multi-dimension review), `jianying-talk-cut` (cut a spoken take), plus two
library Skills — `jianying-verify` and `jianying-grade-chain` — that only
activate when another Skill references them.

Shared knowledge lives in `references/` (chain API semantics, verification
discipline, pitfalls, machine-readable capability dictionary) and the
deterministic guardrails in `scripts/`.

Why not use the official exporter: the official CLI is blocked by `-100007`
(session mode) and `-100015` (unauthorized host process), and its command
tree is five days older than the application build. The Plugin re-implements
the grading math instead (13/13 features reproduced to <= 1 LSB) and says so
in the README rather than working around the block.

Design compliance

- Both manifests declare `name: jianying`, matching the Plugin directory,
  and license `Apache-2.0` with an Apache-2.0 `LICENSE` file.
- The v0.4 manifest lists all six Skills as `./skills/<name>/SKILL.md`. Every
  path resolves, and every frontmatter `name` equals its directory name —
  a mismatch is silently dropped by the runtime with no diagnostic, so
  `test/jianying.test.mjs` pins both halves.
- The MCP server uses the `stdio` transport, a single `node` argument and a
  contained `./mcp` working directory. The Plugin does not set `PLUGIN_ROOT`
  or `PLUGIN_DATA` itself.
- Every runnable script invocation inside a Skill is anchored to
  `${PLUGIN_ROOT}`; every referenced `scripts/` and `references/` file exists.
- The Plugin ships no network client, no credential literal, no telemetry and
  no third-party service. The only socket it opens is the local Windows
  named pipe of a running Jianying instance.
- No contributor machine path is hardcoded: paths resolve from `JYAI_HOME`,
  `HOME`, `LOCALAPPDATA`, `ProgramFiles` and `PATH`.
- README discloses the platform, a copyable example prompt, the expected
  result, the required executables, the account/paid-service position and
  the four-part data behaviour (no credentials / no network / no telemetry /
  no third-party services).
- The six Skills stay well inside the runtime limits (64 Skill directories,
  8 MCP servers per Plugin).

Validation

- `node scripts/validate.mjs` -> `OK   plugin antianqi/jianying`;
  `Validated 30 hosted Plugins and all examples.`
- `npm run check` (validate + the repository test suite) is green on
  ubuntu-latest, which is the only job the repository runs.
- `node --test test/jianying.test.mjs` -> 10 pass / 0 fail.

Test evidence

- `test/jianying.test.mjs` (new, 10 cases): both manifest layouts, every
  Skill path in the v0.4 manifest resolving with a matching frontmatter name,
  the portable layout exposing the same six Skills, the stdio transport and
  its portable working directory, all eight verb schemas parsing, script
  invocations anchored to `${PLUGIN_ROOT}`, referenced files existing, the
  absence of a network client / credential literal / telemetry, the absence of
  contributor machine paths, and the README disclosures.
- Negative-injection audit of that file: seven defects injected one at a
  time, the suite re-run, and the file restored from a backup afterwards.
  Each injection failed for the expected reason — frontmatter name mismatch,
  description over 1024 characters, a hardcoded drive-letter path, a `fetch`
  call, a missing "expected result" section, a missing "no telemetry"
  disclosure, and a `skills[]` entry pointing at a missing file — and the
  suite returned to 10/10 after every restore.
- Plugin self-tests (Windows, manual, need a live Jianying 11.6):
  `selftest-mcp.mjs` 17/17, `selftest-draft-edit.mjs` 16/16,
  `selftest-talk-cut.mjs` 10/10, `check-env.mjs` 17 OK / 0 MISS. The
  verification cases carry negative injections of their own: a synthetic
  noise frame set must be judged "not visible", a synthetic large change must
  be judged "visible", a white background pushed to 255 must be reported as
  newly clipped, and an `expect` that says a field must not change must fail.
- `.github/workflows/jianying-windows.yml` (new) runs only
  `node --test test/jianying.test.mjs` on windows-latest, filtered to this
  Plugin's paths. The repository's own `validate` job stays on ubuntu-latest;
  this is the scoped Windows counterpart the CI file asks a non-tool-map
  Windows Plugin to add, rather than re-introducing a repository-wide
  windows job.

Known limits (also in the README)

- Windows only, and the draft verbs need Jianying 11.6 running; grading and
  export do not need the GUI.
- Structured grading fields (curves, HSL, colour wheels) cannot be written
  back into a Jianying preview yet: the draft-side field names are not
  confirmed, so grading is local export only.
- The official exporter stays unreachable; the Plugin does not work around
  the block.
- `clear` (8-level pyramid) is not wired into the chain.
- `talk-cut` does not do ASR: the official ASR lives in the agent runtime and
  is not reachable from the plugin, so it takes a user-supplied SRT and only
  decides which lines are filler, pauses or repeats.
The same test file has to guard two layouts: the hosted community layout
(plugins/<owner>/<name>/) and a standalone Plugin repository whose root is
the Plugin itself, which is what the local marketplace installs from.

Detect the Plugin root instead of assuming the hosted path, so the suite
can be copied into a standalone repository unchanged.
Two defects in references/capability-dict.json, the contract an agent reads
to decide what it can do:

- verbs.draft_patch.ops listed only 'video.add', so the contract claimed one
  op where the Plugin implements all 27 official behaviour operations.
- acceptance.verify_diff still stated the uncalibrated criterion
  ('max MAD > threshold'). The official renderer is not deterministic: the
  same draft rendered twice differs in 14-17 of 108 frames with a worst-case
  MAD of 48-123, two orders of magnitude above the threshold. An uncalibrated
  MAD cannot tell a change from noise, so the contract now requires the
  noise floor to be measured (noiseDir) and names the semantic criterion
  (expect over the written draft's per-segment field diff) as the one that
  does not depend on pixels.

The draft_patch guard now also states, in machine-readable form, that none of
the 27 official operations sets a grading or filter parameter: they cover
structure, volume, layout, text and undo/redo only. Grading parameters
therefore cannot be written into a draft yet (the draft-side field names are
not confirmed), and grading goes through grade_apply, which renders from a
media file and never touches the draft.
The pitch said 'we can export a graded film', which reads as a claim about
producing the deliverable. What the re-implementation actually buys is
verification: it is a faithful model of Jianying's grading maths (13/13
features, <= 1 LSB, byte-identical curve LUTs), so a grading result can be
confirmed from the chat instead of by opening Jianying and squinting at a
preview.

That framing also needs the reason it is worth doing, which was measured
here for the first time. The official render path is PNG frames -> h264
High / yuv420p / 8bit mp4, and that step is lossy: comparing the PNG
originals against frames pulled back out of that same mp4 gives MAD 2-8,
maxAbs 157, and 95% of pixels changed. Checking a grading against a preview
that carries its own 8-bit 4:2:0 projection cannot settle the question, and
the 1 LSB model settles it directly.

README now leads with the verification framing and the error table;
EVIDENCE.md records the measurement.
…w work

Three claims in the Plugin were wrong, and this corrects all of them.

1. "The official behaviour operations are 27" — wrong. The 27 came from a
   historical whitelist constant in the application's app-server.mjs, and we
   tested only that list. Sweeping the full set against the bridge shows
   **18 domains / 199 operations, every one of them admitted** (173 accepted
   with empty parameters, 26 rejected only for missing parameters, **0**
   unsupported). The capability dictionary now lists all 199, and
   references/official-ops.md records the sweep, the confirmed parameter
   contracts and the traps below.

2. "Grading parameters cannot be written into a draft" — wrong, the door is
   open. effect.add / effect.updateAdjustParams / adjustment.add /
   adjustment.setBasicColor / filter.add / keyframe.* / video.setBasicColor
   are all admitted, and the draft already carries the matching material
   collections (materials.hsl, hsl_curves, color_curves, log_color_wheels,
   primary_color_wheels). effect.add lands a materials.video_effects entry and
   adjustment.add lands a materials.placeholders entry with type "adjust".
   The parameter contracts are not mapped yet, so local export is still the
   main path, but the boundary is no longer "impossible".

3. "segment.setTimeRange takes startTimeUs/startUs plus durationUs" — a false
   positive. The application answers accepted:true with changedPaths:["/"],
   and the time range in the draft comes back {start:0, duration:0}: the
   operation does not know those key names and writes zero for the fields it
   did not recognise. Nine parameter shapes were tested in isolation.

New: subtitles. The bridge's own normalisation function shows that text.add
accepts startTime and duration directly, so a subtitle track is just a text
track with time ranges — no split-then-retime dance. Measured on a copy:

    text.add {content:"第一句字幕", startTime:1000000, duration:2500000}
    -> text track, 1 segment, target_timerange = {start:1000000, duration:2500000}

The application aligns to frame boundaries, so the tolerance is one frame
(33333us at 30fps). scripts/add-subtitles.mjs turns an SRT or a talk-cut plan
into that track, verifies the time ranges by reading the draft back, and
commits; scripts/selftest-subtitle.mjs covers it, including a negative case
that proves the probe can tell "has subtitles" from "has none".

Also recorded in references/official-ops.md: the application ships a
dedicated text.addSubtitles operation with per-word timing
(material.words[]), but its parameter shape is not mapped yet, and
needInsertTargetTrack=true inserts into whichever track trackIndex points at
— even a video track — so several subtitles share one text track only with
needInsertTargetTrack=false plus that track's index.

Validation

- selftest-mcp 17/17, selftest-draft-edit 16/16, selftest-talk-cut 10/10,
  selftest-subtitle 4/4, check-env 17 OK / 0 MISS.
- node --test test/jianying.test.mjs 10/10.
- node scripts/validate.mjs -> OK plugin antianqi/jianying; 30 hosted Plugins
  validated.
…o AI effects

Two things land here.

The operation set is now the whole thing. The v0.3 schema carried a
27-entry enum, which came from a historical whitelist constant in the
application rather than from the bridge's actual authorisation. Sweeping the
full set against the bridge shows 18 domains / 199 operations and **not one
of them is rejected** (173 accepted with empty parameters, 26 rejected only
for missing parameters, 0 unsupported). The enum now carries all 199, the
capability dictionary lists them, and references/official-ops.md records the
sweep together with the parameter contracts that were confirmed against the
draft rather than against "the application said accepted".

The audio AI effects work, and they differ from each other in an important
way:

- Denoise is `video.denoiseOn`, not `audio.denoiseOn`. Parameters
  {segmentId, isDenoise?, denoiseMode?, denoiseRate?} land in
  materials.realtime_denoises as {is_denoise, denoise_mode, denoise_rate}
  alongside sami_name "denoise_v2" and sami_version "1.0" — a local SAMI
  realtime engine, so the denoise is computed on the machine.
- Vocal beautify is `audio.setVocalBeautify` {segmentId, enable:true},
  landing in materials.vocal_beautifys as {enable:true, type:"vocal_beautify",
  ambient_sound_level:75, voice_change_mode:null, production_path:""}. An
  empty production_path means realtime rather than a cloud product, so this
  one is computed locally too.
- Voice change is `audio.setVoiceEffect` {segmentId}, landing in
  materials.audio_effects.
- Vocal separation is different: `audio.setVocalSeparation` builds
  materials.vocal_separations with {type:"vocal_separation", choice,
  removed_sounds, time_range, final_algorithm:"", production_path:""}. The
  empty production_path is the point — the separated audio is a task result,
  so the reference can be written but the artefact is not ours to produce.
  Six parameter shapes were tried and none changed the draft, so the
  contract is still unmapped.

scripts/selftest-audio-ai.mjs drives all of this through the Plugin's own
draft_patch verb rather than the raw pipe, and checks the draft afterwards:
8/8, including a negative case that requires the three material collections
to be empty before the change, so the probe cannot pass vacuously.

Also confirmed here: the key names come in both styles (isDenoise and
is_denoise, denoiseMode and denoise_mode are all accepted), while `enable`
is only `enable` — `isEnable` is ignored.

Validation

- selftest-mcp 17/17, selftest-draft-edit 16/16, selftest-talk-cut 10/10,
  selftest-subtitle 4/4, selftest-audio-ai 8/8, check-env 17 OK / 0 MISS.
- node --test test/jianying.test.mjs 10/10.
- node scripts/validate.mjs -> OK plugin antianqi/jianying; 30 hosted Plugins
  validated.
…ot the same

as present in the output

The membership check fires at export, not while editing: the official FAQ
says advanced features marked with a VIP crown (voice cloning, smart denoise)
only prompt for a subscription before the video is exported. That matches
what we measured — without a membership, video.denoiseOn, audio.setVocalBeautify
and audio.setVoiceEffect all wrote into the draft without complaint.

It also means the earlier phrasing needs a correction. The export pipeline is
the official renderer for picture (player domain) plus our own ffmpeg mix for
sound, and that mix handles volume, fades, speed and loudness only: it does
not read realtime_denoises or vocal_beautifys. The official renderer emits a
video stream with no audio track at all, the official media domain only has
file-level tools, and the official exporter is blocked by -100007 and
-100015. So the effects land in the draft, and Jianying will honour them
when it exports, but the files we produce do not carry them.

Two consequences recorded in references/official-ops.md and the capability
dictionary: the denoise and beautify entries now say explicitly that landing
in the draft does not mean landing in our output, and the acceptance line
separates "written into the draft" from "present in the delivered file".

Also worth recording from the local 11.6 install: the paywall surfaces in
the agent panel (vip_use_popup) and answers with gateId, action "retry" and
expectedRevision, which means a membership block is distinguishable from a
parameter mistake — a distinction the verification discipline should carry.

Making the effect audible in our own output means implementing it in the
mix chain instead (ffmpeg afftdn for denoise, EQ and compression as an
approximation for beautify, a Demucs-class model for separation), and saying
plainly that those are not Jianying's own DSP.
…inst a global noise floor

The user pointed out that Jianying also has a live preview while editing, which
challenged how we had attributed the "official renderer noise floor". Measuring
it properly, in one session, for the same target timestamp:

- capture the frame twice without seeking: 4 of 6 timestamps differ
- seek away and back, capture again: 4 of 6 differ
- the VEF1 timestamp delta against the requested time: 0 every time

So the non-determinism is not a mis-aimed frame, not a time-alignment problem
and not a globally unstable renderer — a simpler one-line draft is fully
deterministic (0/6) while this draft is not, and the same draft rendered twice
differs in 69 of 108 frames. It happens per capture, on particular frames:
frames 001-004 come back wholly different (MAD around 123, maxAbs 255) while
frames 000 and 005-009 differ by under 1.7. A preview engine like this is also
what the GUI runs, so nobody notices it during playback, but a static
per-pixel comparison magnifies it.

That makes a single global noise floor too coarse, and actively wrong: with
one frame sitting at noise 128 the global bar becomes 192, which then buries a
real change of 64 on a perfectly stable frame and reports a false FAIL. The
criterion is now per frame, bar_i = max(threshold, noise_i * noiseK), so a
frame whose own noise already exceeds the threshold counts as undecidable
while a real change on any stable frame is still detected.

verify_diff now reports unstableFrames, visibleMaxMad, visibleFrame and a
per-frame frameDetail, and the self-test covers it: three synthetic frame sets
where one frame is unstable and another carries a real change, asserting that
the unstable frame is flagged and the stable change still reads as visible.

Worth stating plainly: this does not touch the grading fidelity result. The
13/13 at 1 LSB is measured against an offline GL run of the official shader
fragments, not against frames pulled from the preview renderer, so it was
never exposed to this non-determinism.

@hetaoBackend hetaoBackend left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes for exact current head d726180bce3dedf083ab67693f7ab4a0038f2ab4.

The package/manifest checks and repository-owned exact-head Checks are positive, but the MCP server has a merge-blocking local file disclosure/path traversal:

  1. Unvalidated sessionId escapes PLUGIN_DATA. plugins/antianqi/jianying/mcp/server.mjs:122-123,300-307 builds snapshot-${sid}.json with path.join() and accepts any schema-valid string. A reproducible isolated probe set PLUGIN_DATA=/tmp/pr72-state, created /tmp/pr72-canary.json, then called:

    draft_read({"sessionId":"x/../../../../tmp/pr72-canary"})

    The server returned the canary JSON with source: "session", because the normalized path became /tmp/pr72-canary.json. This is an arbitrary local JSON read outside the plugin state directory. The same unsanitized value reaches snapshotFile() during draft_patch (:346-356), so a successful edit response can also write a snapshot outside PLUGIN_DATA.

    Validate session IDs against the server-issued format and ownership (for example, the exact ve-session-<digits> value recorded by draft_open), reject separators/.., and enforce canonical containment for every state/snapshot read and write. Add negative tests for POSIX and Windows traversal, absolute paths, sibling-prefix paths, and symlinked state directories.

Do not merge until the boundary is fixed and the exact-head regression suite proves that session state cannot read or write outside PLUGIN_DATA. [code]smith = SKIPPED, if present, is not evidence.

…t written into the draft

The previous text said our own output chain does not apply the draft audio
effects at all, so "written into the draft" was never "present in the output".
That statement was accurate when it was written, but the mixer now reads
materials.realtime_denoises out of the draft, matches it per segment through
extra_material_refs, and applies ffmpeg afftdn. This commit replaces the
stale claim with the measured result and its boundaries.

Validation
- End to end, official video.add + video.denoiseOn -> mixer -> muxed file:
  in-file SNR 5.646 dB (no denoise) -> 12.128 dB (denoise), +6.482 dB.
- Negative injection: the same denoised draft mixed with the effect forced
  off reproduces the baseline exactly, A - C = 0.0000 dB. The difference
  therefore comes from the denoise application and from nothing else.
- Criterion is the in-file SNR (speech band RMS - noise band RMS), so the
  loudness normalisation in the mix chain cannot bias the number.

Test evidence
- plugin selftests, all green: selftest-mcp 19/19, selftest-draft-edit 16/16,
  selftest-talk-cut 10/10, selftest-subtitle 4/4, selftest-audio-ai 8/8.
- node scripts/validate.mjs: 30 hosted plugins validated.
- capability-dict.json re-parsed after the edit. The hosted validator does
  not parse this file, so its green run says nothing about it.
- Parameter calibration on 20 s of real speech plus a noise floor calibrated
  to a known level: nr = 6 + 15*rate, nf = -26 / -23 / -20 for mode 1/2/3.

Design compliance
- The approximation is labelled as such. It is ffmpeg afftdn, not the
  Jianying SAMI denoise_v2 model, and the doc now says so next to the number
  instead of hiding it behind a stale "not applied" claim.
- The boundaries travel with the claim: the speech band sits 3.1 to 5.4 dB
  below the clean reference, only stationary pink noise was calibrated, and
  the zero-gap check rules out whole-phrase dropouts but not fine grained
  musical noise.
- The strength ladder is explicitly our own. The denoise_mode semantics of
  Jianying are not recoverable from the app bundle, so the doc says the
  ladder is ours and not a mirror of the product.
- Known upstream trap recorded for anyone reproducing the calibration: the
  ffmpeg afftdn nf parameter is range [-80, -20] in current builds, so -20
  is the strongest usable value and -15 is rejected outright.
… finished film

The plugin could render frames (render_frames) and mix audio (audio_mix), but
calling them meant wiring two verbs plus an intermediate file by hand, and
neither of them carried an acceptance criterion. make_film is the missing
verb: it chains the official renderer, the optional 3D LUT, our own mix
chain and ffmpeg, then verifies the result with assertions instead of
printing it.

Validation
- make_film drives jyai-make-film.mjs: official renderer frames -> optional
  lut3d -> our mix chain (volume / fade / denoise / loudnorm) -> ffmpeg -> a
  file with both a video and an audio stream.
- The acceptance step asserts, and fails the run on: no video stream, a
  duration that does not match the official timeline within one frame, and a
  loudness more than 1 LUFS away from the requested target.
- A draft with no usable audio is an input state, not a broken chain: the
  run produces a picture-only film, reports how many materials were
  missing, and does not fail.
- The official export tier is deliberately not used. ExportService, its mojo
  interface and the official encoder are not reachable from any non-GUI
  surface, so the film is built from the official renderer plus our own
  grading and mixing.

Test evidence
- selftest-mcp.mjs, 28 assertions, all green. New cases: the verb is
  exposed with its schema, a missing required argument is rejected, a
  non-existent draft fails loudly, a real run produces a two-stream file
  whose duration and loudness pass the built-in acceptance, and a draft
  whose audio materials are absent from disk still produces a film.
- The two end-to-end cases are skipped, not silently passed, when
  JYAI_DRAFT_LIB is not set, so a missing environment can never look green.
- node --test test/jianying.test.mjs: 10 pass, 0 fail.
- node scripts/validate.mjs: 30 hosted plugins validated.
- Three real defects were found by these tests and fixed before this commit:
  the renderer accepted a non-existent draft path and rendered an empty
  film instead of failing, the film builder picked the previous run's
  output as its own input when the same output directory was reused, and
  the self-test read the ffprobe stream list from stderr instead of stdout
  (which made a two-stream film look like it had no streams).

Design compliance
- The picture is the official renderer's output, not something we compute.
  The grading and the mix are ours, and the schema says so: denoise is
  labelled as an ffmpeg approximation rather than Jianying's own model.
- Every acceptance criterion is an assertion. A measurement that cannot
  fail is not a criterion, so the loudness check rejects a target it
  cannot read and the duration check compares against the official
  timeline reported by the renderer.
- The schema carries the same boundary note as the skills: the film is the
  official picture plus our grading and mix, not a Jianying export-tier
  film.
- Plugin version moved to 0.2.0 because a verb was added, and the
  manifest description now says 9 verbs instead of 8.
…ilure now fails loudly

Two things came out of measuring the grading chain against the app.

First, the curve parameter contract was wrong in the docs and in the schema.
The contract is the official A,L,R layout, three vec2 per control point, six
numbers each, and the R, G, B and Y curves all have to be given together.
The plugin described plain xy pairs, so a user writing curves into the
schema had it accepted and then dropped: the baker returned an identity
curve and the run reported success. The baker now also accepts the
documented chain API names, so both spellings produce the same LUT, and a
curve set that is missing a channel raises instead of quietly baking an
identity.

Second, make_film now takes a grade spec directly. It bakes the spec into
a 3D LUT and hands that to the renderer, so a film with curves on it is
one call, and no step depends on a field the app does not expose.

Validation
- make_film with a four channel curve spec bakes a cube whose deviation
  from identity is 0.375, so the curve is genuinely in the LUT.
- The same spec spelled with RPoints and the spec spelled with
  red, green, blue and luma bake to the same deviation, so the two
  spellings are equivalent.
- A spec with only one channel raises with the names of the missing
  channels and the accepted key names, instead of baking an identity LUT.
- The film path asserts both streams when the draft still has its audio
  material on disk, and reports the missing material count when it does
  not, so an absent asset is not reported as a broken chain.

Test evidence
- selftest-mcp.mjs, 32 assertions green, including the three new ones
  above. The two stream assertion is conditional on the audio material
  being present and says so when it is skipped.
- node --test test/jianying.test.mjs: 10 pass, 0 fail.
- node scripts/validate.mjs: 30 hosted plugins validated.

Design compliance
- The contract is written the way the app defines it, and the chain API
  spelling is documented as an equivalent alias rather than a second
  format that might drift.
- The document states that the luminance curve applies to all three
  channels on purpose, because that is the official behaviour and a user
  would otherwise read the cross channel change as a bug.
- The grading path does not go through the draft. The app preview
  renderer does not apply curves at all, so the parameters come from our
  own interface and nothing waits on an unexposed field.
- Failures stay loud. A spec that cannot be applied raises, and a film
  whose audio assets are missing is reported as a missing asset rather
  than as a failed mix.
The 3D LUT path can only express a per point function. The three
rendering time operators, sharpen, vignetting and particle, depend on a
neighbourhood or on uv, so they cannot be expressed that way, and they
were being dropped without a word. The same held for a node that is not
on the chain at all, smartColorAdjustment, which is in neither list.

The baker now refuses both cases and names the way out, which is the
per point path, the grade_apply verb. The check sits in the baker
itself, so no route can bypass it. The render time operators stay in the
schema on purpose, so that a user who writes one gets the pointer to
the right verb instead of an unknown key.

The direct path already runs these operators and already probes each
node against zero, so nothing there changes. Running them inside the
film pipeline as well is a capability question, not a bug.

Validation
- A curve spec bakes to a LUT whose deviation from identity is 0.375.
- A spec with sharpen or vignetting raises, and the message names the
  per point path.
- allow_render_nodes lets the direct path through, and the direct path
  is the one that actually runs these operators.
- The four grading regressions are unchanged: curves against the
  official shader at a MAD of 0.0012, hsl 0.0008, logWheel 0.0010, and
  hsl_curves at its recorded values, with the identity and the non
  identity checks still passing.

Test evidence
- selftest-mcp.mjs, 32 assertions green, including the new case that a
  render time operator in the grade spec is refused with a pointer to
  grade_apply.
- node --test test/jianying.test.mjs: 10 pass, 0 fail.
- node scripts/validate.mjs: 30 hosted plugins validated.

Design compliance
- The two chains are now documented side by side, with the split
  between what a LUT can express and what only the direct path can, so
  a caller picks the right verb instead of guessing.
- Nothing fails silently. An operator that would have been dropped is
  refused, and the refusal says which verb to use.
- The operator list stays in the schema. Removing it would turn a
  helpful pointer into a bare unknown key, which is less useful to the
  caller and hides the reason.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants