Skip to content

feat(merge)!: texts two sources spell differently are written once - #31

Merged
Ruby184 merged 1 commit into
mainfrom
feat/merge-union
Oct 17, 2026
Merged

Ruby184 merged 1 commit into
mainfrom
feat/merge-union

Conversation

@Ruby184

@Ruby184 Ruby184 commented Oct 17, 2026

Copy link
Copy Markdown
Owner

A guide merged from several sources wrote every spelling of a programme's texts: "DJ Mixes premiere" beside "Dj Mixes Premiere", "Film" beside "film", a description cut off with "…" beside the whole of it. homepi carried ~120 lines of transform to undo it, and measured thousands of such programmes a day. One commit, breaking (guides change on upgrade).

Breaking

  • A programme's texts that say the same thing are written once. A multi-source guide writes fewer titles, categories and descriptions than before; none is rewritten, only left out.
    Migration: merge: { union: 'exact' } writes every text as the sources gave them, as before.
  • ResolvedMatch.normalizeTitle is required. resolveMatch always set it; only a hand-built ResolvedMatch needs the field.

merge.union

Per field, which texts are one — within a language, with the higher-priority source's kept:

field default the same when
title, subTitle 'match' the match's normalizeTitle makes them the same (case, accents and spacing by default), so a title rule that loosens what is one broadcast loosens what is one title too
desc 'contained' one contains the other as whole words, but for case, spacing and a trailing …; the fuller one takes the shorter one's place. Krimi beside Krimiseriál o … stays
category, keyword, country 'caseless' the same but for case, spacing, hyphens and underscores, which is how a profile finds a genre (Sci-Fi and Sci Fi are one)

A field also takes 'exact', { contained: fn } (the source's own suffix taken off first, then compared as 'contained' does), or a key function of its own. union: 'exact' sets every field exact. Languages are never compared with each other; a missing tag and an empty one are the same, and tags are compared regardless of case.

merge: {
  union: {
    desc: { contained: (value) => value.replace(/\s*\(skryté titulky\)$/, '') },
    title: (value) => value.replace(/\s+[IVX]+\.?$/, '').toLowerCase(),
  },
}

Applied as each programme is written

Not as two programmes merge. Matching decides what is one broadcast by its titles, so it has to see every one: a spelling dropped when the first two sources merged would leave a third source's shifted programme with nothing to match, and it would come out as a duplicate. So:

  • mergeProgrammes, mergeInto and backfillInto keep every text, as before.
  • The guide applies the union as it writes each programme, and hands a transform the next programme with the union applied too.
  • mergeProgrammeLists(lists, strategy, match?, fillStopMs?, union?) applies it to what it returns.
  • resolveUnion(union, resolveMatch(match).normalizeTitle).apply(programme) applies it to one programme.

Settled at load

  • A misspelt field or mode, or a union that isn't a plain object, is refused as the config loads, before anything is grabbed.
  • The loaded config carries the union settled, as a ResolvedUnion whose own fields are the union it was settled from:
    • a spread or a clone of it is that plain union again;
    • serve's ETag fingerprint still sees what it says;
    • merge.union's type stays what a config writes.
  • A settled union is recognized across two installs of the package by a registered symbol, and is settled again when the match's normalizeTitle has changed.

Smaller changes

  • genreKey folds Unicode forms together, so a category written decomposed finds its genre in a profile too.
  • The text helpers moved to a leaf module, src/merge/text.ts, which removes an import cycle.
  • resolveMatch accepts null, and merge: null in a config loads as it did before.

Testing

  • New tests for every mode, priority order, languages, the three-source case, backfill and the day boundary, the transform's next, a loaded config's settled union, copies and another install's union, a stale match, serve's ETag, and every refusal.
  • Each behaviour was broken in turn to confirm a test fails.
  • 1,808 tests, passing on Node 22.18, 24.15 and 26.12.
  • The docs example is in test/readme-examples.ts and typechecks.

When two programmes merged, their titles, descriptions and categories were
unioned by the exact text, within each language. Two sources rarely write one
the same to the letter, so a guide kept every spelling: "Dj Mixes Premiere"
beside "DJ Mixes premiere", "film" beside "Film", a description cut off with
"…" beside the whole of it. homepi carried 120 lines of transform to undo it.

`merge.union` says per field which texts are one, within a language, the
higher-priority source's kept. Titles and sub-titles are the same by the
match's own `normalizeTitle`, so a config's title rule loosens both what is
one broadcast and what is one title. A description another contains as whole
words, but for case, spacing and a trailing "…", is dropped, and the one
containing it takes its place. Categories, keywords and countries are the
same but for case, spacing, hyphens and underscores, as a profile finds a
genre. A field takes 'exact', 'caseless', 'match', 'contained',
`{ contained: fn }` or a key function of its own, and `union: 'exact'` sets
every field exact.

It applies as each programme is written, not as two are merged, so matching
still sees every title and a later source's spelling still finds its
broadcast. `mergeProgrammes`, `mergeInto` and `backfillInto` keep every text,
as before; `mergeProgrammeLists` takes a union and applies it to what it
returns, and `resolveUnion` settles one into a `ResolvedUnion`, whose `apply`
writes one programme. A loaded config carries its union settled, with the
union it was settled from as its own fields, and a misspelt field or mode is
refused as the config loads, before anything is grabbed. `ResolvedMatch`
carries the `normalizeTitle` it compares by, and `genreKey` folds Unicode
forms together.

BREAKING CHANGE: a guide built from several sources writes one of two texts
that say the same thing rather than both, so it changes on upgrade: fewer
titles, categories and descriptions, none of them rewritten. Set
`merge.union: 'exact'` to write every text as the sources gave them, as
before.
@Ruby184
Ruby184 merged commit 1493c50 into main Oct 17, 2026
4 checks passed
@Ruby184
Ruby184 deleted the feat/merge-union branch October 17, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant