diff --git a/.claude/rules/changelog.md b/.claude/rules/changelog.md index b57a3bb6b3..06f688bd2a 100644 --- a/.claude/rules/changelog.md +++ b/.claude/rules/changelog.md @@ -1,16 +1,20 @@ --- paths: - "CHANGELOG.md" + - "changelog.d/**/*" + - "towncrier.toml" --- -# CHANGELOG +# Changelog -[Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/). Entries go under `[Unreleased]`. +Unreleased changes are towncrier fragments in `changelog.d/`. A release turns them into a `CHANGELOG.md` section with `scripts/release/notes.py assemble` (`RELEASING.md`, Notes). -- **Sections**, each at most once per version and in this order: `Added`, `Changed`, `Deprecated`, `Removed`, `Fixed`, `Security`. -- **An entry is one line naming the change**, not a sentence explaining it, under about 120 characters. Added and Changed name the new thing (`Open in Window for row inspector text fields.`); Fixed names the bug, not the fix (`Empty context menu when right-clicking a read-only inspector field.`). No file paths or type names; an issue reference goes at the end, `(#1234)`. +- **One file per user-visible change**, named `changelog.d/..md`: `` is the pull request that adds it, `` is `added`, `changed`, `deprecated`, `removed`, `fixed` or `security`. A second entry of the same type in one PR adds `.`: `3452.fixed.2.md`. +- **One line, at most 120 characters, ending with a period.** Added and Changed name the new thing (`Open in Window for row inspector text fields.`); Fixed names the bug, not the fix (`Empty context menu when right-clicking a read-only inspector field.`). No file paths or type names. +- **No refs, no credit, no `- ` prefix.** towncrier links the PR named by the file and adds the bullet; `notes.py` adds ` by @login` when the PR's author is not a maintainer. - **One entry per thing a user would notice.** Defects with the same visible symptom share an entry. -- **No entry** for a fix to something still unreleased (fold it into its entry) or for a docs-only change. -- **Credits are added at release** by `scripts/ci/changelog_credits.py`: `(#PR by @user)` for outside contributors, `(#PR)` for maintainers. Never write the credit by hand, and reword a contributor's entry in their own PR so the credit stays theirs. -- **A version may open with a lead block** of at most six lines before its first `###`, naming what a reader would notice. The update window and the Sparkle feed show only that block. -- After any edit, `grep -n '^## \[' CHANGELOG.md` must still list every released heading. +- **No fragment** for a fix to something still unreleased (edit that fragment instead) or for a docs-only change. +- **Only a `chore(release): vX.Y.Z` PR edits `CHANGELOG.md`.** +- **The lead block**: at most 6 lines, X.Y.0 only, passed as `notes.py assemble --lead`. The update window and the Sparkle feed show only that block. +- **`[CRITICAL]`** on a heading only for a security fix or data loss, never on a beta. +- `scripts/ci/check-changelog.py` checks the fragments and the headings in Repo Hygiene, then `towncrier build --draft --version 0.0.0` renders them. In PR title, `--pr` checks that a PR into main names each new fragment after itself and that only a release PR edits `CHANGELOG.md`. diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index a97e6be490..7b78a38646 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -2,33 +2,40 @@ paths: - ".github/workflows/**/*" - ".github/actions/**/*" - - ".github/scripts/**/*" - ".github/actionlint.yaml" - ".github/*baseline*" - "scripts/ci/**/*" + - "scripts/release/**/*" - ".swiftlint.yml" - "**/.swiftlint.yml" --- # CI -The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. The release and plugin builds: `RELEASING.md`. +The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. Release tags: `RELEASING.md`. -## Rulesets and required checks +## Rulesets +| Ruleset | Refs | Rules | Bypass | +|---|---|---|---| +| 25308039 "tags never move" | tags `v*`, `libs-*` | update, deletion | none | +| 23959123 "release tags are created by admins" | tags `v*`, `libs-*`, `plugin-*` | creation | admins | +| 25308049 "release branches are created by admins" | `release/*` | creation | admins | +| 23957966 "main and release branches require green gates" | default branch, `release/*` | the five required checks, `do_not_enforce_on_create` | none | +| 25129799 "branches keep their history" | default branch, `release/*`, `feeds` | deletion, non-fast-forward | none | + +- `feeds` takes no required checks: CI pushes it with `GITHUB_TOKEN`, and the signed feed proves its content. - There are no gate jobs. A test job runs on `!cancelled() && needs.changes.outputs.run != 'false'`. A job its `if:` skips reports success, and a failed changes job runs the suite rather than skipping it. - `Changes (macOS)`, `Changes (iOS)` and `Changes (Release)` are not required checks. - A PR need not be up to date with main to merge (`strict_required_status_checks_policy: false`). -- A second ruleset blocks force-push and deletion of `main`, with no bypass. - A required check's name is its job's `name:`. Renaming a job changes the ruleset in the same change, and every open PR then merges main in before its checks report the new name. ## Runners and toolchain -- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. `build-plugin.yml` picks its runner from the ref it builds (`RELEASING.md`). +- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. A stable tag builds only on a GA Xcode: `setup-xcode` with `require-release` fails on a beta or Release Candidate. - The Xcode version is written once, in `.github/actions/setup-xcode/action.yml`, which runs `xcode-select --switch` and prints `xcodebuild -version`. A workflow never selects Xcode another way. - XcodeGen comes from `.github/actions/setup-xcodegen`, pinned by version and SHA-256. - `scripts/release/feed.py` runs Sparkle's `generate_appcast` and `sign_update` from `SPARKLE_BIN`, which `.github/actions/setup-sparkle` sets. The action reads the version and revision from the app's `Package.resolved` (`project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. -- The tag release (`build.yml`) signs the feed with the copy `scripts/ci/sign-and-appcast.sh` downloads, pinned by its own `SPARKLE_VERSION` and `SPARKLE_SHA256`. No check compares them with `project.yml`: a Sparkle bump changes all three. - The compilation cache is off on CI (`COMPILATION_CACHE_ENABLE_CACHING=NO`); `Configs/Base.xcconfig` turns it on locally. ## macOS Tests (`macos-tests.yml`) @@ -55,10 +62,16 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Push to main or `release/**`, and dispatch; never a PR. Each commit has its own concurrency group and is never cancelled. Not a required check. - `Changes (Release)` outputs `run` (detector `release`). `Release Build` runs on `xcode-27` with a 150-minute limit. - Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both`, `build-hana.sh both` and create-dmg. -- `scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set (the `.github/plugin-registry.json` entries not bundled) to `build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. +- `scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. - `scripts/release/package.sh app 0.0.0-dev --unsigned` writes the release assets: the DMG, the `.tar.xz` (extracted and compared with the app) and the dSYMs zip; `package.sh plugins` writes one zip per plugin and the plugin dSYMs zip. - The plugins build runs after an app failure too. The xcodebuild logs in `build/release/logs` upload only on failure. +## Release, Withdraw and Pages + +- A `v*` tag runs only `release.yml`; its jobs: `RELEASING.md`, What a tag runs. `withdraw.yml` runs only by dispatch, and `pages.yml` runs from both or by dispatch. Their environments allow only `v*` tags, so a dispatch names one with `--ref`. +- Jobs read the tag, version, build and channel only from `verify` outputs passed through `env:`, never through `${{ }}` inside `run:`. +- The default permission is `contents: read`, and each job lists every scope it needs. A job that calls `pages.yml` grants `pages: write` and `id-token: write`, since a called workflow can only lower what it receives. + ## Summaries (`scripts/ci/summarize-xcresult.sh`) - Runs `scripts/lib/xcresult-cases.py summary`. `verify.sh` reads its verdicts through the same parser, and `scripts/lib/test_xcresult_cases.py` tests it in Repo Hygiene. @@ -77,7 +90,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## PR title (`pr-title.yml`) - Runs on opened, edited, reopened and synchronize, so every head commit gets the check. -- Runs `scripts/ci/check-pr-title.py`, then `scripts/check-banned-words.sh` on the PR's added lines and on its body. The banned-words step runs both and exits with the worse result. +- Runs `scripts/ci/check-pr-title.py`, then `scripts/ci/check-changelog.py --pr` with `GH_TOKEN` (`pull-requests: read`), then `scripts/check-banned-words.sh` on the PR's added lines and on its body. The banned-words step runs both and exits with the worse result. ## Repo Hygiene (`repo-hygiene.yml`) @@ -85,14 +98,15 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Steps, in order: - actionlint, which also shellchecks every inline `run:` block; - `shellcheck --severity=warning` on every `*.sh` under `scripts` and `.claude/skills`; - - the tooling tests: `scripts/ci/test_*.py`, `scripts/lib/test_*.py`, `scripts/test_localization.py`, `.github/scripts/test_update_registry.py` and two shell tests; + - the tooling tests: `scripts/ci/test_*.py`, `scripts/lib/test_*.py`, `scripts/test_localization.py` and `scripts/ci/test_libs_release.sh`; - `scripts/release/test_*.py` on Python 3.9 from `actions/setup-python`, in a venv that installs `scripts/release/requirements.txt` with `pip --require-hashes`; + - in the same venv, `check-changelog.py` and `towncrier build --draft --version 0.0.0`, which fails on a fragment name it cannot parse; - `check-lib-sourcing.py`, `check-plugin-manifest.py`, `check-plugin-provides.py`, `check-deployment-target.sh`, `check-ios-shared-isolation.py`; - `localization.py plugins` and `plugins --target ios`; - `check-string-catalogs.py`, `check-license-inventory.py`, `check-entitlements.py`, `check-test-plans.py`; - `check-pluginkit-bump-cadence.py`; - `scripts/audit-refactor-health.sh --check`, `check-hub-ratchet.py`; - - the Go checks on the HANA helper, when the `hana` detector is true. + - the Go checks on the HANA helper, `govulncheck` among them, when the `hana` detector is true. New advisories between changes arrive as Dependabot alerts. - `audit-refactor-health.sh --check` fails a file defining a production `DatabaseType` outside `Packages/TableProCore/Sources/TableProCoreTypes/DatabaseType.swift`, unless `.github/duplicate-contract-baseline.txt` lists its path. Moving a listed file moves its baseline path in the same commit. Never add an entry to let a new copy pass. - `check-hub-ratchet.py` fails when an app hub grows past `.github/hub-ratchet-baseline.json`. `--update` only lowers a value, so growing one is a hand edit a reviewer sees. @@ -106,22 +120,20 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - `oracle` watches `Packages/TableProOracle/`; `hana` watches `Native/HanaBridge/`. Each suite also watches its own workflow. - `release` watches what the Release build reads: the app, plugin, package, native, config and libs folders, `project.yml`, the app's `Package.resolved`, `.github/plugin-registry.json`, `scripts/release/`, the build and CI scripts it runs and both setup actions. It skips `Plugins/*Tests/` and `Packages/*/Tests/`. - A path a suite newly reads joins that suite's pattern in the same change, or the suite skips on it. -- Release-only files never trigger a suite: `CHANGELOG.md`, `appcast.xml`, `TablePro/Resources/WhatsNew.md`, `.github/release-flags.json`, and `Configs/Version.xcconfig` while its diff touches only the two version keys. -- Every file the release commit stages (`.claude/skills/release/SKILL.md`, Commit, tag, push) is on that list; `appcast.xml` is `build.yml`'s own push. -- A release push that touches anything else runs the suites, and the tag waits for them. +- `CHANGELOG.md` and `changelog.d/` are on no suite's pattern: the What's New phase writes nothing at `0.0.0-dev`, and Repo Hygiene renders every fragment and section instead. ## What runs where - A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title, plus Release Build; never cancelled, `AllPlugins` always, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or release-only push skips the suites through the detector. -- A `v*` tag runs only `build.yml`; a `plugin-*-v*` tag runs only `build-plugin.yml` (`RELEASING.md`). The test workflows never run on a tag. -- `workflow_dispatch` exists on every workflow except `pr-title.yml`. +- A push to main or `release/**`: the same minus PR title, plus Release Build; never cancelled, `AllPlugins` always, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or changelog-only push skips the suites through the detector. +- A `v*` tag runs only `release.yml`. The test workflows never run on a tag. +- `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. - `build-linux.yml` is on `main` only for its Monday schedule, which GitHub runs from the default branch. Until the Linux app's directory lands on `main`, every build job skips and `weekly-dispatch` starts the workflow on the branch the `LINUX_BRANCH` variable names (the linux branch when unset), whose testing guide describes the jobs. ## Caches and runner budget - Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `build.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other tags. +- `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other cache: no libs or DerivedData cache, and none in the iOS, plugin or Ubuntu workflows. - CI fits the GitHub Free and Pro plans: no paid larger runner, no self-hosted Mac. At most 5 macOS jobs run at once, and main runs queue behind PR runs for those slots. diff --git a/.claude/rules/data-sync-storage.md b/.claude/rules/data-sync-storage.md index f6a31d2270..d9de4aa06b 100644 --- a/.claude/rules/data-sync-storage.md +++ b/.claude/rules/data-sync-storage.md @@ -14,6 +14,7 @@ paths: This path holds user data. A silent wrong answer here looks exactly like an empty database, so prove late-completion and cancellation behavior with a test. - **A synced CKRecord type or field reaches CloudKit Production before anything writes it.** Both apps pin the container to Production, where nothing is auto-created, and CloudKit rejects a whole record that carries an undeclared field. Write fields only through `record.fields(SomeSyncField.self)`, never `record["..."]`. To ship one: add it in Development, deploy to Production in the Console, run `scripts/export-cloudkit-schema.sh`, commit `CloudKit/production-schema.ckdb`, then add it to the matching `verifiedInProduction`. `ProductionSchemaParityTests` checks code against the snapshot in both directions, not against the server, so never edit the snapshot by hand: `scripts/export-cloudkit-schema.sh --check` compares it with Production. +- **A storage format stays readable by the previous stable.** Sparkle never downgrades, and a beta user can turn betas off (`RELEASING.md`, Channels). - **Persist, then notify.** `SyncChangeTracker.markDeleted()` runs after the storage write, because its notification can start a sync that re-uploads the record from the stale file. - **Concurrent schema loads await the one in-flight `loadTask`** in `SQLSchemaProvider`; never guard with a flag that returns without data. - **Every metadata read goes through `DatabaseManager.withMetadataDriver`**, never `MetadataConnectionPool.shared` directly, so an embedded engine with `supportsConnectionPooling = false` (DuckDB, PGlite) stays on its session driver; a second connection to it is a different, empty database. diff --git a/.claude/rules/native-bridges.md b/.claude/rules/native-bridges.md index 9345067f85..a77c344421 100644 --- a/.claude/rules/native-bridges.md +++ b/.claude/rules/native-bridges.md @@ -12,4 +12,4 @@ paths: - **First-party bridge code lives in `Native/Bridge`**, built by `scripts/build-.sh`, with output in a gitignored folder beside the source, never in `Libs/`. - **Rust may run in the app process; Go may not.** The Dameng staticlib is built with `panic = "unwind"`, and every export that does driver work wraps it in `catch_unwind`; a new export that does work takes the same guard. Go's `recover()` cannot catch a panic on a goroutine go-hdb starts, so HANA runs out of process as `tablepro-hana-helper`, one per driver session. - **Toolchains and upstream code are pinned**: `rust-toolchain.toml`, `Cargo.lock` with `--locked`, the `toolchain` line of `go.mod`, `go.sum`. A protocol change for Dameng lands on the [TableProApp/rust-dameng](https://github.com/TableProApp/rust-dameng) fork first, then the pinned `rev` moves. -- **Every nested Mach-O is signed with Developer ID, the hardened runtime and a timestamp** before the main binary; `scripts/build-plugin.sh` does it and checks it with `codesign -dvvv`. +- **Every nested Mach-O is signed with Developer ID, the hardened runtime and a timestamp** before the main binary. The `HanaHelperEmbedding` copy phase in `project.yml` signs the HANA helper with the plugin's identity, and `scripts/release/build.sh plugins` checks every executable with `codesign -dvvv`. diff --git a/.claude/rules/plugin-system.md b/.claude/rules/plugin-system.md index adac0eb785..7b7d2090bf 100644 --- a/.claude/rules/plugin-system.md +++ b/.claude/rules/plugin-system.md @@ -8,7 +8,7 @@ paths: - "Packages/*/Package.swift" - "project.yml" - "TableProMobile/project.yml" - - ".github/workflows/build-plugin.yml" + - "Configs/PluginKit.xcconfig" - ".github/plugin-registry.json" - "scripts/*plugin*" --- @@ -24,10 +24,10 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke - Add a requirement only with a default implementation, then bridge it in `PluginDriverAdapter`. - Additive: a new requirement with a default, a new field on a non-`@frozen` struct, a new case on a non-`@frozen` enum, a reordering. - Breaking: a changed, removed or renamed requirement or signature, a requirement without a default, a new case on a `@frozen` enum, a change to a frozen type's layout. -- Every ABI change bumps `TABLEPRO_PLUGINKIT_VERSION` in `Configs/PluginKit.xcconfig`, at most once per release cycle: later changes in the cycle reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. +- Every ABI change bumps `TABLEPRO_PLUGINKIT_VERSION` in `Configs/PluginKit.xcconfig`, at most one past the newest `vX.Y.Z` tag: later changes reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. A `release/*` line never bumps it. - The build stamps that value into `TableProPluginKitVersion` of the app and of every plugin. A plugin's `Info.plist` holds `TABLEPRO_PLUGINKIT_VERSION` and is preprocessed, because every shipped app reads the key with `as? Int`; the source file therefore fails `plutil -lint`. -- An additive change re-publishes nothing. Built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. -- A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`, and every registry plugin is re-released with `scripts/release-all-plugins.sh ` before the app tag, because the release's `Registry Readiness` job fails while a registry driver has no binary in the app's kit range (`RELEASING.md`, The release pipeline). +- After an additive change, built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. +- A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`. Every tag ships every registry plugin built against its own kit, so the release that carries the change carries plugins in its range. - Check with `verify.sh abi ` (`scripts/check-pluginkit-abi.sh`). It builds PluginKit at HEAD and at the base with the same toolchain and diffs the public `.swiftinterface`. A base with no interface passes; a diff prints the additive and breaking rules and exits 1. - It refuses uncommitted changes, so run it after the commit and before the push. No workflow runs it. - The diff is textual, so a `-` line can be a declaration that moved. Only a removed line with no matching added line, or a requirement added without a default, is breaking. This lists the removed lines that do not come back: @@ -65,14 +65,13 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke ## Structure - PluginKit-free logic a plugin shares with the iOS app lives in a TableProCore product, such as `TableProMySQLCore`, `TableProPostgreSQLCore` or `TableProRedisCore`. -- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, an entry in `.github/plugin-registry.json` keyed by its tag slug, a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), its `TableProProvides*` keys (below), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a CHANGELOG entry. +- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, for a registry one an entry in `.github/plugin-registry.json` keyed by its target, with `name` and `iconName` (a bundled one stays out of it), a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), its `TableProProvides*` keys (below), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a changelog fragment. - A new engine ships as a registry plugin (`AGENTS.md`, Scope). - `PluginManager` registers a plugin from the `TableProProvidesDatabaseTypeIds`, `TableProProvidesExportFormatIds` and `TableProProvidesImportFormatIds` arrays in its `Info.plist` and loads its code on first use. A plugin with none of them loads at launch. The first database type id is the primary one. - `scripts/ci/check-plugin-provides.py` checks those arrays against the class's `databaseTypeId`, `additionalDatabaseTypeIds` and `formatId` in Repo Hygiene; `--write` stamps them. - Where a plugin's tests go, and how a test target is added in `project.yml`: `.claude/rules/tests.md`, Where a test goes and Adding a test target. -## Tags and shipped apps +## Releases -- Plugin tags are `plugin--v`, the slug a key in `.github/plugin-registry.json`. Check `git tag -l "plugin-*"` before creating one. -- A driver fix reaches users of a shipped app through `scripts/release-plugin-for-shipped-app.sh [appTag]`. Never relabel a binary built from `main` with an older kit version. -- How a plugin tag builds and publishes: `RELEASING.md`, Plugins. +- Registry plugins have no tags of their own: each app tag builds and publishes all of them at the app's version (`RELEASING.md`, Plugins). +- A driver fix reaches stable users through a patch on `release/`. diff --git a/.claude/rules/theme.md b/.claude/rules/theme.md index cd68b0cde5..2ed78301bb 100644 --- a/.claude/rules/theme.md +++ b/.claude/rules/theme.md @@ -17,7 +17,7 @@ paths: - The editor tab strip track draws `secondarySystemFill` over a themed backdrop on macOS 26 and later, where the track sits on the theme's window color, and its unthemed fill otherwise. Never a palette color. - **A code field takes the editor theme wherever it appears.** Startup Commands, Pre-Connect Script, a favorite's query, Rewind, Agent SQL, the deep-link Startup SQL and the tunnel command preview wear `editor.*`; the form or sheet around them stays system. - **The slot family follows what the surface shows.** Code, values, value trees and wells: `editor.*`, with tree tones from `editor.syntax.*`. Records, schema objects, diagram nodes, chart plots and states that stand in for a grid: `dataGrid.*`. Diff rows: `dataGrid.inserted`, `modified` and `deleted` over the editor background and text. A region behind panes: `window.background`. Identity, severity and status hues stay system. -- **Every site passes what it draws natively.** A themed call site names a slot and, as its fallback, the exact system color or style it drew before, so while the active theme leaves the slot out the pixels are an unthemed app's. A default changes only on purpose, named in the PR and the CHANGELOG. +- **Every site passes what it draws natively.** A themed call site names a slot and, as its fallback, the exact system color or style it drew before, so while the active theme leaves the slot out the pixels are an unthemed app's. A default changes only on purpose, named in the PR and in a changelog fragment. ## The model: `TableProTheme` diff --git a/.claude/skills/fix-issue/SKILL.md b/.claude/skills/fix-issue/SKILL.md index a32517cf71..7074c58ed2 100644 --- a/.claude/skills/fix-issue/SKILL.md +++ b/.claude/skills/fix-issue/SKILL.md @@ -41,7 +41,7 @@ Pick the platform from the report: the issue's Platform field, a `platform-*` la Invoking the skill authorizes the whole run: investigate, branch, implement, verify, commit, push, open the PR and keep it green. There is no approval gate. Ask only when the expected behavior has two reasonable readings or a finding needs a product decision: once, recommendation first, and keep working on what does not depend on the answer. Never ask about the environment (a peer's build, a wedged test runner, a full disk): handle it and mention it in the report. -Never, even in an authorized run: force-push, rewrite published history, touch release tags, publish plugins or libraries, deploy a CloudKit schema, or discard changes you did not make. When the fix needs one of these, build everything else and name what is left for the user. +Never, even in an authorized run: force-push, rewrite published history, touch release tags or `release/*` branches, publish plugins or libraries, deploy a CloudKit schema, or discard changes you did not make. When the fix needs one of these, build everything else and name what is left for the user. ## Phase 0: Intake @@ -79,7 +79,7 @@ Write `/.analysis//plan.md`, under 80 lines, opening with a progress - the design: the API or pattern, its ownership boundary, the design guideline; - the scope as a checklist: files in order, blast-radius cases, each verified related defect with `file:line`; - the tests that fail before the fix, in the targets the profile's Done section says to check first; -- the CHANGELOG, docs, localization and screenshots the change needs. +- the changelog fragment, docs, localization and screenshots the change needs. **Rules read.** List the files `plan.md` touches. Read `/AGENTS.md`, then every `/.claude/rules/*.md` whose row in its Area rules table matches one of them, and name them on a `Rules read:` line in `plan.md`. A file added later gets the same check before you edit it. The tree's copies are the ones that apply; a harness may have loaded another checkout's. @@ -90,7 +90,7 @@ In Full mode the `critic` lane attacks the plan; fold in what survives an eviden - When the change is visible on screen, build the fresh tree and capture the "before" shots first. - Follow the plan's order and do the refactor it calls for. Apply the guide's per-change checklist while you write, not afterwards. - **A large change may be split across implementers**: when the plan spans areas that share no files, give up to six subagents a disjoint file list each in `plan.md`. Each prompt names the tree, `plan.md` and the exact files the implementer owns, and says to edit nothing else, build nothing, and finish with the files it changed. You read every diff, integrate and verify. The harness file says which agent runs them. -- Competitors are research only: never name another client in code, commits, the PR, CHANGELOG or docs. +- Competitors are research only: never name another client in code, commits, the PR, a changelog fragment or docs. ## Phase 4: Verify diff --git a/.claude/skills/fix-issue/references/lanes.md b/.claude/skills/fix-issue/references/lanes.md index 2ca17404be..bf358364f5 100644 --- a/.claude/skills/fix-issue/references/lanes.md +++ b/.claude/skills/fix-issue/references/lanes.md @@ -56,7 +56,7 @@ For each one, read the cited code and its surroundings, and check that nothing u Attack `plan.md`. Report weaknesses, not a summary; a sound part gets one line. Cover three lenses: 1. **Patterns**: where the design fights an existing pattern, helper or convention in the repo (cite it), or breaks an invariant in the guide or its area rules. -2. **Scope**: callers that break, state or persistence that goes stale, and inputs the plan skips (empty, null, duplicate names, sentinel values, very large results, cancellation). Also a CHANGELOG or docs claim that says more than the change does. +2. **Scope**: callers that break, state or persistence that goes stale, and inputs the plan skips (empty, null, duplicate names, sentinel values, very large results, cancellation). Also a changelog fragment or docs claim that says more than the change does. 3. **Decision**: whether refactor-or-patch is right, whether a better documented API exists (name it), and whether the rejected alternative was rejected for a real reason. Each objection gets a severity (`blocking`, `material` or `minor`), evidence (`file:line`, an API ground-truth line or measured output, since reasoning alone is not evidence), and the smallest change to the plan that answers it. Read the files you cite: a confident wrong objection costs more than a missed one. diff --git a/.claude/skills/fix-issue/references/platform-apple.md b/.claude/skills/fix-issue/references/platform-apple.md index 4461e56466..40ab83906b 100644 --- a/.claude/skills/fix-issue/references/platform-apple.md +++ b/.claude/skills/fix-issue/references/platform-apple.md @@ -57,4 +57,5 @@ The macOS app and the iOS app, with everything they build from. Every profile ha ## Ship - Stage a string catalog only for the keys you added. +- A fix for a bug on the current beta or stable gets the label `backport release/`. - A probe worth keeping goes under `scripts/probes/`. diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index b1505356df..e1ab0c8136 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -1,261 +1,87 @@ --- name: release description: > - Ships a TablePro release end to end: bumps Configs/Version.xcconfig, finalizes CHANGELOG.md, - commits, tags, pushes and releases plugins. Use whenever the user says "release", "bump version", - "ship version", "tag a release", "cut a release", or gives a version number ("/release 0.67.0", - "/release plugin-oracle 1.0.0"). Not for the blog post, newsletter or X posts: those are the - announce skill in TableProApp/web. + Cuts a TablePro macOS release: a beta, a stable or a patch. Picks fixes onto a release/X.Y line, + curates the changelog fragments and the lead, decides whether a release is critical, opens the + release PR, hands the maintainer the tag command, watches the run and forward-ports the release. + Use whenever the user says "release", "cut a beta", "ship version", "tag a release", "promote the + beta", "backport", or gives a version number ("/release 0.82.0", "/release 0.82.0-beta.2"). Not for + the blog post, newsletter or X posts (the announce skill in TableProApp/web), and not for the iOS app. --- # Release -- You never push to this repository's `main`. Every push made with the release key (the release - commit, a withdrawal) is the user's to run in their own terminal, because that key never enters - ssh-agent (`RELEASING.md`, Keys and rulesets). Wait for them to confirm it landed. -- Tags are always annotated, `-a -m`: the owner's git has `tag.gpgSign=true`, and a plain - `git tag ` fails with "no tag message?". -- The release commit holds exactly the four files that Commit, tag, push stages. -- Once a release run is past **Publish the release**, only **Re-run failed jobs** - (`RELEASING.md`, When a release run fails). +`RELEASING.md` is the runbook and holds every command. This skill holds the judgment: each section is a decision, then names the runbook section that carries it out. -Run the stages in order, starting one only when the one before it is real: 1 release (always), 2 -plugins (when they changed). Announcing it is the `announce` skill in TableProApp/web -(`../tablepro-web`). +## 1. Ground rules -House rules relied on, not repeated: `.claude/rules/changelog.md`, `.claude/rules/plugin-system.md`, -the commit format in the top comment of `.github/pull_request_template.md`, and `RELEASING.md` for -the pipeline. +- Never push a tag, never create or delete a `release/*` branch, never push to `main` or `release/*`. The maintainer runs every block you hand over, in their own terminal, and tells you when it landed. +- A tag never moves. A fix after a tag is the next number. +- Release PRs and backport PRs go through the five required checks like any other PR. -## Stage 1: Release +## 2. Read the state -### Pre-flight +Before any decision, gather: -Check each. If one fails, stop and say what is wrong. +- the tags, newest first: `git -c versionsort.suffix=-beta tag -l 'v*' --sort=-v:refname | head`; +- the live lines: `git ls-remote --heads origin 'release/*'`; +- open regressions: `gh issue list --label regression --state open`; +- open Dependabot alerts: `gh api repos/TableProApp/TablePro/dependabot/alerts -q '.[] | select(.state == "open") | .security_advisory.summary'`; +- main's push runs: `gh run list --branch main --event push --limit 10`, with macOS Tests, iOS and Package Tests and Release Build green on the commit you would cut; +- whether `main` has the last stable's `## [X.Y.Z]` heading. If not, the forward-port is missing and the next tag will refuse to build: do that first (section 9). -1. The version is semver (`X.Y.Z`; a suffix like `-beta.1` is fine), newer than `MARKETING_VERSION` - in `Configs/Version.xcconfig`, and its tag is free (`git tag -l "v"`). Ask if missing. -2. The tree is clean (`git status --porcelain`), or the user says the changes belong in the - release. `[Unreleased]` has entries; if not, say the release has no notes. -3. `swiftlint lint --strict` is clean: the tag's checks do not cover SwiftLint. Fix findings - through their own pull request. -4. If `Plugins/TableProPluginKit/` changed since the last tag, do Stage 2, PluginKit, now, before - the tag. -5. `scripts/export-cloudkit-schema.sh --check` passes; it needs the cktool management token on this - Mac. A field in the snapshot but not in Production makes CloudKit reject every record carrying - it, so the user deploys it in CloudKit Console before the tag. -6. After `git fetch origin main`, you are on `main` and it equals `origin/main`. Report both suite - verdicts and stop when either is a failure. What the tag's `Tested on main` job checks on top of - this: `RELEASING.md`, The release pipeline. +## 3. Decide what ships - ```bash - for pair in 'macos-tests.yml|macOS Tests' 'ios-tests.yml|iOS and Package Tests'; do - for id in $(gh run list --workflow "${pair%%|*}" --branch main --event push --limit 20 \ - --json databaseId -q '.[].databaseId'); do - verdict=$(gh run view "$id" --json headSha,jobs \ - -q ".headSha[0:9] + \" \" + ([.jobs[] | select(.name == \"${pair#*|}\") | .conclusion][0] // \"\")") - case "$verdict" in *' ' | *skipped) continue ;; esac - echo "${pair#*|}: $verdict (run $id)" - break - done - done - ``` +- **beta.1 of X.Y**: `main` has features since the last X.Y.0. +- **beta.N**: the line has picks since its last beta. +- **Promote to X.Y.0**: the newest beta has no open `regression` issue. +- **Patch X.Y.Z**: the line has fixes only since its last stable. +- **X.0.0**: ask the maintainer. -7. The release key exists: `test -f ~/.ssh/tablepro_release`. If not, stop and give the user both - commands to run, the first with a passphrase: +Say which and why in one line, then wait for the maintainer to agree. Run: `RELEASING.md`, Cutting a beta, or Cutting a stable or patch. - ```bash - ssh-keygen -t ed25519 -f ~/.ssh/tablepro_release -C "release commit push" - gh repo deploy-key add ~/.ssh/tablepro_release.pub --allow-write --title "Release commit push" -R TableProApp/TablePro - ``` +## 4. Storage check -### Credit the entries +Before beta.1 and before a promotion, list the commits that touch user data since the previous stable: `git log --oneline v.. --` followed by every `paths:` entry of `.claude/rules/data-sync-storage.md`. Read each one for a changed storage or sync format. Sparkle never downgrades, so a format the previous stable cannot read blocks the release until it is fixed on `main`. Run: `RELEASING.md`, Channels. -Do this before rewording anything: the script reads each entry's commit with `git blame`, and a line -edited in the working tree blames to nothing. +## 5. Critical or not -```bash -python3 scripts/ci/changelog_credits.py --dry-run | tail -5 -python3 scripts/ci/changelog_credits.py -``` +Critical is only a security fix or data loss, never a beta. Name the fix and the reason, and let the maintainer confirm. A critical stable takes `--critical` on `notes.py assemble`. Run: `RELEASING.md`, Notes. -An outside contributor's entry ends `(#2905 by @digows)`, a maintainer's ends `(#2905)`. The output -ends with every contributor and every entry left bare. A bare entry came from a direct push: credit -it from `git log` or leave it, and never guess a handle. Blame credits the last commit to touch a -line, so check the contributors against `gh pr list --state merged --search "merged:>=" --json author` -and restore any handle a later rewording took. The script is idempotent. +## 6. Backport candidates -### Tighten the entries +- Candidates: merged `main` PRs labeled `backport release/`, plus `fix` PRs for a bug the line has. +- Never a `feat`, and after `X.(Y+1).0` ships, only security or data-loss fixes. +- Already picked: `gh pr list --base release/ --state merged --limit 200`. A pick's title keeps the original subject and its ` (#N)`. +- One fix per pick PR, from `backport/-`. A conflict that needs more than a mechanical resolution goes back to the maintainer. -Entries arrive one PR at a time and drift long. Measure them without their credit, then look for -the repair-story tells: +Run: `RELEASING.md`, Branches. -```bash -awk '/^## \[Unreleased\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep '^- ' \ - | sed -E 's/ \(#[0-9, #]+( by @[A-Za-z0-9-]+)?\)$//' \ - | awk '{ t+=length; n++; if (length>120) o++ } - END { if (!n) { print "no entries"; exit } print n" entries, avg "int(t/n)" chars, "o+0" over 120" }' -awk '/^## \[Unreleased\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep -nE '^- .*( now | no longer |, so )' -``` +## 7. Curate -Bare `length` is the line's length. Never write a dollar sign followed by a digit in this file: -Claude Code replaces it with a skill argument, so `/release 0.77.0` ran `length(0.77.0)`. +- Compare what merged with the fragments: `gh pr list --state merged --base main --search "merged:>=" --limit 1000 --json number,title,files`. A user-visible PR with no fragment needs one; a fragment that says more than its PR did needs rewording. +- Add or reword fragments through a PR to `main`, then pick it to the line. Never edit `CHANGELOG.md` by hand. +- For X.Y.0, write the lead per `.claude/rules/changelog.md`: at most 6 plain lines, what a reader would change their behavior over, in the reader's words. A release of fixes only can go without one. +- Check every fragment against `.claude/rules/changelog.md` and `scripts/check-banned-words.sh`. -`instead of` alone is fine when it describes the bug. Rewrite what runs long or matches to -`.claude/rules/changelog.md`, merging entries for one change and keeping every credit as it is. Diff -the `#` references and handles before and after to prove none were dropped. +Run: `RELEASING.md`, Notes, then Cutting a stable or patch. -### Bump the version +## 8. Watch -In `Configs/Version.xcconfig`, set `MARKETING_VERSION` to the new version and add 1 to -`CURRENT_PROJECT_VERSION`. Touch no other version: plugin and PluginKit versions live in `project.yml`, -and the iOS app's in `Configs/Version-iOS.xcconfig`. The release job checks both values -(`RELEASING.md`, The release pipeline). +- Follow the run with `gh run list --workflow release.yml --limit 3` and `gh run watch `. +- Sort a failure into infrastructure (runner, network, notary service, an expired certificate) or code (a build, test or check the tag's commit fails). Infrastructure is a re-run; code is a fix on `main`, a pick, and the next number. +- Confirm the result from the source, not the cached site: `gh api -H 'Accept: application/vnd.github.raw' 'repos/TableProApp/TablePro/contents/appcast.xml?ref=feeds' | grep -c ''`, and `gh release view v --json assets -q '.assets | length'`. -### Finalize CHANGELOG.md +Run: `RELEASING.md`, What a tag runs, and When a run fails. -1. Add `## [] - ` below `## [Unreleased]`, leaving Unreleased empty. In the - footer, point `[Unreleased]` at `v...HEAD` and add - `[]: https://github.com/TableProApp/TablePro/compare/v...v`. -2. Each `###` heading appears once, in the order Added, Changed, Deprecated, Removed, Fixed, - Security. Merge a repeated heading into the first and move an out-of-order block, then check - nothing was lost: +## 9. After a stable or patch - ```bash - awk '/^## \[\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep '^### ' - awk '/^## \[\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep -c '^- ' - grep -n '^## \[' CHANGELOG.md | head -5 - ``` +- Open the forward-port PR to `main` (and to a newer open line), and the docs changelog PR. +- For an X.Y.0, name the `announce` skill in TableProApp/web as the next step. +- Before the next stable, check `retainFromPluginKit` against the license dashboard. -3. Write the lead block: two or three lines under the version heading, before the first `###`, - naming the `### Added` items a reader would change their behavior over, in the reader's words. - The update window and the Sparkle feed show only this block. A release of pure fixes can skip it. -4. Regenerate the in-app notes so **Help > What's New** matches the feed: - `scripts/generate-whats-new.sh `. +Run: `RELEASING.md`, Cutting a stable or patch, and Plugins. -### Docs changelog +## 10. Withdraw -Add an `` block at the top of `changelog.mdx` in the docs repository (`../tablepro-docs`, -TableProApp/docs; confirm the path and `git pull --ff-only` first), right after the frontmatter. The -docs are English only, so there is no translated copy. - -```mdx - - ### New Features - - - **Feature Name**: what it does for the reader - -``` - -The headings are New Features, Improvements, Removed, Security and Bug Fixes, in that order, each -only when the version has one. Each line is one CHANGELOG entry written for a reader; a New Features -line may add one sentence on what the reader does with it. An outside contributor's line ends -`, by [@user](https://github.com/user)`; a maintainer's carries no credit. - -### Critical update flag - -`.github/release-flags.json` holds `criticalUpdate`. Leave it `false`: updates install in the -background and apply on quit, and the flag is what lets Sparkle interrupt the user. Set it `true` -only when the release fixes one of these, and say which: - -- Data loss or corruption of a database, saved connections or persisted tabs. -- A security issue with a `### Security` entry that users of the old build are exposed to. -- A crash or hang on a common path, or a failure to launch, connect or update. -- A regression from the previous release with no workaround. - -Nothing else; a handful a year at most. Set it in the release commit and back to `false` in the -next. A retag of the same release never gets it. A bad build is withdrawn (Withdraw a release) and -superseded by a corrective release with the flag. What the flag makes Sparkle do, and the release -job's check for a leftover flag: `RELEASING.md`, Staged rollout and critical updates. - -### Commit, tag, push - -The release key skips every check, so before the user pushes with it, prove the push carries the -release commit and nothing else; stop if the check prints nothing. Push the docs changelog in the -same sitting: the docs CI fails while the app's `CHANGELOG.md` carries a version the page does not. -It goes straight to the docs `main` under the admin bypass, and Mintlify deploys it. - -```bash -git add Configs/Version.xcconfig CHANGELOG.md .github/release-flags.json TablePro/Resources/WhatsNew.md -scripts/check-banned-words.sh --staged -git commit -m "release: v" -git fetch origin main -[ "$(git branch --show-current)" = main ] \ - && [ "$(git rev-parse HEAD^)" = "$(git rev-parse origin/main)" ] \ - && [ -z "$(git diff --name-only HEAD^ HEAD | grep -vxE 'CHANGELOG\.md|Configs/Version\.xcconfig|TablePro/Resources/WhatsNew\.md|\.github/release-flags\.json')" ] \ - && echo "only the release commit" -git tag -a v -m "v" -# The user runs this one: -GIT_SSH_COMMAND='ssh -i ~/.ssh/tablepro_release -o IdentitiesOnly=yes' git push origin main -git push origin v -# Then the docs changelog: -git -C ../tablepro-docs add changelog.mdx -git -C ../tablepro-docs commit -m "docs(changelog): v" -git -C ../tablepro-docs push origin main -``` - -- Keep the subject exactly `release: v`. -- Keep the release commit to the four files above. All four are on the change detector's - release-only list (`.claude/rules/ci.md`, Change detector), so the push skips both suites and the - tag takes the verdict of the code it ships. Any other file runs the suites on that push, and the tag waits for - them. A lint fix made on the way lands first, through its own PR. - -### Confirm it shipped - -A pushed tag is not a build: the workflow publishes from its last job. Check the run, the release -assets and the feed: - -```bash -gh run list --workflow build.yml --limit 1 -gh release view v --json tagName,assets -q '"\(.tagName) assets=\(.assets|length)"' -curl -s https://raw.githubusercontent.com/TableProApp/TablePro/main/appcast.xml | grep -o '[^<]*' | head -2 -``` - -That raw URL is the app's `SUFeedURL`; `tablepro.app/appcast.xml` does not exist. `shortVersionString` -is an element, not an attribute. A failed run: `RELEASING.md`, When a release run fails. - -### Withdraw a release - -Only when a build loses data, fails to launch or breaks connecting, and the fix is more than a few -minutes away. Commands and effects: `RELEASING.md`, Withdrawing a release; the user runs its push. -Users already on the build are reached by the corrective release with `criticalUpdate` set. - -## Stage 2: Plugins - -After the app tag is pushed, list the registry plugins with commits since their last tag: - -```bash -python3 .claude/skills/release/scripts/changed-plugins.py -``` - -Slugs, tag names and targets come from `.github/plugin-registry.json`, so nothing is mapped by hand. -`own` counts commits in the plugin's source paths (its folder, plus a package or `Native/` bridge it -builds from), `kit` counts PluginKit commits. Skip `bundled` rows: their changes ship inside the -app, which never applies a registry update to a bundled plugin. A row with only `kit` commits needs -a release only after a breaking bump, through `release-all-plugins.sh`. - -Show the user what changed and ask before tagging, suggesting a patch bump from the last tag. Tag -each approved plugin annotated (top of this file). The tag is the version, so a plugin needs no -version bump or changelog edit. - -```bash -git tag -a plugin--v -m "plugin--v" -git push origin plugin--v -``` - -- One tag per push: GitHub sends no push event for a push carrying more than three tags, so no - build starts. If several went at once, run `gh workflow run build-plugin.yml -f tags=","`. -- Wait for the app build first (`gh run list --workflow build.yml --limit 1`): plugin builds compete - with its `Build` jobs and `Create GitHub Release` for the same macOS runner slots - (`.claude/rules/ci.md`, Caches and runner budget). Then confirm a run started for each tag with - `gh run list --workflow build-plugin.yml --limit 10`. -- A published plugin release cannot take a rebuilt binary, and a rebuild never matches the - published bytes. A plugin run that fails after its release is published ships as the next patch. - -### PluginKit - -If the release touched `Plugins/TableProPluginKit/`, run `scripts/check-pluginkit-abi.sh v` -before publishing anything, and act on the diff as `.claude/rules/plugin-system.md` (ABI) says. -`scripts/release-all-plugins.sh` runs for a breaking bump only: each run uses a registry retention -slot for every plugin. One driver fix reaches users who have not updated through -`scripts/release-plugin-for-shipped-app.sh plugin--v v`. +Only when a build loses data, fails to launch or breaks connecting, and the fix is more than minutes away. The fix ships as the next number, critical when data was lost. Run: `RELEASING.md`, Withdrawing a release. diff --git a/.claude/skills/release/scripts/changed-plugins.py b/.claude/skills/release/scripts/changed-plugins.py deleted file mode 100644 index 4db1c613c9..0000000000 --- a/.claude/skills/release/scripts/changed-plugins.py +++ /dev/null @@ -1,109 +0,0 @@ -#!/usr/bin/env python3 -"""Lists every registry plugin with its last tag and the commits since it. - -The slug, the tag name and the target come from .github/plugin-registry.json. The source paths -come from the target in project.yml: its `folder:` under Plugins/, every local package it depends -on except TableProCore, which every plugin shares, and every Native/ bridge it or its templates -name. -Nothing here restates the plugin list. - -Usage: - python3 .claude/skills/release/scripts/changed-plugins.py [--all] - -Prints only plugins with commits of their own or PluginKit commits since their last tag, unless ---all is given. Run it from anywhere inside the TablePro repo. -""" - -import argparse -import json -import re -import subprocess -import sys -from pathlib import Path - -KIT = "Plugins/TableProPluginKit/" - - -def git(root, *args): - return subprocess.run( - ["git", "-C", str(root), *args], capture_output=True, text=True, check=True - ).stdout.strip() - - -def repo_root(): - try: - return Path(git(Path(__file__).resolve().parent, "rev-parse", "--show-toplevel")) - except subprocess.CalledProcessError: - sys.exit("changed-plugins.py: not inside a git checkout") - - -def blocks(project_yml): - sections = {"targetTemplates": {}, "targets": {}} - section = name = None - for line in project_yml.read_text(encoding="utf-8").splitlines(): - top = re.match(r"^(\w+):", line) - if top: - section = sections.get(top.group(1)) - name = None - continue - header = re.match(r"^ ([A-Za-z0-9]+):\s*$", line) - if section is not None and header: - name = header.group(1) - section[name] = [] - elif section is not None and name: - section[name].append(line) - return sections["targetTemplates"], sections["targets"] - - -def source_paths(root, target, templates, targets): - lines = targets.get(target) - if lines is None: - sys.exit(f"changed-plugins.py: no target {target} in project.yml") - text = "\n".join(lines) - for used in re.findall(r"templates:\s*\[([^\]]*)\]", text): - for template in (t.strip() for t in used.split(",")): - text += "\n" + "\n".join(templates.get(template, [])) - folder = re.search(r"^\s+folder:\s*(\S+)", text, re.M) - if folder is None: - sys.exit(f"changed-plugins.py: target {target} has no folder in project.yml") - paths = [f"Plugins/{folder.group(1)}/"] - for package in re.findall(r"-\s*package:\s*(\S+)", text): - if package != "TableProCore" and (root / "Packages" / package).is_dir(): - paths.append(f"Packages/{package}/") - paths += [f"Native/{bridge}/" for bridge in re.findall(r"Native/(\w+)", text)] - return sorted(set(paths)) - - -def last_tag(root, slug): - tags = git(root, "tag", "-l", f"plugin-{slug}-v*", "--sort=-version:refname") - return tags.splitlines()[0] if tags else None - - -def commit_count(root, since, paths): - revision = f"{since}..HEAD" if since else "HEAD" - return int(git(root, "rev-list", "--count", revision, "--", *paths)) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--all", action="store_true", help="list unchanged plugins too") - args = parser.parse_args() - - root = repo_root() - plugins = json.loads((root / ".github/plugin-registry.json").read_text())["plugins"] - templates, targets = blocks(root / "project.yml") - - print(f"{'slug':<20} {'bundled':<8} {'last tag':<34} own kit paths") - for slug, entry in sorted(plugins.items()): - paths = source_paths(root, entry["target"], templates, targets) - tag = last_tag(root, slug) - own = commit_count(root, tag, paths) - kit = commit_count(root, tag, [KIT]) - if not args.all and own == 0 and kit == 0: - continue - bundled = "yes" if entry["bundled"] else "no" - print(f"{slug:<20} {bundled:<8} {tag or 'never tagged':<34} {own:>3} {kit:>3} {' '.join(paths)}") - - -if __name__ == "__main__": - main() diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 177e2acf0f..667292e7aa 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -54,9 +54,9 @@ body: attributes: label: TablePro version description: >- - About TablePro shows it. On macOS, updates install in the background, so this is often not the newest. - On Linux, `tablepro --version` prints it too. - placeholder: "0.74.0" + About TablePro shows it, with the build number in parentheses and any -beta.N. On macOS, updates install + in the background, so this is often not the newest. On Linux, `tablepro --version` prints it too. + placeholder: "0.82.0 (820099)" validations: required: true diff --git a/.github/actions/apple-signing/action.yml b/.github/actions/apple-signing/action.yml index b8abb77a9b..ba9f90e1bb 100644 --- a/.github/actions/apple-signing/action.yml +++ b/.github/actions/apple-signing/action.yml @@ -1,5 +1,5 @@ name: Apple signing -description: Imports the Developer ID certificate, stores notary credentials, and installs the provisioning profile. +description: Imports the Developer ID certificate, stores the notary credentials as the TablePro profile, and installs the provisioning profile. inputs: certificate-p12: @@ -8,19 +8,19 @@ inputs: certificate-password: description: Password for that .p12. required: true - apple-id: - description: Apple ID the notary submissions are made under. - required: true - team-id: - description: Apple Developer team identifier. - required: true - notary-password: - description: App-specific password for notarytool. - required: true provisioning-profile: description: Base64 of the provisioning profile. Omit to skip installing one. required: false default: "" + asc-key-id: + description: Key ID of the App Store Connect API key notarytool authenticates with. + required: true + asc-issuer-id: + description: Issuer ID of that key. + required: true + asc-key-p8: + description: Contents of that key's .p8 file. + required: true runs: using: composite @@ -45,18 +45,31 @@ runs: security set-key-partition-list -S apple-tool:,apple: -k "" "$KEYCHAIN_PATH" security list-keychain -d user -s "$KEYCHAIN_PATH" login.keychain - - name: Configure notarization + certificate="$RUNNER_TEMP/certificate.pem" + security find-certificate -c "Developer ID Application" -p "$KEYCHAIN_PATH" > "$certificate" + expiry="$(openssl x509 -noout -enddate -in "$certificate" | sed 's/^notAfter=//')" + if ! openssl x509 -noout -checkend $((60 * 24 * 3600)) -in "$certificate" > /dev/null; then + echo "::warning::The Developer ID Application certificate expires on $expiry" + fi + rm -f "$certificate" + echo "Developer ID Application certificate valid until $expiry" + + - name: Store the notary credentials shell: bash env: - APPLE_ID: ${{ inputs.apple-id }} - APPLE_TEAM_ID: ${{ inputs.team-id }} - NOTARY_PASSWORD: ${{ inputs.notary-password }} + ASC_KEY_ID: ${{ inputs.asc-key-id }} + ASC_ISSUER_ID: ${{ inputs.asc-issuer-id }} + ASC_KEY_P8: ${{ inputs.asc-key-p8 }} + # notarytool copies the key into the keychain item, so the file can go at once. run: | set -euo pipefail - xcrun notarytool store-credentials "TablePro" \ - --apple-id "$APPLE_ID" \ - --team-id "$APPLE_TEAM_ID" \ - --password "$NOTARY_PASSWORD" + key="$RUNNER_TEMP/AuthKey_$ASC_KEY_ID.p8" + (umask 077 && printf '%s\n' "$ASC_KEY_P8" > "$key") + status=0 + xcrun notarytool store-credentials "TablePro" --key "$key" --key-id "$ASC_KEY_ID" \ + --issuer "$ASC_ISSUER_ID" || status=$? + rm -f "$key" + exit "$status" - name: Install provisioning profile if: inputs.provisioning-profile != '' @@ -66,5 +79,17 @@ runs: run: | set -euo pipefail profiles="$HOME/Library/MobileDevice/Provisioning Profiles" + decoded="$RUNNER_TEMP/profile.plist" mkdir -p "$profiles" - echo "$PROVISIONING_PROFILE" | base64 --decode > "$profiles/tablepro.provisionprofile" + echo "$PROVISIONING_PROFILE" | base64 --decode > "$RUNNER_TEMP/tablepro.provisionprofile" + security cms -D -i "$RUNNER_TEMP/tablepro.provisionprofile" > "$decoded" + name="$(plutil -extract Name raw -o - "$decoded")" + uuid="$(plutil -extract UUID raw -o - "$decoded")" + expiry="$(plutil -extract ExpirationDate raw -o - "$decoded")" + mv "$RUNNER_TEMP/tablepro.provisionprofile" "$profiles/$uuid.provisionprofile" + rm -f "$decoded" + # Both are ISO 8601 in UTC, so they compare as strings. + if [[ "$expiry" < "$(date -u -v+60d +%Y-%m-%dT%H:%M:%SZ)" ]]; then + echo "::warning::The provisioning profile '$name' expires on $expiry" + fi + echo "Installed the provisioning profile '$name' ($uuid), valid until $expiry" diff --git a/.github/actions/setup-xcode/action.yml b/.github/actions/setup-xcode/action.yml index 24e0226134..803891e043 100644 --- a/.github/actions/setup-xcode/action.yml +++ b/.github/actions/setup-xcode/action.yml @@ -1,12 +1,41 @@ name: Select Xcode description: Selects the one Xcode every TablePro build and test job uses. +inputs: + require-release: + description: true fails the job unless the selected Xcode is a GA release, as a stable build requires. + required: false + default: "false" + # The only place the version is written. runs-on names the image (xcode-27); this picks the build # inside it, the one developers install locally. The image has no Xcode 26. runs: using: composite steps: - shell: bash + env: + XCODE: /Applications/Xcode_27.1.app + REQUIRE_RELEASE: ${{ inputs.require-release }} run: | - sudo xcode-select --switch /Applications/Xcode_27.1.app + set -euo pipefail + sudo xcode-select --switch "$XCODE" xcodebuild -version + [ "$REQUIRE_RELEASE" = true ] || exit 0 + + # The image links Xcode_.app to whichever build it carries, a beta or RC included. + app="$(cd "$XCODE" && pwd -P)" + case "$(basename "$app")" in + *_beta* | *_Release_Candidate*) + echo "::error::$XCODE is $app, not a GA Xcode; a stable release builds only on a GA Xcode" + exit 1 + ;; + esac + license="$app/Contents/Resources/LicenseInfo.plist" + type="$(plutil -extract licenseType raw -o - "$license" 2> /dev/null || true)" + case "$(tr '[:upper:]' '[:lower:]' <<< "$type")" in + *beta*) + echo "::error::$app has the license type '$type', not a GA Xcode; a stable release builds only on a GA Xcode" + exit 1 + ;; + esac + echo "$app is a GA Xcode" diff --git a/.github/plugin-registry.json b/.github/plugin-registry.json index 78d112ef4e..5455fb8da1 100644 --- a/.github/plugin-registry.json +++ b/.github/plugin-registry.json @@ -1,489 +1,109 @@ { - "$comment": [ - "The release metadata for every plugin published to the registry, keyed by the slug in its", - "git tag (plugin--v). This is data the repo owns nowhere else: the summary,", - "the category and the homepage exist only here, and the slug-to-target mapping has no", - "derivation rule (mssql -> MSSQLDriver, cloudflare-d1 -> CloudflareD1DriverPlugin).", - "", - "'icon' and 'databaseTypeIds' do exist elsewhere, in the plugin's own DriverPlugin class,", - "and scripts/ci/check-plugin-manifest.py fails if the two disagree. Seven icons here were", - "already wrong when that check was first run.", - "", - "The 7 plugins absent from this file are the ones bundled inside the app and never published", - "to the registry: MySQL, PostgreSQL, CSV/JSON/SQL export, CSV/JSON import.", - "", - "'bundled' marks a plugin that also ships inside the app. Those ten can still be published", - "individually, but the app never applies a registry update to a bundled plugin, so a published", - "build reaches only users who install its zip from a file. A bulk ABI re-release skips them,", - "because their binaries ride with the next app release." - ], + "retainFromPluginKit": 39, "plugins": { - "beancount": { - "target": "BeancountDriver", - "bundleName": "BeancountDriver", - "bundleId": "com.TablePro.BeancountDriver", - "bundled": false, - "displayName": "Beancount Driver", - "summary": "Read-only Beancount ledger driver using user-provided rledger or Python Beancount", - "databaseTypeIds": [ - "Beancount" - ], - "icon": "beancount-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/beancount" + "BeancountDriver": { + "name": "Beancount Driver", + "iconName": "beancount-icon" }, - "bigquery": { - "target": "BigQueryDriverPlugin", - "bundleName": "BigQueryDriverPlugin", - "bundleId": "com.TablePro.BigQueryDriverPlugin", - "bundled": false, - "displayName": "BigQuery Driver", - "summary": "Google BigQuery analytics database driver via REST API", - "databaseTypeIds": [ - "BigQuery" - ], - "icon": "bigquery-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/bigquery" + "BigQueryDriverPlugin": { + "name": "BigQuery Driver", + "iconName": "bigquery-icon" }, - "bigtable": { - "target": "BigtableDriverPlugin", - "bundleName": "BigtableDriverPlugin", - "bundleId": "com.TablePro.BigtableDriverPlugin", - "bundled": false, - "displayName": "Bigtable Driver", - "summary": "Google Cloud Bigtable driver over the REST API", - "databaseTypeIds": [ - "Bigtable" - ], - "icon": "bigtable-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/bigtable" + "BigtableDriverPlugin": { + "name": "Bigtable Driver", + "iconName": "bigtable-icon" }, - "cassandra": { - "target": "CassandraDriver", - "bundleName": "CassandraDriver", - "bundleId": "com.TablePro.CassandraDriver", - "bundled": false, - "displayName": "Cassandra Driver", - "summary": "Apache Cassandra and ScyllaDB driver via DataStax C driver", - "databaseTypeIds": [ - "Cassandra", - "ScyllaDB" - ], - "icon": "cassandra-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cassandra" + "CassandraDriver": { + "name": "Cassandra Driver", + "iconName": "cassandra-icon" }, - "clickhouse": { - "target": "ClickHouseDriver", - "bundleName": "ClickHouseDriver", - "bundleId": "com.TablePro.ClickHouseDriver", - "bundled": true, - "displayName": "ClickHouse Driver", - "summary": "ClickHouse OLAP database driver via HTTP interface", - "databaseTypeIds": [ - "ClickHouse" - ], - "icon": "clickhouse-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/clickhouse" + "CloudflareD1DriverPlugin": { + "name": "Cloudflare D1 Driver", + "iconName": "cloudflare-d1-icon" }, - "cloudflare-d1": { - "target": "CloudflareD1DriverPlugin", - "bundleName": "CloudflareD1DriverPlugin", - "bundleId": "com.TablePro.CloudflareD1DriverPlugin", - "bundled": false, - "displayName": "Cloudflare D1 Driver", - "summary": "Cloudflare D1 serverless SQLite-compatible database driver via REST API", - "databaseTypeIds": [ - "Cloudflare D1" - ], - "icon": "cloudflare-d1-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cloudflare-d1" + "CloudflareR2SQLDriverPlugin": { + "name": "Cloudflare R2 SQL Driver", + "iconName": "cloudflare-r2-sql-icon" }, - "cloudflare-r2-sql": { - "target": "CloudflareR2SQLDriverPlugin", - "bundleName": "CloudflareR2SQLDriverPlugin", - "bundleId": "com.TablePro.CloudflareR2SQLDriverPlugin", - "bundled": false, - "displayName": "Cloudflare R2 SQL Driver", - "summary": "Read-only Cloudflare R2 SQL driver for Iceberg tables in R2 Data Catalog via REST API", - "databaseTypeIds": [ - "Cloudflare R2 SQL" - ], - "icon": "cloudflare-r2-sql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cloudflare-r2-sql" + "DamengDriver": { + "name": "Dameng Driver", + "iconName": "cylinder" }, - "dameng": { - "target": "DamengDriver", - "bundleName": "DamengDriver", - "bundleId": "com.TablePro.DamengDriver", - "bundled": false, - "displayName": "Dameng Driver", - "summary": "Dameng DM8 database driver via a native wire client", - "databaseTypeIds": [ - "Dameng" - ], - "icon": "cylinder", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/dameng" + "DuckDBDriver": { + "name": "DuckDB Driver", + "iconName": "duckdb-icon" }, - "duckdb": { - "target": "DuckDBDriver", - "bundleName": "DuckDBDriver", - "bundleId": "com.TablePro.DuckDBDriver", - "bundled": false, - "displayName": "DuckDB Driver", - "summary": "DuckDB analytical database driver", - "databaseTypeIds": [ - "DuckDB" - ], - "icon": "duckdb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/duckdb" + "DynamoDBDriverPlugin": { + "name": "DynamoDB Driver", + "iconName": "dynamodb-icon" }, - "dynamodb": { - "target": "DynamoDBDriverPlugin", - "bundleName": "DynamoDBDriverPlugin", - "bundleId": "com.TablePro.DynamoDBDriverPlugin", - "bundled": false, - "displayName": "DynamoDB Driver", - "summary": "Amazon DynamoDB: index-aware browsing, typed editing, PartiQL, the DynamoDB API and table management", - "databaseTypeIds": [ - "DynamoDB" - ], - "icon": "dynamodb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/dynamodb" + "ElasticsearchDriverPlugin": { + "name": "Elasticsearch Driver", + "iconName": "elasticsearch-icon" }, - "elasticsearch": { - "target": "ElasticsearchDriverPlugin", - "bundleName": "ElasticsearchDriverPlugin", - "bundleId": "com.TablePro.ElasticsearchDriverPlugin", - "bundled": false, - "displayName": "Elasticsearch Driver", - "summary": "Elasticsearch driver via the REST API with a Query DSL console", - "databaseTypeIds": [ - "Elasticsearch" - ], - "icon": "elasticsearch-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/elasticsearch" + "EtcdDriverPlugin": { + "name": "etcd Driver", + "iconName": "etcd-icon" }, - "etcd": { - "target": "EtcdDriverPlugin", - "bundleName": "EtcdDriverPlugin", - "bundleId": "com.TablePro.EtcdDriverPlugin", - "bundled": false, - "displayName": "etcd Driver", - "summary": "etcd v3 key-value store driver with prefix-tree browsing and lease management", - "databaseTypeIds": [ - "etcd" - ], - "icon": "etcd-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/etcd" + "FirebaseDriverPlugin": { + "name": "Firebase Driver", + "iconName": "firebase-icon" }, - "firebase": { - "target": "FirebaseDriverPlugin", - "bundleName": "FirebaseDriverPlugin", - "bundleId": "com.TablePro.FirebaseDriverPlugin", - "bundled": false, - "displayName": "Firebase Driver", - "summary": "Cloud Firestore and Firebase Realtime Database driver over their REST APIs", - "databaseTypeIds": [ - "Firestore", - "Firebase Realtime Database" - ], - "icon": "firebase-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/firebase" + "HanaDriver": { + "name": "SAP HANA Driver", + "iconName": "cylinder" }, - "hana": { - "target": "HanaDriver", - "bundleName": "HanaDriver", - "bundleId": "com.TablePro.HanaDriver", - "bundled": false, - "displayName": "SAP HANA Driver", - "summary": "SAP HANA SQL driver via SAP/go-hdb", - "databaseTypeIds": [ - "SAP HANA" - ], - "icon": "cylinder", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/sap-hana" + "KafkaDriverPlugin": { + "name": "Kafka Driver", + "iconName": "kafka-icon" }, - "kafka": { - "target": "KafkaDriverPlugin", - "bundleName": "KafkaDriverPlugin", - "bundleId": "com.TablePro.KafkaDriverPlugin", - "bundled": false, - "displayName": "Kafka Driver", - "summary": "Apache Kafka topic and message browser over the native wire protocol", - "databaseTypeIds": [ - "Kafka" - ], - "icon": "kafka-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/kafka" + "LibSQLDriverPlugin": { + "name": "libSQL / Turso Driver", + "iconName": "libsql-icon" }, - "libsql": { - "target": "LibSQLDriverPlugin", - "bundleName": "LibSQLDriverPlugin", - "bundleId": "com.TablePro.LibSQLDriverPlugin", - "bundled": false, - "displayName": "libSQL / Turso Driver", - "summary": "libSQL and Turso database support via Hrana HTTP protocol", - "databaseTypeIds": [ - "libSQL", - "Turso" - ], - "icon": "libsql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/libsql" + "MSSQLDriver": { + "name": "MSSQL Driver", + "iconName": "mssql-icon" }, - "markdown": { - "target": "MarkdownExport", - "bundleName": "MarkdownExport", - "bundleId": "com.TablePro.MarkdownExportPlugin", - "bundled": true, - "displayName": "Markdown Export", - "summary": "Export data as Markdown tables", - "databaseTypeIds": null, - "icon": "text.alignleft", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "MongoDBDriver": { + "name": "MongoDB Driver", + "iconName": "mongodb-icon" }, - "mongodb": { - "target": "MongoDBDriver", - "bundleName": "MongoDBDriver", - "bundleId": "com.TablePro.MongoDBDriver", - "bundled": false, - "displayName": "MongoDB Driver", - "summary": "MongoDB document database driver via libmongoc", - "databaseTypeIds": [ - "MongoDB" - ], - "icon": "mongodb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/mongodb" + "OracleDriver": { + "name": "Oracle Driver", + "iconName": "oracle-icon" }, - "mql": { - "target": "MQLExport", - "bundleName": "MQLExport", - "bundleId": "com.TablePro.MQLExportPlugin", - "bundled": true, - "displayName": "MQL Export", - "summary": "Export MongoDB data as MQL statements", - "databaseTypeIds": null, - "icon": "leaf", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "ParquetExport": { + "name": "Parquet Export", + "iconName": "square.grid.3x3" }, - "mssql": { - "target": "MSSQLDriver", - "bundleName": "MSSQLDriver", - "bundleId": "com.TablePro.MSSQLDriver", - "bundled": false, - "displayName": "MSSQL Driver", - "summary": "Microsoft SQL Server driver via FreeTDS", - "databaseTypeIds": [ - "SQL Server" - ], - "icon": "mssql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/mssql" + "SnowflakeDriverPlugin": { + "name": "Snowflake Driver", + "iconName": "snowflake-icon" }, - "oracle": { - "target": "OracleDriver", - "bundleName": "OracleDriver", - "bundleId": "com.TablePro.OracleDriver", - "bundled": false, - "displayName": "Oracle Driver", - "summary": "Oracle Database 12c+ driver via OracleNIO", - "databaseTypeIds": [ - "Oracle" - ], - "icon": "oracle-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/oracle" + "SpannerDriverPlugin": { + "name": "Spanner Driver", + "iconName": "spanner-icon" }, - "parquet": { - "target": "ParquetExport", - "bundleName": "ParquetExport", - "bundleId": "com.TablePro.ParquetExportPlugin", - "bundled": false, - "displayName": "Parquet Export", - "summary": "Export data to Apache Parquet", - "databaseTypeIds": null, - "icon": "square.grid.3x3", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "SurrealDBDriverPlugin": { + "name": "SurrealDB Driver", + "iconName": "surrealdb-icon" }, - "redis": { - "target": "RedisDriver", - "bundleName": "RedisDriver", - "bundleId": "com.TablePro.RedisDriver", - "bundled": true, - "displayName": "Redis Driver", - "summary": "Redis in-memory data store driver via hiredis", - "databaseTypeIds": [ - "Redis" - ], - "icon": "redis-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/redis" + "TeradataDriver": { + "name": "Teradata Driver", + "iconName": "teradata-icon" }, - "snowflake": { - "target": "SnowflakeDriverPlugin", - "bundleName": "SnowflakeDriverPlugin", - "bundleId": "com.TablePro.SnowflakeDriverPlugin", - "bundled": false, - "displayName": "Snowflake Driver", - "summary": "Snowflake cloud data warehouse driver via the connector REST protocol", - "databaseTypeIds": [ - "Snowflake" - ], - "icon": "snowflake-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/snowflake" + "TrinoDriverPlugin": { + "name": "Trino Driver", + "iconName": "trino-icon" }, - "spanner": { - "target": "SpannerDriverPlugin", - "bundleName": "SpannerDriverPlugin", - "bundleId": "com.TablePro.SpannerDriverPlugin", - "bundled": false, - "displayName": "Spanner Driver", - "summary": "Google Cloud Spanner driver via REST API", - "databaseTypeIds": [ - "Spanner" - ], - "icon": "spanner-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/spanner" + "TypesenseDriverPlugin": { + "name": "Typesense Driver", + "iconName": "typesense-icon" }, - "sqlimport": { - "target": "SQLImport", - "bundleName": "SQLImport", - "bundleId": "com.TablePro.SQLImportPlugin", - "bundled": true, - "displayName": "SQL Import", - "summary": "Import data from SQL dump files", - "databaseTypeIds": null, - "icon": "doc.text", - "category": "import-format", - "homepage": "https://docs.tablepro.app/features/import-export" - }, - "sqlite": { - "target": "SQLiteDriver", - "bundleName": "SQLiteDriver", - "bundleId": "com.TablePro.SQLiteDriver", - "bundled": true, - "displayName": "SQLite Driver", - "summary": "SQLite embedded database driver", - "databaseTypeIds": [ - "SQLite" - ], - "icon": "sqlite-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/sqlite" - }, - "surrealdb": { - "target": "SurrealDBDriverPlugin", - "bundleName": "SurrealDBDriverPlugin", - "bundleId": "com.TablePro.SurrealDBDriverPlugin", - "bundled": false, - "displayName": "SurrealDB Driver", - "summary": "SurrealDB driver over the HTTP RPC protocol with SurrealQL", - "databaseTypeIds": [ - "SurrealDB" - ], - "icon": "surrealdb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/surrealdb" - }, - "teradata": { - "target": "TeradataDriver", - "bundleName": "TeradataDriver", - "bundleId": "com.TablePro.TeradataDriver", - "bundled": false, - "displayName": "Teradata Driver", - "summary": "Teradata Vantage driver via a native Swift TD2 client", - "databaseTypeIds": [ - "Teradata" - ], - "icon": "teradata-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/teradata" - }, - "trino": { - "target": "TrinoDriverPlugin", - "bundleName": "TrinoDriverPlugin", - "bundleId": "com.TablePro.TrinoDriverPlugin", - "bundled": false, - "displayName": "Trino Driver", - "summary": "Trino distributed SQL engine driver via the REST client protocol", - "databaseTypeIds": [ - "Trino" - ], - "icon": "trino-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/trino" - }, - "typesense": { - "target": "TypesenseDriverPlugin", - "bundleName": "TypesenseDriverPlugin", - "bundleId": "com.TablePro.TypesenseDriverPlugin", - "bundled": false, - "displayName": "Typesense Driver", - "summary": "Typesense driver over the REST API with a request console", - "databaseTypeIds": [ - "Typesense" - ], - "icon": "typesense-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/typesense" - }, - "weaviate": { - "target": "WeaviateDriverPlugin", - "bundleName": "WeaviateDriverPlugin", - "bundleId": "com.TablePro.WeaviateDriverPlugin", - "bundled": false, - "displayName": "Weaviate Driver", - "summary": "Weaviate driver over the REST API with a GraphQL console", - "databaseTypeIds": [ - "Weaviate" - ], - "icon": "weaviate-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/weaviate" - }, - "xlsx": { - "target": "XLSXExport", - "bundleName": "XLSXExport", - "bundleId": "com.TablePro.XLSXExportPlugin", - "bundled": true, - "displayName": "XLSX Export", - "summary": "Export data to Microsoft Excel XLSX format", - "databaseTypeIds": null, - "icon": "tablecells", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" - }, - "xlsximport": { - "target": "XLSXImport", - "bundleName": "XLSXImport", - "bundleId": "com.TablePro.XLSXImportPlugin", - "bundled": true, - "displayName": "XLSX Import", - "summary": "Import data from Excel workbooks", - "databaseTypeIds": null, - "icon": "tablecells", - "category": "import-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "WeaviateDriverPlugin": { + "name": "Weaviate Driver", + "iconName": "weaviate-icon" } } } diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 7e1a85a819..1bdc68f1bf 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,6 +1,6 @@ + Fixes # diff --git a/.github/release-flags.json b/.github/release-flags.json deleted file mode 100644 index cbe1b4e077..0000000000 --- a/.github/release-flags.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "$comment": "Per-release flags read by .github/workflows/build.yml. criticalUpdate: when to set it is the Critical update flag section of .claude/skills/release/SKILL.md; what it does is the Staged rollout and critical updates section of RELEASING.md.", - "criticalUpdate": true -} diff --git a/.github/scripts/test_update_registry.py b/.github/scripts/test_update_registry.py deleted file mode 100644 index 4d90015a4d..0000000000 --- a/.github/scripts/test_update_registry.py +++ /dev/null @@ -1,320 +0,0 @@ -#!/usr/bin/env python3 -"""Tests for update-registry.py PluginKit version pruning (#1322). - -Run: python3 .github/scripts/test_update_registry.py -""" -import contextlib -import importlib.util -import io -import os -import sys -from unittest import mock - -_spec = importlib.util.spec_from_file_location( - "update_registry", - os.path.join(os.path.dirname(__file__), "update-registry.py"), -) -update_registry = importlib.util.module_from_spec(_spec) -_spec.loader.exec_module(update_registry) - - -def test_kit_version_rejects_non_int(): - assert update_registry.kit_version({"pluginKitVersion": 14}) == 14 - assert update_registry.kit_version({"pluginKitVersion": None}) is None - assert update_registry.kit_version({}) is None - - -def test_prune_drops_null_kit_binary(): - binaries = [ - {"architecture": "arm64", "pluginKitVersion": None, "downloadURL": "legacy"}, - {"architecture": "arm64", "pluginKitVersion": 14, "downloadURL": "v14"}, - {"architecture": "arm64", "pluginKitVersion": 13, "downloadURL": "v13"}, - ] - kept = update_registry.prune_old_kit_versions(binaries, keep_count=2) - versions = sorted(update_registry.kit_version(b) for b in kept) - assert versions == [13, 14], versions - assert all(update_registry.kit_version(b) is not None for b in kept) - - -def test_prune_keeps_only_two_newest(): - binaries = [ - {"architecture": "arm64", "pluginKitVersion": v, "downloadURL": str(v)} - for v in (12, 13, 14) - ] - kept = update_registry.prune_old_kit_versions(binaries, keep_count=2) - versions = sorted(update_registry.kit_version(b) for b in kept) - assert versions == [13, 14], versions - - -def test_update_entry_drops_legacy_null_binary(): - manifest = { - "schemaVersion": 2, - "plugins": [ - { - "id": "com.TablePro.DynamoDBDriverPlugin", - "name": "DynamoDB", - "version": "1.0.15", - "summary": "old", - "category": "database-driver", - "binaries": [ - {"architecture": "arm64", "pluginKitVersion": None, "downloadURL": "legacy"}, - ], - } - ], - } - - class Args: - id = "com.TablePro.DynamoDBDriverPlugin" - name = "DynamoDB" - version = "1.0.16" - summary = "new" - db_type_ids = '["dynamodb"]' - arm64_url = "https://x/arm64" - arm64_sha = "a" - x86_64_url = "https://x/x86_64" - x86_64_sha = "b" - min_app_version = "0.43.0" - icon = "icon" - homepage = "https://tablepro.app" - category = "database-driver" - plugin_kit_version = 14 - keep_kit_versions = 2 - - result = update_registry.update_plugin_entry(manifest, Args()) - entry = next(p for p in result["plugins"] if p["id"] == Args.id) - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14], kits - assert all(update_registry.kit_version(b) is not None for b in entry["binaries"]) - - -def _args(version, pkv, keep=2, min_app_version="0.43.0"): - class Args: - id = "com.TablePro.DynamoDBDriverPlugin" - name = "DynamoDB" - summary = "new" - db_type_ids = '["dynamodb"]' - arm64_url = "https://x/arm64" - arm64_sha = "a" - x86_64_url = "https://x/x86_64" - x86_64_sha = "b" - icon = "icon" - homepage = "https://tablepro.app" - category = "database-driver" - - Args.version = version - Args.plugin_kit_version = pkv - Args.keep_kit_versions = keep - Args.min_app_version = min_app_version - return Args - - -def _manifest(*kit_versions): - return { - "schemaVersion": 2, - "plugins": [ - { - "id": "com.TablePro.DynamoDBDriverPlugin", - "name": "DynamoDB", - "version": "1.0.15", - "summary": "old", - "category": "database-driver", - "binaries": [ - { - "architecture": arch, - "pluginKitVersion": pkv, - "downloadURL": f"old-{pkv}-{arch}", - "sha256": "x", - } - for pkv in kit_versions - for arch in ("arm64", "x86_64") - ], - } - ], - } - - -def test_publishing_a_new_kit_version_keeps_the_previous_one(): - """The retention policy's whole point: a user on the previous app can still install. - - Nothing covered this. The one existing merge test starts from a null-version binary, which - is dropped rather than retained, so the surviving-binary path was never exercised. - """ - result = update_registry.update_plugin_entry(_manifest(14), _args("1.0.16", 15)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14, 15, 15], kits - urls = {b["downloadURL"] for b in entry["binaries"]} - assert "old-14-arm64" in urls, urls - - -def test_a_third_kit_version_evicts_the_oldest(): - result = update_registry.update_plugin_entry(_manifest(13, 14), _args("1.0.17", 15)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14, 15, 15], kits - - -def test_republishing_the_same_kit_version_replaces_its_binaries(): - result = update_registry.update_plugin_entry(_manifest(14), _args("1.0.16", 14)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14], kits - urls = {b["downloadURL"] for b in entry["binaries"]} - assert urls == {"https://x/arm64", "https://x/x86_64"}, urls - - -def test_publishing_below_the_retained_window_refuses(): - """Otherwise the entry advertises a version whose binary the prune just discarded.""" - try: - update_registry.update_plugin_entry(_manifest(15, 16), _args("1.0.18", 14)) - except SystemExit as error: - assert "older than the" in str(error), error - else: - raise AssertionError("expected SystemExit when the new binary would be pruned away") - - -def _entry(manifest): - return next(p for p in manifest["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - - -def test_min_app_version_admits_the_oldest_retained_binary(): - manifest = {"schemaVersion": 2, "plugins": []} - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 25, keep=3, min_app_version="0.73.0") - ) - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.17", 33, keep=3, min_app_version="0.76.1") - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.73.0", entry["minAppVersion"] - assert { - (update_registry.kit_version(b), b["minAppVersion"]) for b in entry["binaries"] - } == {(25, "0.73.0"), (33, "0.76.1")}, entry["binaries"] - - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.18", 33, keep=3, min_app_version="0.76.1") - ) - assert _entry(manifest)["minAppVersion"] == "0.73.0", _entry(manifest)["minAppVersion"] - - -def test_min_app_version_compares_numerically(): - manifest = {"schemaVersion": 2, "plugins": []} - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 25, keep=3, min_app_version="0.9.0") - ) - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.17", 33, keep=3, min_app_version="0.10.0") - ) - assert _entry(manifest)["minAppVersion"] == "0.9.0", _entry(manifest)["minAppVersion"] - - -def test_min_app_version_rises_once_the_oldest_binary_is_evicted(): - manifest = {"schemaVersion": 2, "plugins": []} - for pkv, app in ((25, "0.73.0"), (30, "0.74.0"), (32, "0.75.0"), (33, "0.76.1")): - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", pkv, keep=3, min_app_version=app) - ) - entry = _entry(manifest) - assert sorted({update_registry.kit_version(b) for b in entry["binaries"]}) == [30, 32, 33] - assert entry["minAppVersion"] == "0.74.0", entry["minAppVersion"] - - -def test_republishing_a_kit_never_raises_its_minimum_app_version(): - manifest = {"schemaVersion": 2, "plugins": []} - for pkv, app in ((33, "0.76.0"), (33, "0.76.1"), (34, "0.77.0"), (35, "0.78.0")): - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", pkv, keep=3, min_app_version=app) - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.76.0", entry["minAppVersion"] - assert sorted( - b["minAppVersion"] for b in entry["binaries"] if update_registry.kit_version(b) == 33 - ) == ["0.76.0", "0.76.0"], entry["binaries"] - - -def test_republishing_a_kit_never_raises_it_over_a_binary_without_its_own_value(): - manifest = _manifest(33) - _entry(manifest)["minAppVersion"] = "0.76.0" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.1") - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.76.0", entry["minAppVersion"] - assert {b["minAppVersion"] for b in entry["binaries"]} == {"0.76.0"}, entry["binaries"] - - -def test_binaries_without_their_own_min_app_version_keep_the_entry_value(): - manifest = _manifest(30) - _entry(manifest)["minAppVersion"] = "0.74.0" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.1") - ) - assert _entry(manifest)["minAppVersion"] == "0.74.0", _entry(manifest)["minAppVersion"] - - manifest = _manifest(30) - _entry(manifest)["minAppVersion"] = "0.76.1" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.0") - ) - assert _entry(manifest)["minAppVersion"] == "0.76.0", _entry(manifest)["minAppVersion"] - - -_REQUIRED_FLAGS = [ - "--manifest", "plugins.json", - "--id", "com.TablePro.DynamoDBDriverPlugin", - "--name", "DynamoDB", - "--version", "1.0.16", - "--summary", "new", - "--db-type-ids", '["dynamodb"]', - "--arm64-url", "https://x/arm64", - "--arm64-sha", "a", - "--x86_64-url", "https://x/x86_64", - "--x86_64-sha", "b", - "--min-app-version", "0.76.1", - "--icon", "icon", - "--homepage", "https://tablepro.app", - "--plugin-kit-version", "33", -] - - -def _parse(flags): - with mock.patch.object(sys, "argv", ["update-registry.py", *flags]): - return update_registry.parse_args() - - -def test_omitting_the_retention_count_refuses_to_run(): - stderr = io.StringIO() - try: - with contextlib.redirect_stderr(stderr): - args = _parse(_REQUIRED_FLAGS) - except SystemExit as error: - assert error.code != 0, error.code - assert "--keep-kit-versions" in stderr.getvalue(), stderr.getvalue() - else: - raise AssertionError( - f"expected SystemExit without --keep-kit-versions, got {args.keep_kit_versions}" - ) - - -def test_the_retention_count_is_the_one_passed(): - assert _parse([*_REQUIRED_FLAGS, "--keep-kit-versions", "3"]).keep_kit_versions == 3 - - -if __name__ == "__main__": - test_kit_version_rejects_non_int() - test_prune_drops_null_kit_binary() - test_prune_keeps_only_two_newest() - test_update_entry_drops_legacy_null_binary() - test_publishing_a_new_kit_version_keeps_the_previous_one() - test_a_third_kit_version_evicts_the_oldest() - test_republishing_the_same_kit_version_replaces_its_binaries() - test_publishing_below_the_retained_window_refuses() - test_min_app_version_admits_the_oldest_retained_binary() - test_min_app_version_compares_numerically() - test_min_app_version_rises_once_the_oldest_binary_is_evicted() - test_republishing_a_kit_never_raises_its_minimum_app_version() - test_republishing_a_kit_never_raises_it_over_a_binary_without_its_own_value() - test_binaries_without_their_own_min_app_version_keep_the_entry_value() - test_omitting_the_retention_count_refuses_to_run() - test_the_retention_count_is_the_one_passed() - print("All update-registry tests passed.") diff --git a/.github/scripts/update-registry.py b/.github/scripts/update-registry.py deleted file mode 100755 index 9cf3ab0d98..0000000000 --- a/.github/scripts/update-registry.py +++ /dev/null @@ -1,214 +0,0 @@ -#!/usr/bin/env python3 -""" -Update a TablePro plugin registry manifest (plugins.json) with a new binary set. - -For a given plugin ID and PluginKit version, this script: - - Adds or replaces the arm64 and x86_64 binaries for that PluginKit version - - Preserves binaries for other PluginKit versions (up to --keep-kit-versions) - - Drops binaries for PluginKit versions older than (newest - keep + 1) - - Updates plugin-level metadata (name, summary, etc.) from the newest binary set - - Records minAppVersion on each new binary and sets the entry's minAppVersion to the lowest - across the retained binaries. Binaries on one PluginKit version share one ABI, so republishing - a kit keeps the lowest minAppVersion that kit already carried rather than raising it - - Sets schemaVersion to 2 - - Writes atomically via a temp file + rename -""" - -import argparse -import json -import os -import re -import sys -import tempfile - - -def parse_args(): - parser = argparse.ArgumentParser(description="Update plugins.json registry entry") - parser.add_argument("--manifest", required=True) - parser.add_argument("--id", required=True) - parser.add_argument("--name", required=True) - parser.add_argument("--version", required=True) - parser.add_argument("--summary", required=True) - parser.add_argument("--db-type-ids", required=True) - parser.add_argument("--arm64-url", required=True) - parser.add_argument("--arm64-sha", required=True) - parser.add_argument("--x86_64-url", required=True) - parser.add_argument("--x86_64-sha", required=True) - parser.add_argument("--min-app-version", required=True) - parser.add_argument("--icon", required=True) - parser.add_argument("--homepage", required=True) - parser.add_argument("--category", default="database-driver") - parser.add_argument("--plugin-kit-version", required=True, type=int) - parser.add_argument( - "--keep-kit-versions", - required=True, - type=int, - help="Number of distinct PluginKit versions to retain per plugin. Oldest dropped first.", - ) - return parser.parse_args() - - -def load_manifest(path): - with open(path, "r", encoding="utf-8") as file: - return json.load(file) - - -def write_manifest_atomic(path, manifest): - dir_path = os.path.dirname(os.path.abspath(path)) - fd, tmp_path = tempfile.mkstemp(dir=dir_path, suffix=".tmp") - try: - with os.fdopen(fd, "w", encoding="utf-8") as file: - json.dump(manifest, file, indent=2) - file.write("\n") - os.replace(tmp_path, path) - except Exception: - try: - os.unlink(tmp_path) - except OSError: - pass - raise - - -def kit_version(binary): - value = binary.get("pluginKitVersion") - return value if isinstance(value, int) else None - - -def prune_old_kit_versions(binaries, keep_count): - # Drop binaries without a concrete integer PluginKit version. The app matches - # binaries on an exact integer, so a null/missing version can never resolve and - # only shadows valid binaries (the cause of the DynamoDB noCompatibleBinary in #1322). - typed = [b for b in binaries if kit_version(b) is not None] - - versions_seen = [] - for binary in typed: - pkv = kit_version(binary) - if pkv not in versions_seen: - versions_seen.append(pkv) - - versions_seen.sort(reverse=True) - versions_to_keep = set(versions_seen[:keep_count]) - - return [b for b in typed if kit_version(b) in versions_to_keep] - - -def app_version_key(version): - return tuple(int(part) for part in re.findall(r"\d+", version)) - - -def kit_min_app_version(existing_entry, pkv, requested): - if existing_entry is None: - return requested - previous_entry_value = existing_entry.get("minAppVersion") - carried = [ - b.get("minAppVersion") or previous_entry_value - for b in existing_entry.get("binaries", []) - if kit_version(b) == pkv - ] - return min([requested, *filter(None, carried)], key=app_version_key) - - -def entry_min_app_version(binaries, previous_entry_value): - candidates = [b["minAppVersion"] for b in binaries if b.get("minAppVersion")] - if previous_entry_value and any(not b.get("minAppVersion") for b in binaries): - candidates.append(previous_entry_value) - return min(candidates, key=app_version_key) - - -def update_plugin_entry(manifest, args): - bundle_id = args.id - db_type_ids = json.loads(args.db_type_ids) - pkv = args.plugin_kit_version - - existing_plugins = manifest.get("plugins", []) - existing_entry = next((p for p in existing_plugins if p["id"] == bundle_id), None) - min_app_version = kit_min_app_version(existing_entry, pkv, args.min_app_version) - - new_binaries = [ - { - "architecture": "arm64", - "pluginKitVersion": pkv, - "downloadURL": args.arm64_url, - "sha256": args.arm64_sha, - "minAppVersion": min_app_version, - }, - { - "architecture": "x86_64", - "pluginKitVersion": pkv, - "downloadURL": args.x86_64_url, - "sha256": args.x86_64_sha, - "minAppVersion": min_app_version, - }, - ] - - if existing_entry is not None: - surviving = [ - b for b in existing_entry.get("binaries", []) - if kit_version(b) is not None and kit_version(b) != pkv - ] - merged_binaries = surviving + new_binaries - else: - merged_binaries = new_binaries - - merged_binaries = prune_old_kit_versions(merged_binaries, args.keep_kit_versions) - - # The prune keeps the newest N PluginKit versions, so publishing for an older one drops the - # binary that was just built while still advertising its version number. The entry would then - # point every app at a download that does not exist for it. - if not any(kit_version(b) == pkv for b in merged_binaries): - raise SystemExit( - f"PluginKit {pkv} is older than the {args.keep_kit_versions} retained versions; " - f"publishing would advertise {args.version} with no binary for it" - ) - - previous_min_app_version = existing_entry.get("minAppVersion") if existing_entry else None - - updated_entry = { - "id": bundle_id, - "name": args.name, - "version": args.version, - "summary": args.summary, - "author": {"name": "TablePro", "url": "https://tablepro.app"}, - "homepage": args.homepage, - "category": args.category, - "databaseTypeIds": db_type_ids, - "iconName": args.icon, - "isVerified": True, - "minAppVersion": entry_min_app_version(merged_binaries, previous_min_app_version), - "binaries": merged_binaries, - } - - if existing_entry is not None and existing_entry.get("metadata"): - updated_entry["metadata"] = existing_entry["metadata"] - - manifest["plugins"] = [p for p in existing_plugins if p["id"] != bundle_id] - manifest["plugins"].append(updated_entry) - manifest["schemaVersion"] = 2 - - return manifest - - -def main(): - args = parse_args() - - if not os.path.exists(args.manifest): - print(f"ERROR: manifest not found: {args.manifest}", file=sys.stderr) - sys.exit(1) - - manifest = load_manifest(args.manifest) - manifest = update_plugin_entry(manifest, args) - write_manifest_atomic(args.manifest, manifest) - - entry = next(p for p in manifest["plugins"] if p["id"] == args.id) - kept_versions = sorted( - {b.get("pluginKitVersion", 0) for b in entry["binaries"]}, - reverse=True, - ) - print( - f"Updated {args.id} v{args.version} (PluginKit {args.plugin_kit_version}). " - f"Retained PluginKit versions: {kept_versions}" - ) - - -if __name__ == "__main__": - main() diff --git a/.github/workflows/build-plugin.yml b/.github/workflows/build-plugin.yml deleted file mode 100644 index e36897cfaa..0000000000 --- a/.github/workflows/build-plugin.yml +++ /dev/null @@ -1,490 +0,0 @@ -name: Build Plugin - -on: - push: - tags: ["plugin-*-v*"] - workflow_dispatch: - inputs: - tags: - description: > - Comma-separated plugin tag:pluginKitVersion pairs. - pluginKitVersion defaults to TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig. - Examples: plugin-mongodb-v1.0.25, plugin-mongodb-v1.0.25:13 - required: true - type: string - baseRef: - description: > - Build the plugin from this ref instead of the default branch, so its binary links against - that release's PluginKit and declares that release's TableProPluginKitVersion. Use a - shipped app tag (v0.73.0) to publish a driver fix users on that release can install. - Leave empty to build from the branch the workflow runs on. - required: false - type: string - -# Declared per job rather than here. resolve-tags reads an integer out of an xcconfig and needs -# nothing, and a workflow-level grant hands it a write token it has no use for. -permissions: - contents: read - -# Without this, two runs for the same tag race each other through a delete-and-recreate of the same -# GitHub Release. Never cancel one in flight: it holds a notarization submission open with Apple and -# publishes release assets. -concurrency: - group: build-plugin-${{ github.ref }} - cancel-in-progress: false - -jobs: - resolve-tags: - name: Resolve Plugin Tags - runs-on: ubuntu-latest - outputs: - matrix: ${{ steps.tags.outputs.matrix }} - currentPluginKitVersion: ${{ steps.pkv.outputs.version }} - runner: ${{ steps.runner.outputs.runner }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - # The build job runs the setup-xcode action of the ref it builds. A ref from before Xcode 27 - # pins 26.x there, and the xcode-27 image has no Xcode 26. - - name: Pick the runner for the ref being built - id: runner - env: - REF: ${{ inputs.baseRef || github.ref }} - run: | - set -euo pipefail - git fetch --no-tags --depth=1 origin "$REF" - ACTION="$(git show FETCH_HEAD:.github/actions/setup-xcode/action.yml)" - if grep -qE '(default: "|Xcode_)26\.' <<< "$ACTION"; then - echo "runner=macos-26" >> "$GITHUB_OUTPUT" - else - echo "runner=xcode-27" >> "$GITHUB_OUTPUT" - fi - - - name: Validate registry update script - run: python3 .github/scripts/test_update_registry.py - - - name: Check the plugin manifest against the plugin classes - run: python3 scripts/ci/check-plugin-manifest.py - - - name: Read the PluginKit version - id: pkv - run: | - VERSION=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' \ - Configs/PluginKit.xcconfig) - if [ -z "$VERSION" ]; then - echo "::error::Could not parse TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig" - exit 1 - fi - echo "version=$VERSION" >> "$GITHUB_OUTPUT" - echo "TABLEPRO_PLUGINKIT_VERSION=$VERSION" - - - name: Build matrix - id: tags - env: - DEFAULT_PKV: ${{ steps.pkv.outputs.version }} - INPUT_TAGS: ${{ inputs.tags }} - REF_NAME: ${{ github.ref_name }} - run: | - if [ -n "$INPUT_TAGS" ]; then - IFS=',' read -ra RAW_TAGS <<< "$INPUT_TAGS" - else - RAW_TAGS=("$REF_NAME") - fi - - JSON='{"include":[' - FIRST=true - for ITEM in "${RAW_TAGS[@]}"; do - ITEM="${ITEM#"${ITEM%%[![:space:]]*}"}" - ITEM="${ITEM%"${ITEM##*[![:space:]]}"}" - TAG=$(echo "$ITEM" | cut -d: -f1) - PKV=$(echo "$ITEM" | cut -d: -f2 -s) - if [ -z "$PKV" ]; then - PKV="$DEFAULT_PKV" - fi - - # Both values are checked here, on ubuntu, before any macOS job exists to import a - # signing certificate. TAG reaches a shell and PKV is spliced unquoted into the JSON - # below, and neither had ever been validated: nothing downstream checks the version - # format either, and update-registry.py takes --version as an arbitrary string, so a - # malformed tag publishes a garbage version into the plugins.json every app reads. - # Checked against all 366 existing plugin-* tags: every one matches. - if [[ ! "$TAG" =~ ^plugin-[a-z0-9-]+-v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then - echo "::error::Malformed plugin tag: $TAG" - exit 1 - fi - if [[ ! "$PKV" =~ ^[0-9]+$ ]]; then - echo "::error::Non-numeric pluginKitVersion for $TAG: $PKV" - exit 1 - fi - - if [ "$FIRST" = true ]; then FIRST=false; else JSON+=','; fi - JSON+="{\"tag\":\"$TAG\",\"pluginKitVersion\":$PKV}" - done - JSON+=']}' - echo "matrix=$JSON" >> "$GITHUB_OUTPUT" - echo "Matrix: $JSON" - - build-plugin: - name: "Build ${{ matrix.tag }} (PluginKit ${{ matrix.pluginKitVersion }})" - needs: resolve-tags - # A dispatch runs under main's workflow group, so a tag push for the same plugin tag would - # otherwise stage its release at the same time. - concurrency: - group: build-plugin-release-${{ matrix.tag }} - cancel-in-progress: false - permissions: - contents: write - runs-on: ${{ needs.resolve-tags.outputs.runner }} - timeout-minutes: 30 - strategy: - matrix: ${{ fromJson(needs.resolve-tags.outputs.matrix) }} - fail-fast: false - - steps: - # A plugin's declared TableProPluginKitVersion and the symbols it links against both come - # from the tree it is built in, so they can only agree by building at the ref whose kit the - # binary should carry. Relabelling a binary built from main to an older kit would make an - # older app accept it and then fail Bundle.loadAndReturnError, which is #1917 and 0.49.0. - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - lfs: true - ref: ${{ inputs.baseRef || github.ref }} - - - name: Pull LFS files - run: git lfs pull - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - - name: Setup XcodeGen - uses: ./.github/actions/setup-xcodegen - - - name: Download static libraries - env: - GH_TOKEN: ${{ github.token }} - run: scripts/download-libs.sh - - # No provisioning profile: a .tableplugin is signed with Developer ID and notarized, and - # never carries an embedded profile the way the app does. - - name: Set up Apple signing - uses: ./.github/actions/apple-signing - with: - certificate-p12: ${{ secrets.CERTIFICATES_P12 }} - certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} - apple-id: ${{ secrets.APPLE_ID }} - team-id: ${{ secrets.APPLE_TEAM_ID }} - notary-password: ${{ secrets.NOTARY_PASSWORD }} - - - name: Resolve plugin info - id: plugin - env: - MATRIX_TAG: ${{ matrix.tag }} - # The metadata lives in .github/plugin-registry.json. It used to be a 120-line `case` block - # here, which is shell holding data, and seven of its icon values had drifted away from the - # plugin classes they describe with nothing to notice. - run: python3 scripts/ci/plugin-info.py "$MATRIX_TAG" >> "$GITHUB_OUTPUT" - - - name: Build Dameng native bridge - if: ${{ contains(matrix.tag, 'plugin-dameng-') }} - run: ./scripts/build-dameng.sh both - - - name: Set up Go for the SAP HANA helper - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0 - with: - go-version-file: Native/HanaBridge/go.mod - cache-dependency-path: Native/HanaBridge/go.sum - - # Universal, because both architecture builds below embed the same helper. build-plugin.sh - # signs it with Developer ID, the hardened runtime and a timestamp, and checks all three. - - name: Build the SAP HANA helper - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - run: ./scripts/build-hana.sh both - - # The helper ships inside the plugin, and only a release changes what users run. - - name: Check the SAP HANA helper for known vulnerabilities - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - working-directory: Native/HanaBridge - run: go run golang.org/x/vuln/cmd/govulncheck@v1.8.0 ./... - - # NOTARIZE is not optional. Gatekeeper refuses to load an unnotarized plugin bundle - # into TablePro, so a build that skips it publishes a driver no user can open. - - name: Build plugin binaries - env: - TEAM_ID: ${{ secrets.APPLE_TEAM_ID }} - NOTARIZE: "true" - TARGET: ${{ steps.plugin.outputs.target }} - VERSION: ${{ steps.plugin.outputs.version }} - # One invocation, not one per architecture. The script has always taken "both", and - # each invocation regenerates both Xcode projects and wipes build/DerivedData, so calling - # it twice threw away everything the first architecture had compiled. - run: ./scripts/build-plugin.sh "$TARGET" both "$VERSION" - - # Assert the result rather than trusting the flag. The previous notarization step was - # gated on an environment variable nothing defined, so it never ran and nothing noticed - # for every plugin release the registry has ever published. - # Before anything is published. The registry serves these to users directly, so a bundle - # that will not load is a plugin nobody can install. - - name: Verify the built bundles load - env: - BUNDLE_NAME: ${{ steps.plugin.outputs.bundleName }} - run: | - set -euo pipefail - for ARCH in arm64 x86_64; do - WORK="$(mktemp -d)" - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - # The distribution zip carries the plugin alone, so dyld has nothing to resolve - # @rpath/TableProPluginKit.framework against and every plugin fails for the wrong - # reason. build-plugin.sh builds both into one flat CONFIGURATION_BUILD_DIR, so the - # framework is put beside the unzipped bundle rather than added as a second search - # directory, which would make the script scan the build dir as another bundle. - cp -R "build/Plugins/${ARCH}/TableProPluginKit.framework" "$WORK/" - # Only the native slice can be dlopened on this runner; the other is checked by - # `lipo -info` having produced it and by Gatekeeper below. - if [ "$ARCH" = "arm64" ]; then - scripts/ci/verify-plugin-loads.sh "$WORK" - fi - rm -rf "$WORK" - done - - - name: Verify the built bundles are notarized - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - for ARCH in arm64 x86_64; do - WORK=$(mktemp -d) - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - PLUGIN="$WORK/${BUNDLE_NAME}.tableplugin" - if ! xcrun stapler validate "$PLUGIN"; then - echo "::error::${BUNDLE_NAME}-${ARCH} carries no stapled notarization ticket. Gatekeeper refuses to load such a bundle into TablePro." - exit 1 - fi - if ! spctl -a -vvv -t open --context context:primary-signature "$PLUGIN" 2>&1 | grep -q "source=Notarized Developer ID"; then - echo "::error::Gatekeeper rejects ${BUNDLE_NAME}-${ARCH}:" - spctl -a -vvv -t open --context context:primary-signature "$PLUGIN" || true - exit 1 - fi - echo "Verified ${BUNDLE_NAME}-${ARCH}: notarized, stapled and accepted by Gatekeeper." - done - - - name: Verify built PluginKit version matches the release label - env: - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - EXPECTED="$MATRIX_PKV" - for ARCH in arm64 x86_64; do - WORK=$(mktemp -d) - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - PLIST=$(find "$WORK" -path '*.tableplugin/Contents/Info.plist' | head -1) - if [ -z "$PLIST" ]; then - echo "::error::Could not find Info.plist in the built ${BUNDLE_NAME}-${ARCH} bundle." - exit 1 - fi - ACTUAL=$(plutil -extract TableProPluginKitVersion raw "$PLIST") - if [ "$ACTUAL" != "$EXPECTED" ]; then - echo "::error::${BUNDLE_NAME}-${ARCH} was built for PluginKit $ACTUAL but this release is labeled PluginKit $EXPECTED. Refusing to publish a mislabeled binary. Re-release from a commit whose plugin Info.plist matches the target PluginKit version." - exit 1 - fi - echo "Verified ${BUNDLE_NAME}-${ARCH}: built PluginKit $ACTUAL matches the release label." - done - - - name: Read checksums - id: sha - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - ARM64_SHA=$(cat "build/Plugins/${BUNDLE_NAME}-arm64.zip.sha256") - X86_SHA=$(cat "build/Plugins/${BUNDLE_NAME}-x86_64.zip.sha256") - { - echo "arm64=$ARM64_SHA" - echo "x86_64=$X86_SHA" - } >> "$GITHUB_OUTPUT" - - # The checkout above is `baseRef` when one is given, and a shipped app tag can predate - # draft-release.sh. - - name: Check out the release script - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.sha }} - path: .release-tooling - sparse-checkout: scripts/ci - persist-credentials: false - - # A published release is immutable, so both zips go on the draft first. - - name: Create the draft release - id: draft - env: - GH_TOKEN: ${{ github.token }} - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - PLUGIN_VERSION: ${{ steps.plugin.outputs.version }} - run: | - TAG="$MATRIX_TAG" - DISPLAY_NAME="${{ steps.plugin.outputs.displayName }}" - VERSION="$PLUGIN_VERSION" - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - ARM64_SHA="${{ steps.sha.outputs.arm64 }}" - X86_SHA="${{ steps.sha.outputs.x86_64 }}" - PKV="$MATRIX_PKV" - - RELEASE_BODY="## $DISPLAY_NAME v$VERSION - - Plugin release for TablePro (PluginKit $PKV). - - ### Installation - TablePro offers to install this plugin where it is needed. To install it by hand, add the zip below in Settings > Plugins. - - ### SHA-256 - - ARM64: \`$ARM64_SHA\` - - x86_64: \`$X86_SHA\`" - - EXISTING_PKV=$(gh api "repos/${GITHUB_REPOSITORY}/releases/tags/${TAG}" --jq .body 2>/dev/null \ - | grep -oiE 'PluginKit [0-9]+' | grep -oE '[0-9]+' | head -1 || true) - if [ -n "$EXISTING_PKV" ] && [ "$EXISTING_PKV" != "$PKV" ]; then - echo "::error::Release $TAG already exists for PluginKit $EXISTING_PKV. Refusing to overwrite it with PluginKit $PKV; publish the new ABI under a new plugin version." - exit 1 - fi - - # Tagged at the commit this job built, which is `baseRef` when one was given. Without - # --target GitHub tags the default branch's head instead, so a plugin built against a - # shipped release recorded whatever main had moved on to: the v0.75.0 plugins point at a - # commit carrying PluginKit 33, while their binaries are 32. - NOTES="$(mktemp)" - printf '%s' "$RELEASE_BODY" > "$NOTES" - .release-tooling/scripts/ci/draft-release.sh --target "$(git rev-parse HEAD)" \ - "$TAG" "$DISPLAY_NAME v$VERSION" "$NOTES" \ - "build/Plugins/${BUNDLE_NAME}-arm64.zip" "build/Plugins/${BUNDLE_NAME}-x86_64.zip" - - # Without make_latest=false, publishing marks the plugin release Latest instead of the app's. - - name: Publish the release - if: steps.draft.outputs.id != '' - env: - GH_TOKEN: ${{ github.token }} - RELEASE_ID: ${{ steps.draft.outputs.id }} - run: | - gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \ - -F draft=false -f make_latest=false --jq '"Published \(.tag_name): \(.html_url)"' - - - name: Verify published assets match the PluginKit label - env: - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - TAG="$MATRIX_TAG" - PKV="$MATRIX_PKV" - REPO="${{ github.repository }}" - for ARCH in arm64 x86_64; do - URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-${ARCH}.zip" - WORK=$(mktemp -d) - DOWNLOADED="" - for attempt in 1 2 3 4 5; do - if curl -fsSL "$URL" -o "$WORK/asset.zip"; then - DOWNLOADED="yes" - break - fi - echo "Published asset not downloadable yet (attempt $attempt/5), retrying in 5s..." - sleep 5 - done - if [ -z "$DOWNLOADED" ]; then - echo "::error::Could not download the published asset at $URL to verify its PluginKit version." - exit 1 - fi - unzip -oq "$WORK/asset.zip" -d "$WORK" - PLIST=$(find "$WORK" -path '*.tableplugin/Contents/Info.plist' | head -1) - if [ -z "$PLIST" ]; then - echo "::error::No .tableplugin Info.plist in the published ${BUNDLE_NAME}-${ARCH} asset." - exit 1 - fi - ACTUAL=$(plutil -extract TableProPluginKitVersion raw "$PLIST") - if [ "$ACTUAL" != "$PKV" ]; then - echo "::error::Published ${BUNDLE_NAME}-${ARCH} at $URL is PluginKit $ACTUAL but the registry will record $PKV. Refusing to record a mismatched binary." - exit 1 - fi - echo "Verified published ${BUNDLE_NAME}-${ARCH}: served PluginKit $ACTUAL matches the registry label." - done - - - name: Update plugin registry - env: - REGISTRY_DEPLOY_KEY: ${{ secrets.REGISTRY_DEPLOY_KEY }} - GH_TOKEN: ${{ github.token }} - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - PLUGIN_VERSION: ${{ steps.plugin.outputs.version }} - run: | - TAG="$MATRIX_TAG" - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - BUNDLE_ID="${{ steps.plugin.outputs.bundleId }}" - DISPLAY_NAME="${{ steps.plugin.outputs.displayName }}" - VERSION="$PLUGIN_VERSION" - SUMMARY="${{ steps.plugin.outputs.summary }}" - DB_TYPE_IDS='${{ steps.plugin.outputs.dbTypeIds }}' - MIN_APP_VERSION="${{ steps.plugin.outputs.minAppVersion }}" - ICON="${{ steps.plugin.outputs.icon }}" - HOMEPAGE="${{ steps.plugin.outputs.homepage }}" - CATEGORY="${{ steps.plugin.outputs.category }}" - ARM64_SHA="${{ steps.sha.outputs.arm64 }}" - X86_SHA="${{ steps.sha.outputs.x86_64 }}" - PKV="$MATRIX_PKV" - REPO="${{ github.repository }}" - ARM64_URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-arm64.zip" - X86_64_URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-x86_64.zip" - - SCRIPT_PATH="$(pwd)/.github/scripts/update-registry.py" - - WORK=$(mktemp -d) - eval "$(ssh-agent -s)" - trap 'ssh-add -D 2>/dev/null || true; eval "$(ssh-agent -k)" 2>/dev/null || true' EXIT - echo "$REGISTRY_DEPLOY_KEY" | ssh-add - - - git clone git@github.com:TableProApp/plugins.git "$WORK/registry" - cd "$WORK/registry" - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - - MAX_RETRIES=10 - for attempt in $(seq 1 $MAX_RETRIES); do - echo "Registry update attempt $attempt/$MAX_RETRIES" - git reset --hard origin/main - git pull --rebase origin main - - python3 "$SCRIPT_PATH" \ - --manifest plugins.json \ - --id "$BUNDLE_ID" \ - --name "$DISPLAY_NAME" \ - --version "$VERSION" \ - --summary "$SUMMARY" \ - --db-type-ids "$DB_TYPE_IDS" \ - --arm64-url "$ARM64_URL" \ - --arm64-sha "$ARM64_SHA" \ - --x86_64-url "$X86_64_URL" \ - --x86_64-sha "$X86_SHA" \ - --min-app-version "$MIN_APP_VERSION" \ - --icon "$ICON" \ - --homepage "$HOMEPAGE" \ - --category "$CATEGORY" \ - --plugin-kit-version "$PKV" \ - --keep-kit-versions 3 - - git add plugins.json - git commit -m "Update $DISPLAY_NAME to v$VERSION (PluginKit $PKV)" - - if git push; then - echo "Registry updated on attempt $attempt" - curl -sf "https://purge.jsdelivr.net/gh/TableProApp/plugins@main/plugins.json" >/dev/null \ - && echo "Purged jsDelivr cache for plugins.json" \ - || echo "::warning::jsDelivr purge request failed; the cache will refresh on its own shortly" - break - fi - - if [ "$attempt" -eq "$MAX_RETRIES" ]; then - echo "::error::Failed to push registry update after $MAX_RETRIES attempts" - exit 1 - fi - - DELAY=$((2 + RANDOM % 4)) - echo "Push rejected (concurrent update), retrying in ${DELAY}s..." - sleep "$DELAY" - done - - echo "$DISPLAY_NAME v$VERSION released (PluginKit $PKV)" diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml deleted file mode 100644 index 14e249bfa2..0000000000 --- a/.github/workflows/build.yml +++ /dev/null @@ -1,512 +0,0 @@ -name: Build TablePro - -on: - workflow_dispatch: - # No paths filter here. GitHub does not evaluate one for a tag push, and a tag push is the - # only thing that triggers this workflow, so the paths-ignore list that used to sit here - # never excluded anything. - push: - tags: ["v*"] - -# A release must never be cancelled part-way through: the build jobs hold notarization -# submissions open with Apple, and the release job pushes a commit and publishes artifacts. -# Re-running the same tag queues behind the run already in flight rather than racing it. -concurrency: - group: build-tablepro-${{ github.ref }} - cancel-in-progress: false - -# The release job raises this to write for itself. Everything else here builds and notarizes and -# needs nothing beyond the checkout. -permissions: - contents: read - -jobs: - # Runs no suite. A release commit changes only release files, so its own push skips both suites. - # For each suite the verdict is the newest first-parent commit on main where its job ran, waited - # for when still running, and it holds only if nothing the suite reads changed since. - verdict: - name: Tested on main - if: startsWith(github.ref, 'refs/tags/v') - runs-on: ubuntu-latest - timeout-minutes: 240 - permissions: - contents: read - actions: read - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Find the passing runs that cover this commit - env: - GH_TOKEN: ${{ github.token }} - GH_REPO: ${{ github.repository }} - run: | - set -euo pipefail - # The walk below follows the tag's own parents, so a tag off main would borrow main's verdict. - if ! git merge-base --is-ancestor HEAD origin/main; then - echo "::error::$GITHUB_SHA is not on main" - exit 1 - fi - covered() { - local workflow="$1" job="$2" suite="$3" sha run conclusion changed - for sha in $(git rev-list --first-parent --max-count=50 HEAD); do - run=$(gh run list --workflow "$workflow" --commit "$sha" --event push --limit 1 \ - --json databaseId --jq '.[0].databaseId // empty') || return 1 - [ -n "$run" ] || continue - gh run watch "$run" --interval 60 > /dev/null || return 1 - conclusion=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$run/jobs" \ - --jq ".jobs[] | select(.name == \"$job\") | .conclusion") || return 1 - case "$conclusion" in - skipped) continue ;; - success) ;; - *) echo "::error::$job ended '$conclusion' on $sha (run $run)"; return 1 ;; - esac - changed=$(scripts/ci/detect-changed-paths.sh --base "$sha" "$suite") || return 1 - if [ "$changed" != false ]; then - echo "::error::$job passed on $sha, but files it reads changed before $GITHUB_SHA" - return 1 - fi - echo "$job: run $run on $sha" >> "$GITHUB_STEP_SUMMARY" - return 0 - done - echo "::error::no $job run among the last 50 commits on main" - return 1 - } - status=0 - covered macos-tests.yml "macOS Tests" macos || status=1 - covered ios-tests.yml "iOS and Package Tests" ios || status=1 - exit "$status" - - build: - name: Build ${{ matrix.arch }} - runs-on: xcode-27 - timeout-minutes: ${{ matrix.timeout }} - strategy: - # No fail-fast. A failing arch would otherwise cancel its sibling mid-notarization, which is - # the same thing the concurrency block above exists to prevent. - fail-fast: false - matrix: - include: - # One budget for both arms. The arms run on separate runners at the same time, so a - # tighter budget on one buys nothing and only decides which arm dies first. v0.73.0 is - # what that cost: ARM64 reached "Build complete" at 30:01 and was cancelled 13 seconds - # into notarization, while x86_64 finished the whole job in 31:15. The two arms are not - # reliably ordered (over v0.62 to v0.67 x86_64 was the slower one, once by 22 minutes), - # and the notary queue is not something a build budget can predict, so the budget has to - # clear the slowest arm plus a notary wait rather than the arm's own past average. - - arch: arm64 - timeout: 45 - - arch: x86_64 - timeout: 45 - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - - name: Download static libraries - env: - GH_TOKEN: ${{ github.token }} - run: scripts/download-libs.sh --force - - # build-release.sh clones packages into ~/.spm-cache via -clonedSourcePackagesDirPath, - # so this restores the checkouts instead of re-cloning 30 repositories on every release. - # Package.resolved pins every revision and is tracked, which is why it can key the cache. - # macos-tests.yml saves it from main; a save here would land in this tag's scope, which no - # other tag can read. - - name: Restore Swift package checkouts - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - with: - path: ~/.spm-cache - key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} - restore-keys: spm-${{ runner.os }}- - - # create-dmg is the only Homebrew formula either arm needs. The MySQL plugin links - # Libs/libmariadb.a, which download-libs.sh vendors and prepare-libs.sh selects per - # architecture, and its headers live in Plugins/MySQLDriverPlugin/CMariaDB/include. - # LIBRARY_SEARCH_PATHS names $(SRCROOT)/Libs and no Homebrew prefix, so nothing in the - # build ever read mariadb-connector-c. macos-tests.yml proves it: its build job links all - # 31 plugin bundles with no Homebrew mariadb installed at all. - # - # The x86_64 arm is a cross-compile on an arm64 runner: every tool that runs during the - # build is native and the x86_64 slices of the static libraries are vendored in Libs. - # Bootstrapping a second Homebrew prefix cost five minutes a release to install a - # mariadb-connector-c the build never opened. create-dmg is a shell script, so the native - # prefix serves both arms. - - name: Install create-dmg - run: brew list create-dmg &>/dev/null || brew install create-dmg - - - name: Prepare libraries - run: scripts/ci/prepare-libs.sh ${{ matrix.arch }} - - - name: Create Secrets.xcconfig - env: - ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} - run: echo "ANALYTICS_HMAC_SECRET = ${ANALYTICS_HMAC_SECRET}" > Configs/Secrets.xcconfig - - - name: Setup XcodeGen - uses: ./.github/actions/setup-xcodegen - - - name: Set up Apple signing - uses: ./.github/actions/apple-signing - with: - certificate-p12: ${{ secrets.CERTIFICATES_P12 }} - certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} - apple-id: ${{ secrets.APPLE_ID }} - team-id: ${{ secrets.APPLE_TEAM_ID }} - notary-password: ${{ secrets.NOTARY_PASSWORD }} - provisioning-profile: ${{ secrets.PROVISIONING_PROFILE }} - - - name: Build ${{ matrix.arch }} - env: - ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} - NOTARIZE: "true" - run: scripts/build-release.sh ${{ matrix.arch }} - - - name: Verify build - run: scripts/ci/verify-build.sh ${{ matrix.arch }} - - - name: Package artifacts - env: - NOTARIZE: "true" - run: scripts/ci/package-artifacts.sh ${{ matrix.arch }} - - - name: Verify the update archive - run: scripts/ci/verify-update-archive.sh "build/Release/TablePro-${{ matrix.arch }}.tar.xz" - - - name: Upload artifacts - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 - with: - name: artifacts-${{ matrix.arch }} - path: | - build/Release/TablePro-*.dmg - build/Release/TablePro-*.tar.xz - - # Reads two integers out of an xcconfig and fetches a JSON manifest, so it wants nothing - # from macOS and does not need to spend a macOS runner minute (billed at ten times Linux). - registry-readiness: - name: Registry Readiness - runs-on: ubuntu-latest - if: startsWith(github.ref, 'refs/tags/v') - timeout-minutes: 5 - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Verify registry has compatible plugin binaries - run: | - CONFIG="Configs/PluginKit.xcconfig" - CURRENT=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") - FLOOR=$(sed -nE 's/^TABLEPRO_PLUGINKIT_MINIMUM_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") - echo "PluginKit floor=$FLOOR current=$CURRENT" - python3 scripts/check-registry-readiness.py --floor "$FLOOR" --current "$CURRENT" - - release: - name: Create GitHub Release - runs-on: xcode-27 - needs: [verdict, build, registry-readiness] - if: startsWith(github.ref, 'refs/tags/v') - # Holds APPCAST_DEPLOY_KEY, released only to v* tags. - environment: release - timeout-minutes: 10 - permissions: - contents: write - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - # Only the two build artifacts. Unfiltered, this also pulled the four test-result bundles and - # test-products, which is 874MB fetched to use 106MB of it, and every megabyte is another - # chance for the blob store to drop the transfer: v0.69.0 failed here twice in a row with - # "Artifact download failed after 5 retries", both times inside two minutes. Flatten takes any - # *.dmg or *.tar.xz it finds, so one inside a test bundle could also reach the release. - - name: Download build artifacts - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 - with: - pattern: artifacts-* - path: artifacts-raw/ - merge-multiple: true - - - name: Flatten artifacts - run: | - mkdir -p artifacts/ - find artifacts-raw/ -type f \( -name "*.dmg" -o -name "*.tar.xz" \) -exec mv {} artifacts/ \; - rm -rf artifacts-raw/ - echo "Artifacts:" - ls -lh artifacts/ - - - name: Verify the tag matches the declared version - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - TAG_VERSION="${GITHUB_REF#refs/tags/v}" - XCCONFIG_VERSION=$(sed -n 's/^MARKETING_VERSION[[:space:]]*=[[:space:]]*//p' Configs/Version.xcconfig | tr -d ' ') - if [ "$TAG_VERSION" != "$XCCONFIG_VERSION" ]; then - echo "::error::tag v$TAG_VERSION does not match MARKETING_VERSION $XCCONFIG_VERSION" - exit 1 - fi - echo "tag and MARKETING_VERSION agree on $TAG_VERSION" - - # CURRENT_PROJECT_VERSION is what Sparkle compares, as . A build number - # that did not increase means no installed app is offered the release, and until now only - # a sentence in the release runbook stopped that. Compared against the newest PUBLISHED - # release rather than the newest tag, because re-cutting a release that failed leaves a - # tag at the same build and that is a legitimate state to be in. - BUILD=$(sed -n 's/^CURRENT_PROJECT_VERSION[[:space:]]*=[[:space:]]*//p' Configs/Version.xcconfig | tr -d ' ') - case "$BUILD" in - ''|*[!0-9]*) - echo "::error::CURRENT_PROJECT_VERSION '$BUILD' is not a plain integer" - exit 1 - ;; - esac - - # No `|| true`. A gh outage used to turn this gate into "no earlier published release" - # and exit 0, which is the same output a genuine first release produces. - PREVIOUS_TAG=$(gh release list --limit 100 --json tagName,isPrerelease,publishedAt \ - -q '[.[] | select(.tagName|startswith("v")) | select(.isPrerelease==false)] - | sort_by(.publishedAt) | last | .tagName') - if [ "$PREVIOUS_TAG" = "null" ]; then - PREVIOUS_TAG="" - fi - if [ -z "$PREVIOUS_TAG" ] || [ "$PREVIOUS_TAG" = "v$TAG_VERSION" ]; then - echo "no earlier published release to compare the build number against" - exit 0 - fi - - PREVIOUS_BUILD=$(git show "${PREVIOUS_TAG}:Configs/Version.xcconfig" \ - | sed -n 's/^CURRENT_PROJECT_VERSION[[:space:]]*=[[:space:]]*//p' | tr -d ' ') - if [ -n "$PREVIOUS_BUILD" ] && [ "$BUILD" -le "$PREVIOUS_BUILD" ]; then - echo "::error::CURRENT_PROJECT_VERSION $BUILD is not above $PREVIOUS_BUILD from $PREVIOUS_TAG; no install would be offered this update" - exit 1 - fi - echo "build $BUILD is above $PREVIOUS_BUILD from $PREVIOUS_TAG" - - - name: Verify and organize artifacts for release - run: | - VERSION=${GITHUB_REF#refs/tags/v} - - if [ -z "$VERSION" ]; then - echo "❌ ERROR: Failed to extract version from ref: $GITHUB_REF" - exit 1 - fi - - echo "Preparing artifacts for version: $VERSION" - echo "Contents of artifacts directory:" - ls -la artifacts/ - - # The DMGs are already versioned by package-artifacts.sh; the update archives are not. - # Both renames used to be guarded by `if [ -f ... ]` with no else, so a missing or - # differently named archive printed "organized successfully" and the release published - # whatever the upload glob happened to match. - for ARCH in arm64 x86_64; do - if [ ! -f "artifacts/TablePro-${ARCH}.tar.xz" ]; then - echo "::error::artifacts/TablePro-${ARCH}.tar.xz is missing; the ${ARCH} build did not produce an update archive" - ls -la artifacts/ >&2 - exit 1 - fi - mv "artifacts/TablePro-${ARCH}.tar.xz" "artifacts/TablePro-${VERSION}-${ARCH}.tar.xz" - - if [ ! -f "artifacts/TablePro-${VERSION}-${ARCH}.dmg" ]; then - echo "::error::artifacts/TablePro-${VERSION}-${ARCH}.dmg is missing" - ls -la artifacts/ >&2 - exit 1 - fi - done - - echo "✅ Artifacts organized successfully" - echo "Final artifacts:" - ls -lh artifacts/ - - # No `if: env.SPARKLE_PRIVATE_KEY != ''` on these steps. This job only runs for a v* tag, - # so the key is always meant to be present, and gating on it meant a missing or rotated - # secret published a release that no existing install could ever see. A signing key that - # is not there is now a red job, not a silent skip. - # The tag's own appcast.xml is the feed as of that commit, which is missing every version - # published since. The merge base has to be the feed that is actually live. - - name: Fetch the published feed to merge into - run: | - git fetch origin main - mkdir -p appcast - git show origin/main:appcast.xml > appcast/base.xml - - # A flag left true from the previous release would mark an ordinary build critical and - # interrupt every user, so it has to have been touched in this release's own diff. - - name: Read the release flags - id: flags - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - CRITICAL=$(python3 -c " - import json - print('1' if json.load(open('.github/release-flags.json')).get('criticalUpdate') is True else '0') - ") - echo "critical=$CRITICAL" >> "$GITHUB_OUTPUT" - if [ "$CRITICAL" != "1" ]; then - echo "Ordinary release, staged over 36 hours" - exit 0 - fi - - RELEASES=$(gh release list --limit 100 --json tagName,isPrerelease,publishedAt \ - -q '[.[] | select(.tagName|startswith("v")) | select(.isPrerelease==false)] - | sort_by(.publishedAt) | .[].tagName') - # A re-run after this tag's release went out finds it among the published ones. - PREVIOUS_TAG=$(printf '%s\n' "$RELEASES" | grep -vxF "${GITHUB_REF#refs/tags/}" | tail -1) - if [ -n "$PREVIOUS_TAG" ] && [ "$PREVIOUS_TAG" != "null" ] \ - && git diff --quiet "$PREVIOUS_TAG" HEAD -- .github/release-flags.json; then - echo "::error::criticalUpdate is true but .github/release-flags.json has not changed since $PREVIOUS_TAG, which is what a flag left over from the previous release looks like" - exit 1 - fi - echo "::warning::This release is marked critical and will interrupt every user" - - - name: Sign update archives with Sparkle - env: - SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} - BASE_APPCAST: appcast/base.xml - CRITICAL_UPDATE: ${{ steps.flags.outputs.critical }} - run: scripts/ci/sign-and-appcast.sh "${GITHUB_REF#refs/tags/v}" - - - name: Upload appcast artifact - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 - with: - name: appcast-${{ github.sha }} - path: appcast/appcast.xml - retention-days: 90 - - # Fails before anything is published when the secret does not match a deploy key. - - name: Check the release deploy key - env: - APPCAST_DEPLOY_KEY: ${{ secrets.APPCAST_DEPLOY_KEY }} - run: | - eval "$(ssh-agent -s)" > /dev/null - trap 'ssh-agent -k > /dev/null' EXIT - ssh-add - <<< "$APPCAST_DEPLOY_KEY" - git ls-remote "git@github.com:${GITHUB_REPOSITORY}.git" HEAD > /dev/null - - # A published release is immutable, so every asset goes on the draft first. - - name: Create the draft release - id: draft - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - PRERELEASE: ${{ contains(github.ref, '-beta') || contains(github.ref, '-alpha') || contains(github.ref, '-rc') }} - run: | - TAG="${GITHUB_REF#refs/tags/}" - scripts/ci/draft-release.sh --prerelease "$PRERELEASE" "$TAG" "$TAG" release_notes.md \ - artifacts/*.dmg artifacts/*.tar.xz - - - name: Publish the release - if: steps.draft.outputs.id != '' - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - RELEASE_ID: ${{ steps.draft.outputs.id }} - run: | - gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" -F draft=false \ - --jq '"Published \(.tag_name): \(.html_url)"' - - # Publishing the feed comes after publishing the artifacts it points at, so there is no - # window where Sparkle advertises a download URL that still 404s. A draft's URLs 404 too. - # - # This step used to be continue-on-error, which made a failed push indistinguishable from - # a release that went out clean: every user's updater would keep reporting "up to date" - # for a version that shipped. A rejected push now rebases and retries, and a push that - # still will not land fails the job. - # Pushed with the release deploy key, the ruleset's only bypass actor: a fresh commit on main - # has no checks of its own yet. - - name: Commit appcast.xml to repo - env: - APPCAST_DEPLOY_KEY: ${{ secrets.APPCAST_DEPLOY_KEY }} - run: | - VERSION="${GITHUB_REF#refs/tags/v}" - - if [ ! -f appcast/items/arm64.xml ] || [ ! -f appcast/items/x86_64.xml ]; then - echo "❌ ERROR: the signed per-architecture items are missing, the Sparkle step produced no feed" >&2 - exit 1 - fi - - # The merge below runs after the tree has been reset to main, so the scripts are taken - # from the tag first. Otherwise a release would publish its feed with whatever - # scripts/ci main happened to hold, rather than the code the tag was tested with. - TOOLS=$(mktemp -d) - cp -R scripts/ci "$TOOLS/" - - eval "$(ssh-agent -s)" > /dev/null - trap 'ssh-agent -k > /dev/null' EXIT - ssh-add - <<< "$APPCAST_DEPLOY_KEY" - - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - git checkout -B main - - # The whole attempt is inside the retry, because the merge is what has to be redone when - # main moves. It used to run once against a snapshot taken before the build and then be - # copied over main, which reverted anything pushed in between: a pull-release.py - # withdrawal landing inside that window was undone and the withdrawn build went back on - # offer. The rewind guard could not see it, because a restoration adds versions rather - # than dropping them. Rebasing was no answer either, since the merge rewrites the file - # wholesale and every retry conflicted. - for attempt in 1 2 3; do - git fetch origin main - git reset --hard origin/main - - python3 "$TOOLS/ci/merge-appcast.py" \ - --base appcast.xml \ - --version "$VERSION" \ - --arm64 appcast/items/arm64.xml \ - --x86-64 appcast/items/x86_64.xml \ - --download-prefix "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/download/v${VERSION}/" \ - --out appcast.xml - - # The feed is pruned to the newest releases, so an old version leaving it is the policy - # working. What must never leave is the newest item, the only one any install can be - # offered, or the release being published. - # An empty newest_before is a feed with no items yet, which is a legitimate first - # release rather than something to refuse. - newest_before=$(git show origin/main:appcast.xml \ - | grep -o '[^<]*' | sed 's/.*>//' | head -1) - for required in "$newest_before" "$VERSION"; do - if [ -z "$required" ]; then - continue - fi - if ! grep -q "${required}" appcast.xml; then - echo "❌ ERROR: the generated feed no longer advertises $required" >&2 - exit 1 - fi - done - first=$(grep -o '[^<]*' appcast.xml | sed 's/.*>//' | head -1) - if [ "$first" != "$VERSION" ]; then - echo "❌ ERROR: the feed leads with $first, not the release being published ($VERSION)" >&2 - exit 1 - fi - - git add appcast.xml - if git diff --cached --quiet; then - echo "appcast.xml already advertises v${VERSION}" - exit 0 - fi - - git commit -m "Update appcast.xml for v${VERSION}" - if git push "git@github.com:${GITHUB_REPOSITORY}.git" HEAD:main; then - exit 0 - fi - echo "Push rejected, re-merging onto the current origin/main (attempt ${attempt})" - done - - echo "❌ ERROR: could not push appcast.xml after 3 attempts" >&2 - exit 1 - - - name: Notify Telegram - if: success() && env.TELEGRAM_BOT_TOKEN != '' - env: - TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} - TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }} - TELEGRAM_TOPIC_ID: ${{ secrets.TELEGRAM_TOPIC_ID }} - run: scripts/ci/notify-telegram.sh "${GITHUB_REF#refs/tags/v}" diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml new file mode 100644 index 0000000000..f247191350 --- /dev/null +++ b/.github/workflows/pages.yml @@ -0,0 +1,40 @@ +name: Pages + +# Deploys the feeds branch to updates.tablepro.app. A branch-source Pages site would not do: a push +# made with GITHUB_TOKEN starts no Pages build. Dispatch it at a v* tag, which github-pages requires. + +on: + workflow_call: + workflow_dispatch: + +permissions: + contents: read + +jobs: + deploy: + name: Deploy + runs-on: ubuntu-latest + timeout-minutes: 10 + environment: + name: github-pages + url: ${{ steps.deploy.outputs.page_url }} + # Never cancelled. A newer pending deploy replaces an older pending one, and both deploy the feeds head. + concurrency: + group: pages + cancel-in-progress: false + permissions: + contents: read + pages: write + id-token: write + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: feeds + persist-credentials: false + + - uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 + with: + path: . + + - id: deploy + uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1 diff --git a/.github/workflows/pr-title.yml b/.github/workflows/pr-title.yml index 0c34f11cc5..60fe33b74a 100644 --- a/.github/workflows/pr-title.yml +++ b/.github/workflows/pr-title.yml @@ -12,6 +12,7 @@ concurrency: permissions: contents: read + pull-requests: read jobs: title: @@ -29,6 +30,12 @@ jobs: TITLE: ${{ github.event.pull_request.title }} run: python3 scripts/ci/check-pr-title.py "$TITLE" + - name: Check CHANGELOG.md and the changelog fragments the pull request changes + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + run: python3 scripts/ci/check-changelog.py --pr "$PR" + # The checkout is the merge commit, so its first parent is the base the pull request lands on. - name: Check the added lines and the description against the banned words env: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000000..5a07bdddf8 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,367 @@ +name: Release + +# A v* tag builds, signs and notarizes the app and every registry plugin, publishes the GitHub +# release, then adds it to the feeds branch and deploys that to updates.tablepro.app. + +on: + push: + tags: ['v*'] + +# Never cancelled: the build jobs hold notary submissions, and every step after them is idempotent, +# so a re-run of the same tag finishes what the first run left. +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +permissions: + contents: read + +jobs: + verify: + name: Verify + runs-on: ubuntu-latest + timeout-minutes: 240 + permissions: + contents: read + actions: read + pull-requests: read + outputs: + tag: ${{ steps.parse.outputs.tag }} + version: ${{ steps.parse.outputs.version }} + build: ${{ steps.parse.outputs.build }} + channel: ${{ steps.parse.outputs.channel }} + critical: ${{ steps.notes.outputs.critical }} + env: + TAG: ${{ github.ref_name }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + filter: blob:none + + - name: Set up Python 3.9 + id: python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.9" + update-environment: false + + - name: Install the release tooling + env: + SETUP_PYTHON: ${{ steps.python.outputs.python-path }} + run: | + set -euo pipefail + "$SETUP_PYTHON" -m venv "$RUNNER_TEMP/release-venv" + "$RUNNER_TEMP/release-venv/bin/pip" install --quiet --disable-pip-version-check --require-hashes \ + -r scripts/release/requirements.txt + + - name: Check the tag can build + run: '"$RUNNER_TEMP/release-venv/bin/python" scripts/release/version.py check "$TAG"' + + - name: Read the version + id: parse + run: | + set -euo pipefail + parsed="$("$RUNNER_TEMP/release-venv/bin/python" scripts/release/version.py parse "$TAG")" + printf 'tag=%s\n%s\n' "$TAG" "$parsed" >> "$GITHUB_OUTPUT" + + - name: Write the release notes + id: notes + env: + VERSION: ${{ steps.parse.outputs.version }} + run: | + set -euo pipefail + python="$RUNNER_TEMP/release-venv/bin/python" + mkdir -p build/notes + for kind in body feed lead; do + "$python" scripts/release/notes.py "$kind" "$VERSION" > "build/notes/$kind.md" + done + critical="$("$python" scripts/release/notes.py critical "$VERSION")" + echo "critical=$critical" >> "$GITHUB_OUTPUT" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: notes + path: build/notes + if-no-files-found: error + retention-days: 30 + + app: + name: App + needs: verify + runs-on: xcode-27 + environment: signing + timeout-minutes: 120 + permissions: + contents: read + env: + VERSION: ${{ needs.verify.outputs.version }} + BUILD: ${{ needs.verify.outputs.build }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-xcode + with: + require-release: ${{ needs.verify.outputs.channel == 'stable' }} + + - name: Check the Production CloudKit schema + env: + CLOUDKIT_MANAGEMENT_TOKEN: ${{ secrets.CLOUDKIT_MANAGEMENT_TOKEN }} + run: | + set -euo pipefail + xcrun cktool save-token --type management --method file --force "$CLOUDKIT_MANAGEMENT_TOKEN" + scripts/export-cloudkit-schema.sh --check + + - uses: ./.github/actions/setup-xcodegen + + - name: Download static libraries + env: + GH_TOKEN: ${{ github.token }} + run: scripts/download-libs.sh + + - name: Write Secrets.xcconfig + env: + ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} + run: printf 'ANALYTICS_HMAC_SECRET = %s\n' "$ANALYTICS_HMAC_SECRET" > Configs/Secrets.xcconfig + + - name: Generate the Xcode project + run: scripts/generate-project.sh macos + + # macos-tests.yml saves this cache; a save in tag scope would be unreadable from other refs. + - name: Restore Swift package checkouts + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.spm-cache + key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} + restore-keys: spm-${{ runner.os }}- + + - uses: ./.github/actions/apple-signing + with: + certificate-p12: ${{ secrets.CERTIFICATES_P12 }} + certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} + provisioning-profile: ${{ secrets.PROVISIONING_PROFILE }} + asc-key-id: ${{ secrets.ASC_KEY_ID }} + asc-issuer-id: ${{ secrets.ASC_ISSUER_ID }} + asc-key-p8: ${{ secrets.ASC_KEY_P8 }} + + - name: Install create-dmg + run: brew list create-dmg > /dev/null 2>&1 || brew install create-dmg + + - name: Build the app + run: scripts/release/build.sh app + + - name: Notarize the app + run: scripts/release/notarize.sh build/release/TablePro.app + + - name: Package the app + run: scripts/release/package.sh app "$VERSION" + + - name: Notarize the disk image + run: scripts/release/notarize.sh "dist/TablePro-$VERSION.dmg" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: app + path: dist/ + if-no-files-found: error + retention-days: 30 + + plugins: + name: Plugins + needs: verify + runs-on: xcode-27 + environment: signing + timeout-minutes: 90 + permissions: + contents: read + env: + VERSION: ${{ needs.verify.outputs.version }} + BUILD: ${{ needs.verify.outputs.build }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-xcode + with: + require-release: ${{ needs.verify.outputs.channel == 'stable' }} + + - uses: ./.github/actions/setup-xcodegen + + - name: Download static libraries + env: + GH_TOKEN: ${{ github.token }} + run: scripts/download-libs.sh + + - name: Generate the Xcode project + run: scripts/generate-project.sh macos + + - name: Restore Swift package checkouts + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.spm-cache + key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} + restore-keys: spm-${{ runner.os }}- + + - name: Set up Go for the SAP HANA helper + uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0 + with: + go-version-file: Native/HanaBridge/go.mod + cache: false + + # The Dameng and HANA targets copy these artifacts in and never build them. + - name: Build the native bridges + run: | + scripts/build-dameng.sh both + scripts/build-hana.sh both + + - uses: ./.github/actions/apple-signing + with: + certificate-p12: ${{ secrets.CERTIFICATES_P12 }} + certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} + asc-key-id: ${{ secrets.ASC_KEY_ID }} + asc-issuer-id: ${{ secrets.ASC_ISSUER_ID }} + asc-key-p8: ${{ secrets.ASC_KEY_P8 }} + + - name: Build the registry plugins + run: scripts/release/build.sh plugins + + - name: Notarize the plugins + run: scripts/release/notarize.sh build/release/plugins/*.tableplugin + + - name: Package the plugins + run: scripts/release/package.sh plugins "$VERSION" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: plugins + path: dist/ + if-no-files-found: error + retention-days: 30 + + github-release: + name: GitHub release + needs: [verify, app, plugins] + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: write + id-token: write + attestations: write + env: + TAG: ${{ needs.verify.outputs.tag }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: app + path: dist + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: plugins + path: dist + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + # A published release is immutable, so every asset goes on the draft and is checked there first. + - name: Draft the release + id: draft + run: python3 scripts/release/publish.py draft "$TAG" --notes notes/body.md dist/* >> "$GITHUB_OUTPUT" + + - name: Attest the assets + if: steps.draft.outputs.state == 'draft' + uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2 + with: + subject-path: dist/* + + - name: Publish the release + run: python3 scripts/release/publish.py release "$TAG" + + feed: + name: Feed + needs: [verify, github-release] + runs-on: macos-latest + environment: release + timeout-minutes: 20 + permissions: + contents: write + env: + TAG: ${{ needs.verify.outputs.tag }} + BUILD: ${{ needs.verify.outputs.build }} + CHANNEL: ${{ needs.verify.outputs.channel }} + CRITICAL: ${{ needs.verify.outputs.critical }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-sparkle + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + - name: Add the release to the feed and the plugin manifest + env: + GH_TOKEN: ${{ github.token }} + SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} + run: | + set -euo pipefail + flags=() + [ "$CRITICAL" != true ] || flags+=(--critical) + python3 scripts/release/feed.py add "$TAG" --build "$BUILD" --channel "$CHANNEL" \ + --notes notes/feed.md ${flags[@]+"${flags[@]}"} + + pages: + name: Pages + needs: feed + permissions: + contents: read + pages: write + id-token: write + uses: ./.github/workflows/pages.yml + + notify: + name: Notify + needs: [verify, pages] + if: needs.verify.outputs.channel == 'stable' + runs-on: ubuntu-latest + environment: release + continue-on-error: true + timeout-minutes: 5 + permissions: + contents: read + env: + TAG: ${{ needs.verify.outputs.tag }} + VERSION: ${{ needs.verify.outputs.version }} + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + # Plain text: the lead goes out as written, with no parse_mode to escape for. + - name: Post to Telegram + env: + TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} + TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }} + TELEGRAM_TOPIC_ID: ${{ secrets.TELEGRAM_TOPIC_ID }} + run: | + set -euo pipefail + { + printf 'TablePro %s\n' "$VERSION" + if [ -s notes/lead.md ]; then + printf '\n' + cat notes/lead.md + fi + printf '\n%s/%s/releases/tag/%s\n' "$GITHUB_SERVER_URL" "$GITHUB_REPOSITORY" "$TAG" + } > msg.txt + topic=() + [ -z "$TELEGRAM_TOPIC_ID" ] || topic=(--data-urlencode "message_thread_id=$TELEGRAM_TOPIC_ID") + curl -fsS --retry 3 -o /dev/null --data-urlencode "chat_id=$TELEGRAM_CHAT_ID" \ + --data-urlencode text@msg.txt ${topic[@]+"${topic[@]}"} \ + "https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage" diff --git a/.github/workflows/repo-hygiene.yml b/.github/workflows/repo-hygiene.yml index 0a0368de9a..2d021b5f93 100644 --- a/.github/workflows/repo-hygiene.yml +++ b/.github/workflows/repo-hygiene.yml @@ -42,16 +42,12 @@ jobs: mapfile -t SCRIPTS < <(find scripts .claude/skills -name '*.sh' | sort) shellcheck --severity=warning "${SCRIPTS[@]}" - # update-registry's tests are plain functions, which unittest discovery would skip. - name: Test the CI and release tooling run: | set -euo pipefail python3 -m unittest discover -s scripts/ci -p 'test_*.py' python3 -m unittest discover -s scripts/lib -p 'test_*.py' python3 -m unittest discover -s scripts -p 'test_localization.py' - python3 .github/scripts/test_update_registry.py - bash scripts/ci/test_create_dmg_args.sh - bash scripts/ci/test_draft_release.sh bash scripts/ci/test_libs_release.sh # 3.9 is the macOS /usr/bin/python3 that runs notes.py in the build. The other steps keep the @@ -75,6 +71,13 @@ jobs: - name: Test the release tooling run: '"$RUNNER_TEMP/release-venv/bin/python" -m unittest discover -s scripts/release -p "test_*.py"' + # towncrier refuses a fragment name it cannot parse once its config sets ignore. + - name: Check the changelog fragments and CHANGELOG.md + run: | + set -euo pipefail + "$RUNNER_TEMP/release-venv/bin/python" scripts/ci/check-changelog.py + "$RUNNER_TEMP/release-venv/bin/towncrier" build --draft --version 0.0.0 + # shellcheck cannot tell a library function from a command, so a missing source is exit 127 # at release time (v0.67.1). - name: Check every script sources the libraries it calls into @@ -141,8 +144,7 @@ jobs: go-version-file: Native/HanaBridge/go.mod cache: false - # The helper is pure Go, so Linux analyses it as well as the Mac that builds it. govulncheck - # runs in build-plugin.yml, before a HANA plugin ships. + # The helper is pure Go, so Linux analyses it as well as the Mac that builds it. - name: Check the SAP HANA helper if: steps.native.outputs.hana == 'true' working-directory: Native/HanaBridge @@ -159,3 +161,5 @@ jobs: # go tool would build staticcheck for GOOS too, and Linux cannot run a darwin binary. go build -modfile=tools.mod -o "$RUNNER_TEMP/staticcheck" honnef.co/go/tools/cmd/staticcheck GOOS=darwin "$RUNNER_TEMP/staticcheck" ./... + GOBIN="$RUNNER_TEMP/bin" go install golang.org/x/vuln/cmd/govulncheck@v1.8.0 + GOOS=darwin "$RUNNER_TEMP/bin/govulncheck" ./... diff --git a/.github/workflows/withdraw.yml b/.github/workflows/withdraw.yml new file mode 100644 index 0000000000..a728e5100b --- /dev/null +++ b/.github/workflows/withdraw.yml @@ -0,0 +1,49 @@ +name: Withdraw + +# Removes a version from the feed and the plugin manifest, marks the highest stable left as Latest, +# and redeploys. Dispatch it at the newest v* tag, which the release environment requires. +# The GitHub release stays. + +on: + workflow_dispatch: + inputs: + version: + description: The version to withdraw, such as 0.82.1 or 0.83.0-beta.2. + required: true + type: string + +permissions: + contents: read + +jobs: + withdraw: + name: Withdraw + runs-on: macos-latest + environment: release + timeout-minutes: 20 + permissions: + contents: write + env: + VERSION: ${{ inputs.version }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-sparkle + + - name: Remove the version from the feed and the plugin manifest + env: + SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} + run: python3 scripts/release/feed.py withdraw "$VERSION" + + - name: Mark the highest stable as Latest + run: python3 scripts/release/publish.py latest + + pages: + name: Pages + needs: withdraw + permissions: + contents: read + pages: write + id-token: write + uses: ./.github/workflows/pages.yml diff --git a/.gitignore b/.gitignore index c71089fc73..a78a15eff8 100644 --- a/.gitignore +++ b/.gitignore @@ -145,6 +145,9 @@ Derived/ *.xcarchive /dist/ +# towncrier venv for scripts/release/notes.py (RELEASING.md, Notes) +/.venv/ + # Static libraries (downloaded from GitHub Releases via scripts/download-libs.sh) Libs/*.a Libs/.downloaded diff --git a/AGENTS.md b/AGENTS.md index 1c35aa67aa..86364dea00 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,7 +31,7 @@ Every capability is core, a module or a plugin. The first rule that fits decides ## Repository map - `TablePro/`: the app (`Core/`, `Views/`, `Models/`, `ViewModels/`, `Extensions/`, `Theme/`). -- `Plugins/`: drivers and import/export formats as `.tableplugin` bundles, plus `TableProPluginKit`. Bundled plugins are the app's `copy: { destination: plugins }` targets in `project.yml`; the rest ship through TableProApp/plugins, and the app never offers a registry install or update over a bundled one. +- `Plugins/`: drivers and import/export formats as `.tableplugin` bundles, plus `TableProPluginKit`. Bundled plugins are the app's `copy: { destination: plugins }` targets in `project.yml`; the rest ship as assets of each app release, and the app never offers a registry install or update over a bundled one. - `Packages/`: `TableProCore` (both apps), `TableProMacCore` (macOS-only modules; never in the iOS project), `TableProOracle`, `TableProEditor`, `TableProGrammars`. `TableProCore`'s PluginKit source is a symlink: edit `Plugins/TableProPluginKit/` only. Each manifest sets its own Swift language mode, so never pass `SWIFT_VERSION=` to `xcodebuild`. - Tests: `TableProTests/`, `TableProUITests/`, `Plugins/*Tests/`, `Packages/*/Tests/`, `TableProMobile/TableProMobileTests/`; plans in `TestPlans/`. Which holds what: `.claude/rules/tests.md`. - `Native/`: first-party Rust and Go bridges (`.claude/rules/native-bridges.md`). @@ -101,7 +101,7 @@ scripts/generate-project.sh [both|macos|ios] # needs the libs; rerun after edit ## Every change 1. **Tests.** Every testable change has a test in the target owning the code. Read `.claude/rules/tests.md` before creating a test file: a path rule loads only once a matching file is open. A UI flow only for a new or changed end-to-end flow (`.claude/rules/ui-tests.md`). -2. **CHANGELOG.md.** A user-visible change gets one fragment under `[Unreleased]`; format: `.claude/rules/changelog.md`. +2. **Changelog.** A user-visible change adds `changelog.d/..md`; only a release PR edits `CHANGELOG.md`. Format: `.claude/rules/changelog.md`. 3. **Localization.** `String(localized:)` outside SwiftUI literals, never interpolated: `String(format: String(localized: "Preview %@"), name)`. Technical terms stay unlocalized. A plugin or package string lives in the host app's catalog: add it with `python3 scripts/localization.py plugins --add`, plus `--target ios` for a package the iOS app links. Never hand-edit a `.xcstrings`. 4. **Docs.** A new shortcut, UI or settings change, or driver change gets a paired draft PR in TableProApp/docs naming this one; it merges after this one. `verify.sh docs` runs its checks here first. 5. **Lint** each changed Swift file with `verify.sh lint`. Run `verify.sh agent-docs` when `CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`, `RELEASING.md`, `.github/pull_request_template.md` or anything under `.claude/` changed. @@ -118,7 +118,7 @@ scripts/generate-project.sh [both|macos|ios] # needs the libs; rerun after edit ## CI -- A PR merges on five green checks: macOS Tests, iOS and Package Tests, SwiftLint, Repo Hygiene and PR title. Only release deploy keys bypass them (`RELEASING.md`, Keys and rulesets). +- A PR into `main` or `release/*` merges on five green checks: macOS Tests, iOS and Package Tests, SwiftLint, Repo Hygiene and PR title. Nothing bypasses them. - Read why a CI check failed before any re-run. - Workflows: `.claude/rules/ci.md`. Red checks: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. @@ -129,8 +129,8 @@ Claude Code loads a rule when it reads or edits a file the rule covers. Under Co | Rule | Read before editing | | --- | --- | | `ai-mcp-security.md` | `TablePro/Core/AI`, `TablePro/Core/MCP`, `Packages/TableProMacCore/Sources/TableProMCP` | -| `changelog.md` | `CHANGELOG.md` | -| `ci.md` | `.github/workflows`, `.github/actions`, `.github/scripts`, `.github/actionlint.yaml`, `.github/*baseline*`, `scripts/ci`, `.swiftlint.yml`, `**/.swiftlint.yml` | +| `changelog.md` | `CHANGELOG.md`, `changelog.d`, `towncrier.toml` | +| `ci.md` | `.github/workflows`, `.github/actions`, `.github/actionlint.yaml`, `.github/*baseline*`, `scripts/ci`, `scripts/release`, `.swiftlint.yml`, `**/.swiftlint.yml` | | `connection-window.md` | `TablePro/Core/Services/Infrastructure`, `TablePro/Models/Connection`, `TablePro/Core/Database/DatabaseManager*.swift`, `Packages/TableProCore/Sources/TableProDatabase`, `TablePro/Views/Main` | | `data-grid.md` | `TablePro/Views/Results`, `TablePro/Core/DataGrid`, `TableProTests/Views/Results` | | `data-sync-storage.md` | `TablePro/Core/**/*Storage*.swift`, `TablePro/Core/**/*Sync*.swift`, `TablePro/Core/Database`, `TablePro/Core/Plugins/PluginMetadataRegistry*.swift`, `TablePro/Core/Services/Query`, `Packages/TableProCore/Sources/TableProSyncTransport`, `CloudKit` | @@ -139,7 +139,7 @@ Claude Code loads a rule when it reads or edits a file the rule covers. Under Co | `mongodb-driver.md` | `Plugins/MongoDBDriverPlugin`, `Plugins/MQLExportPlugin` | | `mysql-driver.md` | `Plugins/MySQLDriverPlugin`, `Packages/TableProCore/Sources/TableProMySQLCore`, `TableProMobile/**/*MySQL*`, `TableProMobile/**/*MariaDB*` | | `native-bridges.md` | `Native`, `Plugins/DamengDriverPlugin`, `Plugins/HanaDriverPlugin`, `scripts/build-dameng.sh`, `scripts/build-hana.sh` | -| `plugin-system.md` | `Plugins`, `Packages/TableProCore/Sources/TableProPluginKit`, `Packages/TableProCore/Sources/TableProMySQLCore`, `Packages/TableProCore/Sources/TableProPostgreSQLCore`, `Packages/TableProCore/Sources/TableProRedisCore`, `Packages/*/Package.swift`, `project.yml`, `TableProMobile/project.yml`, `.github/workflows/build-plugin.yml`, `.github/plugin-registry.json`, `scripts/*plugin*` | +| `plugin-system.md` | `Plugins`, `Packages/TableProCore/Sources/TableProPluginKit`, `Packages/TableProCore/Sources/TableProMySQLCore`, `Packages/TableProCore/Sources/TableProPostgreSQLCore`, `Packages/TableProCore/Sources/TableProRedisCore`, `Packages/*/Package.swift`, `project.yml`, `TableProMobile/project.yml`, `Configs/PluginKit.xcconfig`, `.github/plugin-registry.json`, `scripts/*plugin*` | | `redis-driver.md` | `Plugins/RedisDriverPlugin`, `Packages/TableProCore/Sources/TableProRedisCore` | | `split-views.md` | `TablePro/**/*Split*.swift`, `TablePro/Core/Services/Infrastructure/WorkspacePane*.swift` | | `tests.md` | `TableProTests`, `TableProUITests`, `Plugins/*Tests`, `Packages/*/Tests`, `TableProMobile/TableProMobileTests`, `TestPlans`, `scripts/ci/check-test-plans.py`, `scripts/clean-test-residue.sh` | diff --git a/CHANGELOG.md b/CHANGELOG.md index 15c4be2ab1..e031de04a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,85 +5,9 @@ All notable changes to TablePro will be documented in this file. The format is based on [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +Unreleased changes are files in `changelog.d/`. -### Added - -- Generate UUID and Generate UUIDv7 in Set Value, the row inspector and Fill Column, a new value for each cell. (#3443) -- Edit > Set Value in the menu bar, with rows in Settings > Keyboard. (#3443) -- New UUID in a UUID primary key the server does not fill, on Add Row, Duplicate Row and Paste. (#3443) -- Solarized, GitHub, Catppuccin, Gruvbox and Tokyo Night themes in light and dark, plus One Dark and One Light. -- VS Code color theme import in Settings > Appearance, from the theme's `.json` file. -- Theme Editor window with a live preview, undo and Revert Changes. Open windows show each edit as you make it. -- iCloud sync of custom themes and the chosen light and dark themes, with Settings on in Settings > Sync. -- Theme import by dropping files on the Themes list, and export by dragging a theme to Finder. -- Get beta updates in Settings > General, from a signed update feed at updates.tablepro.app. - -### Changed - -- Set Value applies to every selected cell, or to the clicked column across the selected rows. (#3443) -- SQL functions in Set Value and the row inspector listed per database and only where they fit the column. (#3443) -- UUID functions in SQL Functions replaced by Generate UUID, which writes the value itself. (#3443) -- Text typed into a column that is not a date or time saved as typed, even when it reads like `now()`. (#3443) -- Set NULL and Set DEFAULT in the row inspector shown only where the column takes them, as in the data grid. (#3443) -- Default only on new rows on SQLite, libSQL, Turso, D1, ClickHouse, Databend, Snowflake, Trino, Cassandra. (#3443) -- Pending SQL functions in the data grid drawn like a pending default. (#3443) -- PluginKit ABI 39, additive: existing plugins keep loading. (#3443) -- Appearance (System, Light or Dark) kept per Mac instead of syncing. Older versions that sync settings switch to System. -- Data Files, Compare & Sync and Open in Window value windows show the theme in the titlebar on macOS 26 and later. -- SQL aliases and parameters drawn in the editor text color instead of the NULL color. -- SQL previews in history, filters, maintenance and review sheets highlighted as in the editor, in the editor font. -- Server Dashboard sessions and slow queries shown as data grids, with Cancel Query and Terminate Session in the header. -- Terminate Session and Cancel Query confirmations name the session's process ID and user. -- Compare & Sync status bar shows the target connection's icon instead of a colored dot. -- Themes move to the Trash instead of being deleted. Undo reverts a trash, rename, duplicate or import. -- Editor tab strip over a themed window on macOS 26 and later drawn in a system fill instead of the theme color. -- Plugin download progress shown as the bytes arrive instead of jumping to full at the end. -- Welcome window and What's New show the build number after the version. -- Registry drivers such as Oracle, SQL Server and MongoDB load on first use instead of at launch. - -### Fixed - -- Every ClickHouse query after connecting failing with `UNKNOWN_SETTING` on 23.8.0 to 23.8.8. (#3444) -- ClickHouse users with a read-only profile unable to connect. -- ClickHouse indexes, view definitions, functions, table comments and sizes failing to load on servers before 22.3. -- A ClickHouse query that failed partway showing its partial rows and the error text as a row instead of the error. -- A ClickHouse query with a `FORMAT` clause showing garbled rows in the grid and in an export. -- ClickHouse SQL export writing the current database's table structure for a table in another database. -- Opening a ClickHouse executable function's definition showing a syntax error. -- SQL Server 2016 RTM offered a `CREATE OR ALTER VIEW` template it cannot run, and Azure SQL a drop-and-create one. -- `gen_random_uuid()`, `UUID()`, `NEWID()` and `datetime('now')` from SQL Functions saved as quoted text. (#3443) -- SQL functions saved as text on SQL Server, Teradata, Spanner and BigQuery. (#3443) -- `NOW()` offered on SQLite, Oracle, Cassandra and MongoDB, where it fails or saves the text. (#3443) -- SQLite's `datetime('now')` offered on DuckDB and `gen_random_uuid()` on Redshift. (#3443) -- Stored text such as `CURRENT_DATE` turning into today's date on Duplicate Row or Restore Previous Values. (#3443) -- Copy as INSERT and Copy as UPDATE quoting a staged SQL function. (#3443) -- Default on a saved row failing on SQLite, and storing `__DEFAULT__` on Teradata and DynamoDB. (#3443) -- Duplicate Row and Paste failing to save on a UUID primary key with no default. (#3443) -- Date functions and a date picker offered on Cassandra and ScyllaDB `timeuuid` columns. (#3443) -- MariaDB `uuid` columns read as `CHAR`. (#3443) -- Empty field in the multi-row inspector after the selected rows were given different values. (#3443) -- Inline cell editor, the array editor's Edit as Text, and the inspector's hex dump and map pane ignoring the theme. -- No-match, invalid and NULL states in the JSON, PHP and array viewers drawn without the theme's background. -- Agent result states and proposed SQL ignoring the theme. -- Connecting and Not connected panes, and Agent mode, in the system window color under a theme before macOS 26. -- Selected text in SQL previews, such as query history, ignoring the theme's selection color. -- Server Dashboard lists and the Query Insights chart ignoring the theme. -- Compare & Sync row states and Data Files loading and failure states drawn without the theme's grid background. -- Structure and Create Table code placeholders, Startup SQL in a link and the tunnel command preview ignoring the theme. -- Bars in query, history, filter, structure, plan, compare and other panes drawing a system band over a themed window. -- System header band on the data grid and the Query Plan and Users & Roles trees under a theme with header colors. -- System-colored alternate rows, header and booleans in a theme that sets only the grid background and text. -- Selected row text in white over a theme's selection color, or in the theme's color over the system selection. -- Translucent theme selection, as in Dracula and Nord, in a different shade on rows, row numbers and column headers. -- No outline around a selected cell range in Dracula and Nord. -- Shapes in the result Map view keeping the previous Accent color after it changes. -- Edit as Text in the array editor turning straight quotes and dashes into curly quotes and long dashes. -- Theme editor bringing back a theme file deleted in Finder on the next change. -- Changes to a theme file made outside TablePro not shown until Settings > Appearance was opened. -- Theme files with a JSON syntax error reported without the line and column. -- Two themes with the same name impossible to tell apart in the theme pickers. -- No warning on a connection in the Welcome window whose driver plugin failed to load or is out of date. + ## [0.81.0] - 2026-10-16 @@ -5956,7 +5880,6 @@ TablePro is a native macOS database client built with SwiftUI and AppKit, design - Custom SQL query templates - Performance optimized for large datasets -[Unreleased]: https://github.com/TableProApp/TablePro/compare/v0.81.0...HEAD [0.81.0]: https://github.com/TableProApp/TablePro/compare/v0.80.0...v0.81.0 [0.80.0]: https://github.com/TableProApp/TablePro/compare/v0.79.0...v0.80.0 [0.79.0]: https://github.com/TableProApp/TablePro/compare/v0.78.0...v0.79.0 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ff3f0c45e6..03d1253aca 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,7 +39,7 @@ TABLEPRO_DEVELOPMENT_TEAM = TABLEPRO_APP_BUNDLE_IDENTIFIER = com..TablePro ``` -A free personal team works. Debug builds sign against `TablePro/TablePro.Debug.entitlements`, which drops iCloud, so sync turns itself off. Plugin bundles and the MCP helper sign ad hoc. The file is gitignored, so these settings never reach a commit and survive regenerating the project. Signing changed in the Xcode UI does not: the next `scripts/generate-project.sh` discards it. To check it, save a connection password, relaunch, and reopen the connection: the password is still there. +A free personal team works. Debug builds sign against `TablePro/TablePro.Debug.entitlements`, which drops iCloud, so sync turns itself off. Plugin bundles sign with the same team, and the MCP helper signs ad hoc. The file is gitignored, so these settings never reach a commit and survive regenerating the project. Signing changed in the Xcode UI does not: the next `scripts/generate-project.sh` discards it. To check it, save a connection password, relaunch, and reopen the connection: the password is still there. ## Building @@ -57,7 +57,7 @@ The app scheme compiles only the plugins the app embeds. The `AllPlugins` aggreg xcodebuild build -project TablePro.xcodeproj -scheme AllPlugins -destination 'platform=macOS' -skipPackagePluginValidation CODE_SIGNING_ALLOWED=NO ``` -`scripts/build-plugin.sh ` builds one signed, packaged bundle. Target names are the keys under `targets:` in `project.yml`, such as `OracleDriver`. +`scripts/release/build.sh plugins --unsigned` builds the registry plugins the way a release does, universal, into `build/release/plugins`: [`RELEASING.md`](RELEASING.md#reproducing-a-release-build). ### Clean builds @@ -113,14 +113,14 @@ The architecture and its invariants: [`AGENTS.md`](AGENTS.md), Architecture and ## Branch naming -Branch off `main`: `feat/add-cassandra-support`, `fix/query-editor-crash`. +Branch off `main`: `feat/add-cassandra-support`, `fix/query-editor-crash`. PRs target `main`; a `release/` line takes only cherry-picks of fixes already on `main` ([`RELEASING.md`](RELEASING.md#branches)). ## Pull requests One logical change per PR, described with the template the PR form opens with. Before you open it: - Tests added or updated. -- `CHANGELOG.md` updated under `[Unreleased]`, with the credit left off: the release adds it (`.claude/rules/changelog.md`). +- A changelog fragment in `changelog.d/` for a user-visible change (`.claude/rules/changelog.md`). - User-facing behavior documented in a paired pull request in [TableProApp/docs](https://github.com/TableProApp/docs) that links this one. - User-facing strings localized. - SwiftLint clean. @@ -154,7 +154,7 @@ The export builds the app first, so run `scripts/download-libs.sh` and `scripts/ | A new source file does not compile | Run `scripts/generate-project.sh` | | `Undefined symbols for architecture arm64` | `Libs/` is empty or stale: `scripts/download-libs.sh`, or `--force` to overwrite | | `Libs does not match the checksums committed in git.` | Re-run `scripts/download-libs.sh --force`. A library rebuilt on purpose is published as [`RELEASING.md`](RELEASING.md#static-libraries) describes | -| `Build input file cannot be found: '…/tablepro-hana-helper'`, or the helper has no slice for your architecture | Run `scripts/build-hana.sh` with no argument, which builds both architectures | +| `error: …/tablepro-hana-helper is missing or has no slice` | Run `scripts/build-hana.sh` with no argument, which builds both architectures | | Signing errors after a fresh clone | `Configs/Base.xcconfig` defaults to the maintainer's team and includes `Secrets.xcconfig` optionally, so a build without it still runs and fails to sign. Create the file with your team | | Package resolution fails | `Cmd+Shift+K` and reopen Xcode; then `rm -rf ~/Library/Developer/Xcode/DerivedData` | | `Timed out while enabling automation mode`, and UI tests run nothing | Answer the Enable UI Automation prompt with an admin password, or run `sudo automationmodetool enable-automationmode-without-authentication` once | diff --git a/Configs/Base.xcconfig b/Configs/Base.xcconfig index 323f91bffe..9f76303ed8 100644 --- a/Configs/Base.xcconfig +++ b/Configs/Base.xcconfig @@ -68,10 +68,7 @@ CODE_SIGN_STYLE = Automatic TABLEPRO_DEVELOPMENT_TEAM = D7HJ5TFYCU TABLEPRO_APP_BUNDLE_IDENTIFIER = com.TablePro -// Empty by default. scripts/build-release.sh passes a profile UUID on the xcodebuild -// command line; because only the app target reads this variable, the profile cannot -// leak into plugin or Swift package targets the way a bare -// PROVISIONING_PROFILE_SPECIFIER override would. +// Only the app target reads this, so a profile passed here never reaches a plugin or package target. TABLEPRO_PROVISIONING_PROFILE = #include? "Secrets.xcconfig" diff --git a/Configs/Release.xcconfig b/Configs/Release.xcconfig index 8aea3fe798..dc3d6989a0 100644 --- a/Configs/Release.xcconfig +++ b/Configs/Release.xcconfig @@ -5,10 +5,9 @@ ENABLE_NS_ASSERTIONS = NO MTL_ENABLE_DEBUG_INFO = NO SWIFT_COMPILATION_MODE = wholemodule -// scripts/build-release.sh used to pass these three on the xcodebuild command line, where an -// override reaches every target in the graph, SPM packages included. They belong to this project's -// Release configuration and nothing else. LLVM_LTO is the Clang knob: it reaches the nine C bridge -// modules, not Swift, which has its own SWIFT_LTO and leaves it off. +// Here rather than on the xcodebuild command line, where an override reaches every target in the +// graph, SPM packages included. LLVM_LTO is the Clang knob: it reaches the C bridge modules, not +// Swift, which has its own SWIFT_LTO and leaves it off. GCC_OPTIMIZATION_LEVEL = s SWIFT_OPTIMIZATION_LEVEL = -O LLVM_LTO = YES_THIN diff --git a/Configs/Version.xcconfig b/Configs/Version.xcconfig deleted file mode 100644 index fa8e3b186e..0000000000 --- a/Configs/Version.xcconfig +++ /dev/null @@ -1,6 +0,0 @@ -// Single source of truth for the macOS app version. -// The release skill and .github/workflows/build-plugin.yml read this file. -// Do not restate these values anywhere else. - -MARKETING_VERSION = 0.81.0 -CURRENT_PROJECT_VERSION = 140 diff --git a/Plugins/HanaDriverPlugin/README.md b/Plugins/HanaDriverPlugin/README.md index f78fc008df..0a7efd020c 100644 --- a/Plugins/HanaDriverPlugin/README.md +++ b/Plugins/HanaDriverPlugin/README.md @@ -4,7 +4,7 @@ This plugin connects TablePro to SAP HANA Cloud and SAP HANA 2.0 through [SAP/go The Go bridge in `Native/HanaBridge` is built by `scripts/build-hana.sh` into `tablepro-hana-helper`, an executable with no cgo. The plugin carries it in `Contents/MacOS`, starts one helper for each session, stops it when the session closes, and talks to it over stdin and stdout in length-prefixed JSON frames. The metadata connection pool opens drivers of its own, so one connection can run several helpers. A crash in go-hdb ends that driver's session, not TablePro. The helper owns the HANA session and returns JSON, and the Swift side writes every message the user reads. -Xcode copies the helper into the bundle and does not build it. Run `scripts/build-hana.sh` with no argument, which builds a universal arm64 and x86_64 helper, before building `HanaDriver` or `AllPlugins`. A thin build, `scripts/build-hana.sh arm64` or `x86_64`, only works in a build of that one architecture. `scripts/build-plugin.sh` signs the helper with Developer ID, the hardened runtime and a timestamp before it signs the plugin. +Xcode copies the helper into the bundle and does not build it. Run `scripts/build-hana.sh` with no argument, which builds a universal arm64 and x86_64 helper, before building `HanaDriver` or `AllPlugins`. A thin build, `scripts/build-hana.sh arm64` or `x86_64`, only works in a build of that one architecture. The copy phase signs the helper with the plugin's identity, so a release build gives it Developer ID, the hardened runtime and a timestamp. What the plugin does: diff --git a/RELEASING.md b/RELEASING.md index 03f003853a..bf3b904bc9 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,182 +1,150 @@ # Releasing -`scripts/build-release.sh` is the whole release build: it signs every nested binary (plugins, -helpers, frameworks) and the app bundle last, so signing again by hand undoes its work. +This runbook covers the macOS app, its registry plugins and the static libraries; the iOS app ships by hand. The `release` skill (`.claude/skills/release/SKILL.md`) decides what ships and points at the sections here. Every command below is the maintainer's to run. -Everything here needs the maintainer's credentials; a contributor wants `CONTRIBUTING.md`, Building. -The `release` skill in `.claude/skills/release` runs a release on top of this file and owns the -procedure and its decisions. +## Channels -## Release builds +- Stable and beta share one appcast, `https://updates.tablepro.app/appcast.xml`. A beta item carries `beta`. +- The app takes betas only while Settings > General > Get beta updates is on (`allowedChannels(for:)`). The key, `com.TablePro.updates.receivesBetaUpdates`, can be pinned by a configuration profile ([Managed updates](https://docs.tablepro.app/security/managed-updates)). +- Sparkle never downgrades: with the toggle off, a beta install keeps its beta until a stable passes it. So a beta never writes storage the previous stable cannot read (`.claude/rules/data-sync-storage.md`). +- A stable rolls out in phases of 21600 s. A beta is not phased, and on GitHub it is a prerelease. Latest goes only to the highest stable, so a patch on an older line publishes with `--latest=false`. +- Critical means a security fix or data loss, nothing else. It is ` [CRITICAL]` on the version's `CHANGELOG.md` heading, never on a beta. Every later stable item carries the highest critical build forward (`sparkle:criticalUpdate sparkle:version`). +- CI writes the orphan `feeds` branch, which holds `appcast.xml` and the v3 plugin manifest (`https://updates.tablepro.app/plugins/v3.json`), and `pages.yml` deploys it to `updates.tablepro.app`. The appcast is EdDSA-signed, and the app requires that (`SURequireSignedFeed`, `SUVerifyUpdateBeforeExtraction`). +- The legacy `appcast.xml` on `main` and `plugins.json` in TableProApp/plugins serve apps up to 0.81, and no tag writes them. v0.82.0 is the one bridge: once it publishes, a PR copies its feed item verbatim into the legacy `appcast.xml` and a PR in TableProApp/plugins points `plugins.json` at its plugin zips. Both files then freeze. -```bash -scripts/build-release.sh arm64 # or x86_64, or both (the default with no argument) -``` +## Versions -Output lands in `build/Release/TablePro-arm64.app` and `build/Release/TablePro-x86_64.app`. +- A tag is `vX.Y.Z` or `vX.Y.0-beta.N`, N from 1 to 98. There is no `-rc`. +- `MARKETING_VERSION` is the tag without the `v`. The build, `CURRENT_PROJECT_VERSION`, is `((MAJOR*1000+MINOR)*100+PATCH)*100+R`, where R is N for a beta and 99 for a stable: `v0.82.0-beta.1` is 820001, `v0.82.0` is 820099, `v0.82.1` is 820199. MINOR stays at most 999 and PATCH at most 99. +- A local build is `0.0.0-dev`, build 1 (`project.yml`, `settings.base`). `scripts/release/version.py parse ` prints a tag's version, build, channel and line. Every "is newer" check, in the app and in the plugin manifest, compares the build (`CFBundleVersion`). +- Every X.Y.0 ships at least one published `vX.Y.0-beta.N` first; patches go straight to stable. Promote the newest beta when it has no open `regression` issue. +- One universal build, arm64 and x86_64. It becomes arm64-only once the deployment target reaches macOS 27. -The script extracts the matching slice from the universal static libraries, copies every non-system -dynamic library the binaries reference into `Contents/Frameworks` and rewrites their install names, -strips the binaries, then signs with hardened runtime and `TablePro/TablePro.entitlements`. +## Branches -| Variable | Default | Purpose | -|---|---|---| -| `SIGN_IDENTITY` | the maintainer's Developer ID Application | codesign identity for every binary | -| `NOTARIZE` | `false` | `true` notarizes and staples after the build | -| `NOTARY_PROFILE` | `TablePro` | `notarytool` keychain profile holding the credentials | +- An admin cuts `release/` from a green `main` commit at beta.1. Only admins create `release/*` branches. +- Every fix lands on `main` first, then reaches the line through a cherry-pick PR from `backport/-`: one fix per PR, titled with the original subject and its ` (#N)`, which is the provenance. A release PR comes from `prep/v`. No PR head branch lives under `release/*`. +- A line takes patches until `X.(Y+1).0` ships, then only security or data-loss fixes. At most two lines are live. +- After a stable or patch publishes, a PR with the same title cherry-picks its release commit to `main`, and to a newer open line when one exists. The next tag refuses to build until it has. -Create the notary profile once with `xcrun notarytool store-credentials "TablePro"` plus -`--apple-id`, `--team-id` and `--password`. `scripts/lib/notarize.sh` zips, submits, waits and -staples. It reads both the exit status and the reported status, because `notarytool` has exited 0 -on a rejected submission. +## Notes -## Packaging a DMG +- Fragments in `changelog.d/` (`.claude/rules/changelog.md`) become a `CHANGELOG.md` section. The tooling needs towncrier: `python3 -m venv .venv && .venv/bin/pip install --require-hashes -r scripts/release/requirements.txt`. +- `python3 scripts/release/notes.py assemble X.Y.Z [--lead FILE] [--critical] [--date YYYY-MM-DD]` runs on `prep/v`. It refuses a beta, a branch with commits beyond `origin/release/` other than a revert, an empty `changelog.d/`, and a version `CHANGELOG.md` already has. It credits each fragment ` by @login` when its PR's author is not an OWNER or MEMBER (read with `gh`), runs `towncrier build`, which deletes the fragments, adds the lead and the critical marker, and stages `CHANGELOG.md`. +- The lead: X.Y.0 only, at most 6 plain lines, naming what a reader would change their behavior over. The Sparkle notes, What's New and the Telegram post show only the lead. +- A beta gets no section. Its GitHub release body and feed notes are a towncrier draft of the fragments at the tag. -`scripts/create-dmg.sh [source app]` packages one architecture. The source -app defaults to the one `build-release.sh` wrote for that architecture. An empty version reads -`MARKETING_VERSION` from `Configs/Version.xcconfig`, the only place the app version is declared. +## Cutting a beta + +beta.1 cuts the line at the newest `main` commit whose push runs are all green; beta.N tags the line's head once the picks are on it. ```bash -scripts/create-dmg.sh 0.80.0 arm64 # writes build/Release/TablePro-0.80.0-arm64.dmg +git fetch origin +git push origin :refs/heads/release/X.Y # beta.1 only +git tag -s vX.Y.0-beta.N -m vX.Y.0-beta.N origin/release/X.Y +git push origin vX.Y.0-beta.N ``` -It prefers `create-dmg` (`brew install create-dmg`) and falls back to `hdiutil`. `NOTARIZE=true` -notarizes the DMG too. +## Cutting a stable or patch -## Static libraries +```bash +git fetch origin +git switch -c prep/vX.Y.Z origin/release/X.Y +python3 scripts/release/notes.py assemble X.Y.Z --lead lead.txt # --lead on X.Y.0 only, --critical when it is +git commit -m "chore(release): vX.Y.Z" +git push -u origin prep/vX.Y.Z +gh pr create --base release/X.Y --title "chore(release): vX.Y.Z" --body "Release notes for vX.Y.Z." +``` -The `.a` files under `Libs/` and the xcframeworks under `Libs/ios/` live on the immutable -`libs-` GitHub release that `Libs/release` names, not in git. `scripts/download-libs.sh` fetches -it and verifies every file against the checksums at `HEAD` on every run. Publish a rebuilt library -through the script, then commit what it writes: +The PR changes only `CHANGELOG.md` and `changelog.d/`. Squash-merge it, then tag its merge commit: ```bash -scripts/publish-libs.sh libmongoc_arm64.a libmongoc_x86_64.a libmongoc_universal.a libmongoc.a -git add Libs/release Libs/checksums.sha256 Libs/ios/checksums.sha256 -git commit -m "build(libs): publish libs-" +git fetch origin +git tag -s vX.Y.Z -m vX.Y.Z "$(gh pr view --json mergeCommit -q .mergeCommit.oid)" +git push origin vX.Y.Z ``` -- Name every library you rebuilt, added or removed. The script verifies every other one against the - checksums at `HEAD`, creates the next `libs-` and rewrites `Libs/release` and both checksum files. -- `--dry-run` prints what it would publish. -- A published `libs-` never changes; a fix is the next ``. - -## The release pipeline - -`.github/workflows/build.yml` runs on a `v*` tag push or a manual dispatch, and a run is never -cancelled (concurrency group `build-tablepro-`). Every job except the two builds needs a `v*` -ref, so a dispatch from a branch only builds. The tag runs no test suite. - -- `Tested on main` (Ubuntu) fails when the tagged commit is not an ancestor of `origin/main`. -- For `macOS Tests` and `iOS and Package Tests` separately, it walks up to 50 first-parent commits - back from the tag. On each it takes the newest push run of that workflow and waits for it if it is - still going. A commit with no run, or whose job was skipped, is passed over; any conclusion other - than success fails the release. -- A passing run counts only when `scripts/ci/detect-changed-paths.sh --base ` - prints `false`, so nothing the suite reads changed between that commit and the tag. The step - summary names the covering run for each suite. -- The two suites may be covered by different commits. Requiring one commit for both would fail - whenever only one suite ran later. -- `Build arm64` and `Build x86_64` run on `xcode-27` with no fail-fast; x86_64 cross-compiles on an - arm64 runner. Each runs `scripts/build-release.sh` with notarization, then makes the DMG and the - tar.xz update archive. -- `Registry Readiness` (Ubuntu) runs `scripts/check-registry-readiness.py` with `--floor` and - `--current` set to the two kit versions in `TablePro/Core/Plugins/PluginManager.swift`. It fails - when a registry driver has no binary in that range, so the app never ships ahead of its plugins. -- `Create GitHub Release` needs all three and runs under the `release` environment. It fails when - the tag disagrees with `MARKETING_VERSION` (`tag v... does not match MARKETING_VERSION ...`). -- It also fails when `CURRENT_PROJECT_VERSION` is not above the newest published release's. Sparkle - compares that number as ``, so a flat one ships a release no install is offered. -- It signs both update archives with the Sparkle key and takes the notes from `CHANGELOG.md`. -- It checks that the appcast deploy key works before publishing anything. Then it puts both DMGs - and both update archives on a draft release, publishes the draft in one step, and only then - commits `appcast.xml`, whose download URLs 404 until the release is public. -- The appcast is merged with the tag's own copy of `scripts/ci/merge-appcast.py` and pushed with - `APPCAST_DEPLOY_KEY`, up to 3 attempts, each re-merging onto the current `origin/main`. -- `generate_appcast` sees only the new release. The merge script splices its items in and moves - published items byte for byte, because Sparkle rewrites an existing item's download URL and drops - its notes when a staged archive matches it. -- A tag containing `-alpha`, `-beta` or `-rc` publishes as a prerelease. - -## Keys and rulesets - -- `main` takes no direct push. Its ruleset requires the checks `AGENTS.md` lists under CI, and - deploy keys bypass it, so the appcast push and the release commit push need no check run. -- The appcast push uses `APPCAST_DEPLOY_KEY`, a write deploy key held by the `release` environment, - which only `v*` tags can use. -- The release commit and a withdrawal push with the release key, `~/.ssh/tablepro_release`, a second - write deploy key. The release skill's pre-flight step 7 creates it the first time a release needs - it. -- The release key has a passphrase and is never added to ssh-agent: agents run on the maintainer's - Mac and the key skips every check, so only the maintainer runs a push with it. -- The push command is in `.claude/skills/release/SKILL.md`, Commit, tag, push. -- The `v*` tag ruleset lets only admins create, move or delete a release tag. - -## When a release run fails - -- A published release is immutable: GitHub refuses to add, replace or delete an asset, or to move - the tag. -- Before **Publish the release**, only a draft exists. Fix forward on `main` through a pull request, - then move the tag to the commit it lands as. The next run reuses the draft. - - ```bash - git fetch origin main - git tag -f -a v -m "v" origin/main - git push origin v --force - ``` - -- After it (a failed `appcast.xml` push, say), use **Re-run failed jobs**: the release job finds the - same files already published and carries on. **Re-run all jobs** rebuilds, and the release job - refuses the new files. -- A published build that is wrong ships as a new version. - -## Staged rollout and critical updates - -- An ordinary release carries - `21600`. Sparkle splits installs - into seven cohorts by a per-install random id, so the last cohort is offered the build 36 hours - after the first. **Check for Updates…** is never phased. -- `"criticalUpdate": true` in `.github/release-flags.json` passes `--critical-update-version ""` to - `generate_appcast` instead. That writes `` with no version, critical for - every host: Sparkle skips phasing, hides Skip and Remind Me Later, retitles the alert, and shows - it even under automatic downloads. -- The release job fails when the flag is `true` and the file has not changed since the previous - tag, which is what a leftover flag looks like. -- When to set it: `.claude/skills/release/SKILL.md`, Critical update flag. +After the run publishes, forward-port the release commit, then write the docs changelog: -## Withdrawing a release +```bash +git fetch origin +git switch -c forward/vX.Y.Z origin/main +git cherry-pick +git push -u origin forward/vX.Y.Z +gh pr create --base main --title "chore(release): vX.Y.Z" --body "Forward-port of #." +python3 scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro-docs/changelog.mdx +``` + +- A cherry-pick conflict on a fragment reworded on `main`: delete the fragment; on `CHANGELOG.md`: keep both sections, in version order. The docs PR is `docs(changelog): vX.Y.Z` in TableProApp/docs, opened with the maintainer's own `gh` (a PR opened with `GITHUB_TOKEN` starts no workflows) and merged once the release is published. + +## What a tag runs + +A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. + +- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it. Then it needs macOS Tests, iOS and Package Tests and Release Build green on the tag commit or the nearest first-parent commit that ran them, waiting on a pending run. Last, it writes the notes. +- `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal, signature and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. +- `plugins` (`signing`): the Dameng and HANA bridges, `build.sh plugins` (each target `registry.py targets` prints, universal, signed with Developer ID, the hardened runtime and a timestamp), one notary submission for all, `package.sh plugins` (the zips, described as the manifest will read them). +- `github-release`: `publish.py draft` puts every asset on a draft and checks each digest, `attest-build-provenance` attests them, `publish.py release` publishes. +- `feed` (`release`): `feed.py add` downloads the archive and the zips from the release, checks their digests, runs `generate_appcast` and `registry.py add`, runs `feed.py check`, and pushes `feeds`, retrying from a fresh fetch up to 3 times. +- `pages` deploys `feeds`. `notify`, stable only and allowed to fail, posts the lead to Telegram. +- Assets: `TablePro-.dmg`, `TablePro-.tar.xz` (the Sparkle archive), `TablePro--dSYMs.zip`, `TablePro--Plugins-dSYMs.zip` and one `-.zip` per registry plugin. + +## When a run fails + +- Tags never move. Before `github-release` publishes, an infrastructure failure is a re-run of the failed jobs; a code failure is fixed on `main`, picked to the line, and ships as the next number. +- After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, or a feed item or manifest release with the same build and sha256. +- A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). +- No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. -Removing the version's items from `appcast.xml` is the only way to un-ship a Sparkle release. +## Withdrawing a release ```bash -python3 scripts/ci/pull-release.py --dry-run # reports what it would remove -GIT_SSH_COMMAND='ssh -i ~/.ssh/tablepro_release -o IdentitiesOnly=yes' \ - python3 scripts/ci/pull-release.py # edits, commits and pushes to main +gh workflow run withdraw.yml --ref v -f version=X.Y.Z ``` -- The second command pushes with the release key, so the maintainer runs it (Keys and rulesets). -- It refuses to push from any branch but `main`, or when local `main` is not `origin/main`, since - the key would carry any other commit past every check. It also refuses to empty the feed. -- The GitHub release stays up, so a direct download link still resolves. Users already on the bad - build are reached by shipping the fix. +- It removes the version's feed items and plugin releases, re-signs the feed, marks the highest stable left as Latest, and redeploys. It refuses to leave the feed with no stable item. The `--ref` is a tag because the `release` environment allows only `v*` tags. +- The GitHub release stays. The fix ships as the next number, critical when the bad build loses data. + +## Keys and environments + +| Environment | Refs | Secrets | +|---|---|---| +| `signing` | tags `v*` | `CERTIFICATES_P12`, `CERTIFICATES_PASSWORD`, `PROVISIONING_PROFILE`, `ASC_KEY_ID`, `ASC_ISSUER_ID`, `ASC_KEY_P8`, `ANALYTICS_HMAC_SECRET`, `CLOUDKIT_MANAGEMENT_TOKEN` | +| `release` | tags `v*` | `SPARKLE_PRIVATE_KEY`, `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_TOPIC_ID` | +| `github-pages` | tags `v*` | none | + +- No repo-level secret or deploy key is left, and no environment has required reviewers: admin-only, immutable tags are the gate. Rulesets: `.claude/rules/ci.md`, Rulesets. +- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` stores as the `TablePro` notarytool profile. The action warns when the certificate or the provisioning profile expires within 60 days. +- An offline backup of the Sparkle EdDSA key exists and the maintainer holds it. The key never rotates without a signed transition, since every app from 0.82 verifies the feed with it. +- `feeds` takes pushes from `github-actions[bot]` with `GITHUB_TOKEN` and has no required checks; the signed feed proves its content. It was seeded once with the legacy `appcast.xml` and an empty v3 manifest. ## Plugins -- `.github/workflows/build-plugin.yml` runs on a `plugin-*-v*` tag, or on a dispatch with `tags` - (`tag[:pluginKitVersion],...`) and an optional `baseRef`. Runs are grouped per ref and per matrix - tag and never cancelled. -- `Resolve Plugin Tags` (Ubuntu) picks the runner: `macos-26` when the ref being built pins Xcode 26 - in its own `.github/actions/setup-xcode`, else `xcode-27`. -- It runs `.github/scripts/test_update_registry.py` and `scripts/ci/check-plugin-manifest.py`, and - validates each tag against `^plugin-[a-z0-9-]+-v[0-9]+\.[0-9]+\.[0-9]+$` and each kit version as - an integer. A tag given without a kit version takes `currentPluginKitVersion`. -- The build job checks out `baseRef` when one is given. It builds the Dameng bridge for - `plugin-dameng-*`, a universal HANA helper plus `govulncheck` for `plugin-hana-*`, then runs - `scripts/build-plugin.sh "$TARGET" both "$VERSION"` with notarization. -- Before publishing: `scripts/ci/verify-plugin-loads.sh` dlopens the arm64 bundle beside its - PluginKit framework; both zips must be stapled and accepted by Gatekeeper; the built - `TableProPluginKitVersion` must equal the release label. -- It publishes the release from a draft, then records the binary in the TableProApp/plugins registry - with `.github/scripts/update-registry.py`. The registry decides which binary a user gets: - [Plugin Registry](https://docs.tablepro.app/development/plugin-registry) covers the manifest, the - tag names and the bulk re-release after a PluginKit ABI change. +- One tag builds the app and every registry plugin, and the plugin zips are assets of the app's release. Each plugin's version and build are the app's. There are no plugin tags. +- `.github/plugin-registry.json` holds `retainFromPluginKit` and the publish set, keyed by target, with each plugin's `name` and `iconName`. Bundled plugins are not in it. +- The v3 manifest lists each plugin's releases: version, build, channel, PluginKit version, `minimumSystemVersion`, URL and sha256. The bundle id and `databaseTypeIds` come from the built bundle. +- Selection: a release in the running build's channel (a `-beta.N` or local build takes beta and stable, any other stable only), with its PluginKit version in the app's range and its `minimumSystemVersion` at or below the running macOS. The highest build wins. Installed registry plugins update after the app does. +- Retention per channel, kit and `minimumSystemVersion`: the two highest builds stay; a beta goes once a stable with the same key has a higher build; a kit below `retainFromPluginKit` goes. A release pruned on arrival is a logged no-op. +- `retainFromPluginKit` is the kit of the oldest app version still on at least 1% of the machines seen in the last 30 days (license dashboard, App Versions). Check it before each stable and raise it with a normal PR. The manifest keeps the higher of its own value and the tag's, so an older line never lowers it. +- Accepted risks: the manifest is trusted through HTTPS, the plugin's Team ID check and the sha256, so a rollback or freeze of the manifest goes undetected. +- PluginKit ABI rules: `.claude/rules/plugin-system.md`, ABI. + +## Static libraries + +The `.a` files under `Libs/` and the xcframeworks under `Libs/ios/` live on the immutable `libs-` GitHub release that `Libs/release` names, not in git; `scripts/download-libs.sh` fetches it and verifies every file against the checksums at `HEAD`. `scripts/publish-libs.sh` takes every library you rebuilt, added or removed, verifies the others against `HEAD`, creates the next `libs-` and rewrites `Libs/release` and both checksum files (`--dry-run` only prints). A published `libs-` never changes: a fix is the next ``. + +```bash +scripts/publish-libs.sh libmongoc_arm64.a libmongoc_x86_64.a libmongoc_universal.a libmongoc.a +git add Libs/release Libs/checksums.sha256 Libs/ios/checksums.sha256 && git commit -m "build(libs): publish libs-" +``` + +## Reproducing a Release build + +```bash +scripts/download-libs.sh && scripts/generate-project.sh macos && brew install create-dmg +scripts/release/build.sh app --unsigned && scripts/release/package.sh app 0.0.0-dev --unsigned +scripts/build-dameng.sh both && scripts/build-hana.sh both +scripts/release/build.sh plugins --unsigned && scripts/release/package.sh plugins 0.0.0-dev --unsigned +``` + +- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. Release Build (`release-build.yml`) runs the same on every push to `main` and `release/**`. Signed: drop `--unsigned`, set `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs the profile `xcrun notarytool store-credentials TablePro --key --key-id --issuer ` creates. diff --git a/TablePro/Resources/WhatsNew.md b/TablePro/Resources/WhatsNew.md deleted file mode 100644 index 020b26e0f8..0000000000 --- a/TablePro/Resources/WhatsNew.md +++ /dev/null @@ -1,9 +0,0 @@ -# TablePro 0.81.0 - -Cloud Firestore, Firebase Realtime Database and Google Cloud Bigtable connect through new registry plugins. - -Each database in the connections strip keeps its own tabs, and Oracle query tabs get a schema picker. - -Themes color every content view and the window background. Themes made for earlier versions need to be recreated. - -Plugins signed by other developers stop loading, and HTML and XML export are removed. diff --git a/TablePro/TablePro.entitlements b/TablePro/TablePro.entitlements index 1f880e34d9..e094e5eb3d 100644 --- a/TablePro/TablePro.entitlements +++ b/TablePro/TablePro.entitlements @@ -2,8 +2,6 @@ - com.apple.application-identifier - D7HJ5TFYCU.com.TablePro com.apple.developer.icloud-container-identifiers iCloud.com.TablePro @@ -14,15 +12,13 @@ com.apple.developer.icloud-container-environment Production - com.apple.developer.team-identifier - D7HJ5TFYCU com.apple.security.app-sandbox com.apple.security.cs.disable-library-validation keychain-access-groups - D7HJ5TFYCU.com.TablePro.shared + $(AppIdentifierPrefix)com.TablePro.shared diff --git a/TablePro/Views/Welcome/WelcomeActionsPanel.swift b/TablePro/Views/Welcome/WelcomeActionsPanel.swift index d58cb53b06..98b8ce8f84 100644 --- a/TablePro/Views/Welcome/WelcomeActionsPanel.swift +++ b/TablePro/Views/Welcome/WelcomeActionsPanel.swift @@ -21,7 +21,7 @@ struct WelcomeActionsPanel: View { .removeDuplicates() /// Captured once: the same appearance that reads the stored value overwrites it. - @State private var lastSeenAppVersion = "" + @State private var showsWhatsNew: Bool? var body: some View { VStack(spacing: 0) { @@ -54,7 +54,7 @@ struct WelcomeActionsPanel: View { .accessibilityIdentifier("welcome-check-for-updates-link") } - if showsWhatsNew { + if showsWhatsNew == true { Button(String(format: String(localized: "What's New in %@"), Bundle.main.appVersion)) { NSApp.sendAction(#selector(AppDelegate.openChangelog(_:)), to: nil, from: nil) } @@ -115,16 +115,8 @@ struct WelcomeActionsPanel: View { } private func recordAppVersion() { - let defaults = AppStorageEnvironment.shared.defaults - let key = PreferenceKeys.lastSeenAppVersion.name - let current = Bundle.main.appVersion - guard lastSeenAppVersion.isEmpty else { return } - lastSeenAppVersion = defaults.string(forKey: key) ?? current - defaults.set(current, forKey: key) - } - - private var showsWhatsNew: Bool { - !lastSeenAppVersion.isEmpty && lastSeenAppVersion != Bundle.main.appVersion + guard showsWhatsNew == nil else { return } + showsWhatsNew = WhatsNewNotice(bundle: .main, defaults: AppStorageEnvironment.shared.defaults).recordLaunch() } /// The badge follows entitlement and the support link follows whether anything has been paid, diff --git a/TablePro/Views/WhatsNew/WhatsNewNotice.swift b/TablePro/Views/WhatsNew/WhatsNewNotice.swift new file mode 100644 index 0000000000..015b08ea77 --- /dev/null +++ b/TablePro/Views/WhatsNew/WhatsNewNotice.swift @@ -0,0 +1,16 @@ +import Foundation + +internal struct WhatsNewNotice { + let bundle: Bundle + let defaults: UserDefaults + + func recordLaunch() -> Bool { + // A build without notes records nothing, so a Debug run never marks a release as seen. + guard WhatsNewContent.load(from: bundle) != nil else { return false } + let key = PreferenceKeys.lastSeenAppVersion.name + let current = bundle.appVersion + let previous = defaults.string(forKey: key) + defaults.set(current, forKey: key) + return previous.map { $0 != current } ?? false + } +} diff --git a/TablePro/Views/WhatsNew/WhatsNewView.swift b/TablePro/Views/WhatsNew/WhatsNewView.swift index 96446b2e3e..59cb699134 100644 --- a/TablePro/Views/WhatsNew/WhatsNewView.swift +++ b/TablePro/Views/WhatsNew/WhatsNewView.swift @@ -4,7 +4,7 @@ struct WhatsNewView: View { private let content: WhatsNewContent private let onViewFullChangelog: () -> Void - init(content: WhatsNewContent = .bundled, onViewFullChangelog: @escaping () -> Void) { + init(content: WhatsNewContent = .load() ?? .changelogPointer, onViewFullChangelog: @escaping () -> Void) { self.content = content self.onViewFullChangelog = onViewFullChangelog } @@ -63,13 +63,17 @@ struct WhatsNewContent { let title: String let paragraphs: [AttributedString] - static var bundled: WhatsNewContent { - guard let url = Bundle.main.url(forResource: "WhatsNew", withExtension: "md"), + static var changelogPointer: WhatsNewContent { + WhatsNewContent( + title: String(localized: "What's New"), + paragraphs: [AttributedString(String(localized: "Release notes are on the changelog."))] + ) + } + + static func load(from bundle: Bundle = .main) -> WhatsNewContent? { + guard let url = bundle.url(forResource: "WhatsNew", withExtension: "md"), let markdown = try? String(contentsOf: url, encoding: .utf8) else { - return WhatsNewContent( - title: String(localized: "What's New"), - paragraphs: [AttributedString(String(localized: "Release notes are on the changelog."))] - ) + return nil } return parse(markdown) } @@ -98,7 +102,7 @@ struct WhatsNewContent { } if body.isEmpty { - body = [AttributedString(String(localized: "Release notes are on the changelog."))] + body = changelogPointer.paragraphs } return WhatsNewContent(title: title, paragraphs: body) } diff --git a/TableProTests/Views/WhatsNewContentTests.swift b/TableProTests/Views/WhatsNewContentTests.swift index 7b6c7c1885..28d152fab4 100644 --- a/TableProTests/Views/WhatsNewContentTests.swift +++ b/TableProTests/Views/WhatsNewContentTests.swift @@ -54,13 +54,75 @@ struct WhatsNewContentTests { #expect(content.paragraphs.count == 1) } - /// The file is generated by scripts/generate-whats-new.sh and shipped in the bundle, so a - /// build where it is missing or unreadable must still open a usable window. - @Test("The bundled file parses into something displayable") - func bundledFileParses() { - let content = WhatsNewContent.bundled + @Test("A bundle with notes loads them") + func bundleWithNotesLoads() throws { + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n\nBeta updates.\n") { bundle in + let content = try #require(WhatsNewContent.load(from: bundle)) - #expect(!content.title.isEmpty) - #expect(!content.paragraphs.isEmpty) + #expect(content.title == "TablePro 0.82.0") + #expect(content.paragraphs.map { String($0.characters) } == ["Beta updates."]) + } + } + + @Test("A bundle without notes loads nothing") + func bundleWithoutNotesLoadsNothing() throws { + try withBundle(version: "0.0.0-dev", notes: nil) { bundle in + #expect(WhatsNewContent.load(from: bundle) == nil) + } + } + + @Test("Without notes the Welcome link stays hidden and the seen version is not written") + func noNotesNoLinkNoWrite() throws { + let suite = try EphemeralDefaults() + suite.defaults.set("0.81.0", forKey: PreferenceKeys.lastSeenAppVersion.name) + + try withBundle(version: "0.0.0-dev", notes: nil) { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(!showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.81.0") + } + } + + @Test("With notes a new version shows the Welcome link and is recorded as seen") + func notesShowLinkForNewVersion() throws { + let suite = try EphemeralDefaults() + suite.defaults.set("0.81.0", forKey: PreferenceKeys.lastSeenAppVersion.name) + + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n") { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.82.0") + } + } + + @Test("A first launch records the version without the Welcome link") + func firstLaunchHasNoLink() throws { + let suite = try EphemeralDefaults() + + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n") { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(!showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.82.0") + } + } + + private func withBundle(version: String, notes: String?, _ body: (Bundle) throws -> Void) throws { + let root = FileManager.default.temporaryDirectory + .appendingPathComponent("WhatsNewContentTests-\(UUID().uuidString)", isDirectory: true) + defer { try? FileManager.default.removeItem(at: root) } + let bundleURL = root.appendingPathComponent("Probe.bundle", isDirectory: true) + let resources = bundleURL.appendingPathComponent("Contents/Resources", isDirectory: true) + try FileManager.default.createDirectory(at: resources, withIntermediateDirectories: true) + let info = ["CFBundleIdentifier": "com.TablePro.WhatsNewProbe", "CFBundleShortVersionString": version] + let plist = try PropertyListSerialization.data(fromPropertyList: info, format: .xml, options: 0) + try plist.write(to: bundleURL.appendingPathComponent("Contents/Info.plist")) + if let notes { + try Data(notes.utf8).write(to: resources.appendingPathComponent("WhatsNew.md")) + } + let bundle = try #require(Bundle(url: bundleURL)) + try body(bundle) } } diff --git a/changelog.d/.gitkeep b/changelog.d/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/changelog.d/3445.fixed.2.md b/changelog.d/3445.fixed.2.md new file mode 100644 index 0000000000..0077ab4ca1 --- /dev/null +++ b/changelog.d/3445.fixed.2.md @@ -0,0 +1 @@ +ClickHouse users with a read-only profile unable to connect. diff --git a/changelog.d/3445.fixed.3.md b/changelog.d/3445.fixed.3.md new file mode 100644 index 0000000000..3222574f87 --- /dev/null +++ b/changelog.d/3445.fixed.3.md @@ -0,0 +1 @@ +ClickHouse indexes, view definitions, functions, table comments and sizes failing to load on servers before 22.3. diff --git a/changelog.d/3445.fixed.4.md b/changelog.d/3445.fixed.4.md new file mode 100644 index 0000000000..924548688a --- /dev/null +++ b/changelog.d/3445.fixed.4.md @@ -0,0 +1 @@ +A ClickHouse query that failed partway showing its partial rows and the error text as a row instead of the error. diff --git a/changelog.d/3445.fixed.5.md b/changelog.d/3445.fixed.5.md new file mode 100644 index 0000000000..3714983d0b --- /dev/null +++ b/changelog.d/3445.fixed.5.md @@ -0,0 +1 @@ +A ClickHouse query with a `FORMAT` clause showing garbled rows in the grid and in an export. diff --git a/changelog.d/3445.fixed.6.md b/changelog.d/3445.fixed.6.md new file mode 100644 index 0000000000..2abc737804 --- /dev/null +++ b/changelog.d/3445.fixed.6.md @@ -0,0 +1 @@ +ClickHouse SQL export writing the current database's table structure for a table in another database. diff --git a/changelog.d/3445.fixed.7.md b/changelog.d/3445.fixed.7.md new file mode 100644 index 0000000000..91f6cccc3c --- /dev/null +++ b/changelog.d/3445.fixed.7.md @@ -0,0 +1 @@ +Opening a ClickHouse executable function's definition showing a syntax error. diff --git a/changelog.d/3445.fixed.8.md b/changelog.d/3445.fixed.8.md new file mode 100644 index 0000000000..4305b675b5 --- /dev/null +++ b/changelog.d/3445.fixed.8.md @@ -0,0 +1 @@ +SQL Server 2016 RTM offered a `CREATE OR ALTER VIEW` template it cannot run, and Azure SQL a drop-and-create one. diff --git a/changelog.d/3445.fixed.md b/changelog.d/3445.fixed.md new file mode 100644 index 0000000000..372c28e82a --- /dev/null +++ b/changelog.d/3445.fixed.md @@ -0,0 +1 @@ +Every ClickHouse query after connecting failing with `UNKNOWN_SETTING` on 23.8.0 to 23.8.8. diff --git a/changelog.d/3446.added.2.md b/changelog.d/3446.added.2.md new file mode 100644 index 0000000000..f256004406 --- /dev/null +++ b/changelog.d/3446.added.2.md @@ -0,0 +1 @@ +Edit > Set Value in the menu bar, with rows in Settings > Keyboard. diff --git a/changelog.d/3446.added.3.md b/changelog.d/3446.added.3.md new file mode 100644 index 0000000000..efe41eb712 --- /dev/null +++ b/changelog.d/3446.added.3.md @@ -0,0 +1 @@ +New UUID in a UUID primary key the server does not fill, on Add Row, Duplicate Row and Paste. diff --git a/changelog.d/3446.added.md b/changelog.d/3446.added.md new file mode 100644 index 0000000000..56f4369878 --- /dev/null +++ b/changelog.d/3446.added.md @@ -0,0 +1 @@ +Generate UUID and Generate UUIDv7 in Set Value, the row inspector and Fill Column, a new value for each cell. diff --git a/changelog.d/3446.changed.2.md b/changelog.d/3446.changed.2.md new file mode 100644 index 0000000000..1a31b68aa6 --- /dev/null +++ b/changelog.d/3446.changed.2.md @@ -0,0 +1 @@ +SQL functions in Set Value and the row inspector listed per database and only where they fit the column. diff --git a/changelog.d/3446.changed.3.md b/changelog.d/3446.changed.3.md new file mode 100644 index 0000000000..28908dfbc9 --- /dev/null +++ b/changelog.d/3446.changed.3.md @@ -0,0 +1 @@ +UUID functions in SQL Functions replaced by Generate UUID, which writes the value itself. diff --git a/changelog.d/3446.changed.4.md b/changelog.d/3446.changed.4.md new file mode 100644 index 0000000000..202fd11346 --- /dev/null +++ b/changelog.d/3446.changed.4.md @@ -0,0 +1 @@ +Text typed into a column that is not a date or time saved as typed, even when it reads like `now()`. diff --git a/changelog.d/3446.changed.5.md b/changelog.d/3446.changed.5.md new file mode 100644 index 0000000000..0fd52ff651 --- /dev/null +++ b/changelog.d/3446.changed.5.md @@ -0,0 +1 @@ +Set NULL and Set DEFAULT in the row inspector shown only where the column takes them, as in the data grid. diff --git a/changelog.d/3446.changed.6.md b/changelog.d/3446.changed.6.md new file mode 100644 index 0000000000..f6b8d566fe --- /dev/null +++ b/changelog.d/3446.changed.6.md @@ -0,0 +1 @@ +Default only on new rows on SQLite, libSQL, Turso, D1, ClickHouse, Databend, Snowflake, Trino, Cassandra. diff --git a/changelog.d/3446.changed.7.md b/changelog.d/3446.changed.7.md new file mode 100644 index 0000000000..0b12cf249e --- /dev/null +++ b/changelog.d/3446.changed.7.md @@ -0,0 +1 @@ +Pending SQL functions in the data grid drawn like a pending default. diff --git a/changelog.d/3446.changed.8.md b/changelog.d/3446.changed.8.md new file mode 100644 index 0000000000..102874b070 --- /dev/null +++ b/changelog.d/3446.changed.8.md @@ -0,0 +1 @@ +PluginKit ABI 39, additive: existing plugins keep loading. diff --git a/changelog.d/3446.changed.md b/changelog.d/3446.changed.md new file mode 100644 index 0000000000..b2eee69177 --- /dev/null +++ b/changelog.d/3446.changed.md @@ -0,0 +1 @@ +Set Value applies to every selected cell, or to the clicked column across the selected rows. diff --git a/changelog.d/3446.fixed.10.md b/changelog.d/3446.fixed.10.md new file mode 100644 index 0000000000..3de1acf839 --- /dev/null +++ b/changelog.d/3446.fixed.10.md @@ -0,0 +1 @@ +MariaDB `uuid` columns read as `CHAR`. diff --git a/changelog.d/3446.fixed.11.md b/changelog.d/3446.fixed.11.md new file mode 100644 index 0000000000..0dbd8f5afc --- /dev/null +++ b/changelog.d/3446.fixed.11.md @@ -0,0 +1 @@ +Empty field in the multi-row inspector after the selected rows were given different values. diff --git a/changelog.d/3446.fixed.2.md b/changelog.d/3446.fixed.2.md new file mode 100644 index 0000000000..1a5946c3b7 --- /dev/null +++ b/changelog.d/3446.fixed.2.md @@ -0,0 +1 @@ +SQL functions saved as text on SQL Server, Teradata, Spanner and BigQuery. diff --git a/changelog.d/3446.fixed.3.md b/changelog.d/3446.fixed.3.md new file mode 100644 index 0000000000..854eb5441e --- /dev/null +++ b/changelog.d/3446.fixed.3.md @@ -0,0 +1 @@ +`NOW()` offered on SQLite, Oracle, Cassandra and MongoDB, where it fails or saves the text. diff --git a/changelog.d/3446.fixed.4.md b/changelog.d/3446.fixed.4.md new file mode 100644 index 0000000000..d6bd75444a --- /dev/null +++ b/changelog.d/3446.fixed.4.md @@ -0,0 +1 @@ +SQLite's `datetime('now')` offered on DuckDB and `gen_random_uuid()` on Redshift. diff --git a/changelog.d/3446.fixed.5.md b/changelog.d/3446.fixed.5.md new file mode 100644 index 0000000000..64f5698583 --- /dev/null +++ b/changelog.d/3446.fixed.5.md @@ -0,0 +1 @@ +Stored text such as `CURRENT_DATE` turning into today's date on Duplicate Row or Restore Previous Values. diff --git a/changelog.d/3446.fixed.6.md b/changelog.d/3446.fixed.6.md new file mode 100644 index 0000000000..436f692a21 --- /dev/null +++ b/changelog.d/3446.fixed.6.md @@ -0,0 +1 @@ +Copy as INSERT and Copy as UPDATE quoting a staged SQL function. diff --git a/changelog.d/3446.fixed.7.md b/changelog.d/3446.fixed.7.md new file mode 100644 index 0000000000..052ec295c9 --- /dev/null +++ b/changelog.d/3446.fixed.7.md @@ -0,0 +1 @@ +Default on a saved row failing on SQLite, and storing `__DEFAULT__` on Teradata and DynamoDB. diff --git a/changelog.d/3446.fixed.8.md b/changelog.d/3446.fixed.8.md new file mode 100644 index 0000000000..6544d509f7 --- /dev/null +++ b/changelog.d/3446.fixed.8.md @@ -0,0 +1 @@ +Duplicate Row and Paste failing to save on a UUID primary key with no default. diff --git a/changelog.d/3446.fixed.9.md b/changelog.d/3446.fixed.9.md new file mode 100644 index 0000000000..13991df837 --- /dev/null +++ b/changelog.d/3446.fixed.9.md @@ -0,0 +1 @@ +Date functions and a date picker offered on Cassandra and ScyllaDB `timeuuid` columns. diff --git a/changelog.d/3446.fixed.md b/changelog.d/3446.fixed.md new file mode 100644 index 0000000000..b06bc3718e --- /dev/null +++ b/changelog.d/3446.fixed.md @@ -0,0 +1 @@ +`gen_random_uuid()`, `UUID()`, `NEWID()` and `datetime('now')` from SQL Functions saved as quoted text. diff --git a/changelog.d/3452.added.2.md b/changelog.d/3452.added.2.md new file mode 100644 index 0000000000..ff44c9880c --- /dev/null +++ b/changelog.d/3452.added.2.md @@ -0,0 +1 @@ +VS Code color theme import in Settings > Appearance, from the theme's `.json` file. diff --git a/changelog.d/3452.added.3.md b/changelog.d/3452.added.3.md new file mode 100644 index 0000000000..038da3f4e6 --- /dev/null +++ b/changelog.d/3452.added.3.md @@ -0,0 +1 @@ +Theme Editor window with a live preview, undo and Revert Changes. Open windows show each edit as you make it. diff --git a/changelog.d/3452.added.4.md b/changelog.d/3452.added.4.md new file mode 100644 index 0000000000..0ed6586c14 --- /dev/null +++ b/changelog.d/3452.added.4.md @@ -0,0 +1 @@ +iCloud sync of custom themes and the chosen light and dark themes, with Settings on in Settings > Sync. diff --git a/changelog.d/3452.added.5.md b/changelog.d/3452.added.5.md new file mode 100644 index 0000000000..f5bdb6564a --- /dev/null +++ b/changelog.d/3452.added.5.md @@ -0,0 +1 @@ +Theme import by dropping files on the Themes list, and export by dragging a theme to Finder. diff --git a/changelog.d/3452.added.md b/changelog.d/3452.added.md new file mode 100644 index 0000000000..f12d4a3c66 --- /dev/null +++ b/changelog.d/3452.added.md @@ -0,0 +1 @@ +Solarized, GitHub, Catppuccin, Gruvbox and Tokyo Night themes in light and dark, plus One Dark and One Light. diff --git a/changelog.d/3452.changed.2.md b/changelog.d/3452.changed.2.md new file mode 100644 index 0000000000..b7f4be3de7 --- /dev/null +++ b/changelog.d/3452.changed.2.md @@ -0,0 +1 @@ +Data Files, Compare & Sync and Open in Window value windows show the theme in the titlebar on macOS 26 and later. diff --git a/changelog.d/3452.changed.3.md b/changelog.d/3452.changed.3.md new file mode 100644 index 0000000000..0d293ea97d --- /dev/null +++ b/changelog.d/3452.changed.3.md @@ -0,0 +1 @@ +SQL aliases and parameters drawn in the editor text color instead of the NULL color. diff --git a/changelog.d/3452.changed.4.md b/changelog.d/3452.changed.4.md new file mode 100644 index 0000000000..2cf54f5c68 --- /dev/null +++ b/changelog.d/3452.changed.4.md @@ -0,0 +1 @@ +SQL previews in history, filters, maintenance and review sheets highlighted as in the editor, in the editor font. diff --git a/changelog.d/3452.changed.5.md b/changelog.d/3452.changed.5.md new file mode 100644 index 0000000000..a7be1d7ba9 --- /dev/null +++ b/changelog.d/3452.changed.5.md @@ -0,0 +1 @@ +Server Dashboard sessions and slow queries shown as data grids, with Cancel Query and Terminate Session in the header. diff --git a/changelog.d/3452.changed.6.md b/changelog.d/3452.changed.6.md new file mode 100644 index 0000000000..4e0ec6bb27 --- /dev/null +++ b/changelog.d/3452.changed.6.md @@ -0,0 +1 @@ +Terminate Session and Cancel Query confirmations name the session's process ID and user. diff --git a/changelog.d/3452.changed.7.md b/changelog.d/3452.changed.7.md new file mode 100644 index 0000000000..98ca5f3b81 --- /dev/null +++ b/changelog.d/3452.changed.7.md @@ -0,0 +1 @@ +Compare & Sync status bar shows the target connection's icon instead of a colored dot. diff --git a/changelog.d/3452.changed.8.md b/changelog.d/3452.changed.8.md new file mode 100644 index 0000000000..ab71034434 --- /dev/null +++ b/changelog.d/3452.changed.8.md @@ -0,0 +1 @@ +Themes move to the Trash instead of being deleted. Undo reverts a trash, rename, duplicate or import. diff --git a/changelog.d/3452.changed.9.md b/changelog.d/3452.changed.9.md new file mode 100644 index 0000000000..e12e3a46bc --- /dev/null +++ b/changelog.d/3452.changed.9.md @@ -0,0 +1 @@ +Editor tab strip over a themed window on macOS 26 and later drawn in a system fill instead of the theme color. diff --git a/changelog.d/3452.changed.md b/changelog.d/3452.changed.md new file mode 100644 index 0000000000..7a6e0beac9 --- /dev/null +++ b/changelog.d/3452.changed.md @@ -0,0 +1 @@ +Appearance (System, Light or Dark) kept per Mac instead of syncing. Older versions that sync settings switch to System. diff --git a/changelog.d/3452.fixed.10.md b/changelog.d/3452.fixed.10.md new file mode 100644 index 0000000000..fe65f4303d --- /dev/null +++ b/changelog.d/3452.fixed.10.md @@ -0,0 +1 @@ +System header band on the data grid and the Query Plan and Users & Roles trees under a theme with header colors. diff --git a/changelog.d/3452.fixed.11.md b/changelog.d/3452.fixed.11.md new file mode 100644 index 0000000000..8c7b248691 --- /dev/null +++ b/changelog.d/3452.fixed.11.md @@ -0,0 +1 @@ +System-colored alternate rows, header and booleans in a theme that sets only the grid background and text. diff --git a/changelog.d/3452.fixed.12.md b/changelog.d/3452.fixed.12.md new file mode 100644 index 0000000000..531ec9d8ba --- /dev/null +++ b/changelog.d/3452.fixed.12.md @@ -0,0 +1 @@ +Selected row text in white over a theme's selection color, or in the theme's color over the system selection. diff --git a/changelog.d/3452.fixed.13.md b/changelog.d/3452.fixed.13.md new file mode 100644 index 0000000000..fe68e81560 --- /dev/null +++ b/changelog.d/3452.fixed.13.md @@ -0,0 +1 @@ +Translucent theme selection, as in Dracula and Nord, in a different shade on rows, row numbers and column headers. diff --git a/changelog.d/3452.fixed.14.md b/changelog.d/3452.fixed.14.md new file mode 100644 index 0000000000..c206978d37 --- /dev/null +++ b/changelog.d/3452.fixed.14.md @@ -0,0 +1 @@ +No outline around a selected cell range in Dracula and Nord. diff --git a/changelog.d/3452.fixed.15.md b/changelog.d/3452.fixed.15.md new file mode 100644 index 0000000000..11d5aafcc6 --- /dev/null +++ b/changelog.d/3452.fixed.15.md @@ -0,0 +1 @@ +Shapes in the result Map view keeping the previous Accent color after it changes. diff --git a/changelog.d/3452.fixed.16.md b/changelog.d/3452.fixed.16.md new file mode 100644 index 0000000000..76e3047808 --- /dev/null +++ b/changelog.d/3452.fixed.16.md @@ -0,0 +1 @@ +Edit as Text in the array editor turning straight quotes and dashes into curly quotes and long dashes. diff --git a/changelog.d/3452.fixed.17.md b/changelog.d/3452.fixed.17.md new file mode 100644 index 0000000000..6afc32a86b --- /dev/null +++ b/changelog.d/3452.fixed.17.md @@ -0,0 +1 @@ +Theme editor bringing back a theme file deleted in Finder on the next change. diff --git a/changelog.d/3452.fixed.18.md b/changelog.d/3452.fixed.18.md new file mode 100644 index 0000000000..3ab2c13fb4 --- /dev/null +++ b/changelog.d/3452.fixed.18.md @@ -0,0 +1 @@ +Changes to a theme file made outside TablePro not shown until Settings > Appearance was opened. diff --git a/changelog.d/3452.fixed.19.md b/changelog.d/3452.fixed.19.md new file mode 100644 index 0000000000..8660028e1b --- /dev/null +++ b/changelog.d/3452.fixed.19.md @@ -0,0 +1 @@ +Theme files with a JSON syntax error reported without the line and column. diff --git a/changelog.d/3452.fixed.2.md b/changelog.d/3452.fixed.2.md new file mode 100644 index 0000000000..ffe0c76863 --- /dev/null +++ b/changelog.d/3452.fixed.2.md @@ -0,0 +1 @@ +No-match, invalid and NULL states in the JSON, PHP and array viewers drawn without the theme's background. diff --git a/changelog.d/3452.fixed.20.md b/changelog.d/3452.fixed.20.md new file mode 100644 index 0000000000..3dd695d1bb --- /dev/null +++ b/changelog.d/3452.fixed.20.md @@ -0,0 +1 @@ +Two themes with the same name impossible to tell apart in the theme pickers. diff --git a/changelog.d/3452.fixed.3.md b/changelog.d/3452.fixed.3.md new file mode 100644 index 0000000000..93eabe2870 --- /dev/null +++ b/changelog.d/3452.fixed.3.md @@ -0,0 +1 @@ +Agent result states and proposed SQL ignoring the theme. diff --git a/changelog.d/3452.fixed.4.md b/changelog.d/3452.fixed.4.md new file mode 100644 index 0000000000..a8589277cd --- /dev/null +++ b/changelog.d/3452.fixed.4.md @@ -0,0 +1 @@ +Connecting and Not connected panes, and Agent mode, in the system window color under a theme before macOS 26. diff --git a/changelog.d/3452.fixed.5.md b/changelog.d/3452.fixed.5.md new file mode 100644 index 0000000000..5ee9687031 --- /dev/null +++ b/changelog.d/3452.fixed.5.md @@ -0,0 +1 @@ +Selected text in SQL previews, such as query history, ignoring the theme's selection color. diff --git a/changelog.d/3452.fixed.6.md b/changelog.d/3452.fixed.6.md new file mode 100644 index 0000000000..14ad866d1a --- /dev/null +++ b/changelog.d/3452.fixed.6.md @@ -0,0 +1 @@ +Server Dashboard lists and the Query Insights chart ignoring the theme. diff --git a/changelog.d/3452.fixed.7.md b/changelog.d/3452.fixed.7.md new file mode 100644 index 0000000000..bfcc3b1736 --- /dev/null +++ b/changelog.d/3452.fixed.7.md @@ -0,0 +1 @@ +Compare & Sync row states and Data Files loading and failure states drawn without the theme's grid background. diff --git a/changelog.d/3452.fixed.8.md b/changelog.d/3452.fixed.8.md new file mode 100644 index 0000000000..6776e7a0a6 --- /dev/null +++ b/changelog.d/3452.fixed.8.md @@ -0,0 +1 @@ +Structure and Create Table code placeholders, Startup SQL in a link and the tunnel command preview ignoring the theme. diff --git a/changelog.d/3452.fixed.9.md b/changelog.d/3452.fixed.9.md new file mode 100644 index 0000000000..d13974a2f1 --- /dev/null +++ b/changelog.d/3452.fixed.9.md @@ -0,0 +1 @@ +Bars in query, history, filter, structure, plan, compare and other panes drawing a system band over a themed window. diff --git a/changelog.d/3452.fixed.md b/changelog.d/3452.fixed.md new file mode 100644 index 0000000000..37d6f54ad5 --- /dev/null +++ b/changelog.d/3452.fixed.md @@ -0,0 +1 @@ +Inline cell editor, the array editor's Edit as Text, and the inspector's hex dump and map pane ignoring the theme. diff --git a/changelog.d/3478.fixed.md b/changelog.d/3478.fixed.md new file mode 100644 index 0000000000..027b32d710 --- /dev/null +++ b/changelog.d/3478.fixed.md @@ -0,0 +1 @@ +No warning on a connection in the Welcome window whose driver plugin failed to load or is out of date. diff --git a/changelog.d/3491.changed.md b/changelog.d/3491.changed.md new file mode 100644 index 0000000000..29076adc78 --- /dev/null +++ b/changelog.d/3491.changed.md @@ -0,0 +1 @@ +Plugin download progress shown as the bytes arrive instead of jumping to full at the end. diff --git a/changelog.d/3497.changed.md b/changelog.d/3497.changed.md new file mode 100644 index 0000000000..825a6c6890 --- /dev/null +++ b/changelog.d/3497.changed.md @@ -0,0 +1 @@ +Welcome window and What's New show the build number after the version. diff --git a/changelog.d/3501.added.md b/changelog.d/3501.added.md new file mode 100644 index 0000000000..78b4745548 --- /dev/null +++ b/changelog.d/3501.added.md @@ -0,0 +1 @@ +Get beta updates in Settings > General, from a signed update feed at updates.tablepro.app. diff --git a/changelog.d/3502.changed.md b/changelog.d/3502.changed.md new file mode 100644 index 0000000000..c867f0c789 --- /dev/null +++ b/changelog.d/3502.changed.md @@ -0,0 +1 @@ +Registry drivers such as Oracle, SQL Server and MongoDB load on first use instead of at launch. diff --git a/project.yml b/project.yml index 4b1562726d..c386eb3bc6 100644 --- a/project.yml +++ b/project.yml @@ -31,6 +31,9 @@ settings: base: CLANG_ANALYZER_LOCALIZABILITY_NONLOCALIZED: YES DEAD_CODE_STRIPPING: YES + # Local placeholders. A release passes the tag's version and build: RELEASING.md, Versions. + MARKETING_VERSION: 0.0.0-dev + CURRENT_PROJECT_VERSION: 1 fileGroups: - Configs @@ -96,7 +99,6 @@ targetTemplates: settings: base: COMBINE_HIDPI_IMAGES: YES - CURRENT_PROJECT_VERSION: 1 DEVELOPMENT_TEAM: $(TABLEPRO_DEVELOPMENT_TEAM) ENABLE_HARDENED_RUNTIME: YES GENERATE_INFOPLIST_FILE: YES @@ -108,7 +110,6 @@ targetTemplates: LD_RUNPATH_SEARCH_PATHS: - $(inherited) - "@executable_path/../Frameworks" - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.${target_name} PRODUCT_NAME: $(TARGET_NAME) SKIP_INSTALL: YES @@ -251,36 +252,29 @@ targetTemplates: - package: TableProCore products: [TableProNumberFormatting, TableProTLSClientIdentity] - # The SAP HANA driver runs go-hdb in tablepro-hana-helper, which scripts/build-hana.sh builds - # and this phase copies into Contents/MacOS, where the plugin looks it up. A missing helper - # stops the build with "Build input file cannot be found" naming its path. The copy is signed - # in a directory from mktemp: the user-script sandbox lets the phase write its declared output - # and nothing beside it, codesign writes a temporary file next to whatever it signs, and it - # reads every parent of that file, which the sandbox refuses inside the build folders. - # scripts/release/build.sh passes OTHER_CODE_SIGN_FLAGS=--timestamp; other builds skip the timestamp server. + # The SAP HANA driver runs go-hdb in tablepro-hana-helper, which scripts/build-hana.sh builds. + # The plugin looks it up in Contents/MacOS, and the copy there signs it with the plugin's identity. HanaHelperEmbedding: - postBuildScripts: - - name: Embed tablepro-hana-helper + sources: + # A file type and optional, so a project generated before scripts/build-hana.sh runs still copies it. + - path: Native/HanaBridge/bin/tablepro-hana-helper + type: file + optional: true + buildPhase: + copyFiles: + destination: executables + attributes: [CodeSignOnCopy] + preBuildScripts: + - name: Check tablepro-hana-helper architectures inputFiles: - $(SRCROOT)/Native/HanaBridge/bin/tablepro-hana-helper - outputFiles: - - $(TARGET_BUILD_DIR)/$(EXECUTABLE_FOLDER_PATH)/tablepro-hana-helper + basedOnDependencyAnalysis: false script: | - set -eu - helper="$SCRIPT_INPUT_FILE_0" for arch in $ARCHS; do - if ! /usr/bin/lipo "$helper" -verify_arch "$arch"; then - echo "error: $helper has no $arch slice. Run scripts/build-hana.sh $arch, or both." >&2 - exit 1 - fi + /usr/bin/lipo "$SCRIPT_INPUT_FILE_0" -verify_arch "$arch" && continue + echo "error: $SCRIPT_INPUT_FILE_0 is missing or has no $arch slice. Run scripts/build-hana.sh $arch, or both." >&2 + exit 1 done - staging="$(mktemp -d)" - trap 'rm -rf "$staging"' EXIT - cp "$helper" "$staging/tablepro-hana-helper" - if [ "${CODE_SIGNING_ALLOWED:-NO}" = "YES" ]; then - /usr/bin/codesign --force --options runtime ${OTHER_CODE_SIGN_FLAGS:---timestamp=none} --sign "${EXPANDED_CODE_SIGN_IDENTITY:--}" "$staging/tablepro-hana-helper" - fi - cp -f "$staging/tablepro-hana-helper" "$SCRIPT_OUTPUT_FILE_0" targets: @@ -317,31 +311,13 @@ targets: # The MySQL plugin's socket key and path rules, so the app writes and checks the socket path # exactly as the driver reads it. - Plugins/MySQLDriverPlugin/MySQLLocalSocket.swift - configFiles: - Debug: Configs/Version.xcconfig - Release: Configs/Version.xcconfig dependencies: - target: TableProPluginKit embed: true codeSign: true removeHeaders: true - # Embedded rather than reached through an absolute rpath into the source tree. - # TablePro.debug.dylib and the MySQL, PostgreSQL and Redis plugins all link - # @rpath/libssl.3.dylib and @rpath/libcrypto.3.dylib, and both dylibs already carry an - # @rpath install name, so @executable_path/../Frameworks resolves them once they are here. - # Without this the app only launched from a checkout that had Libs/dylibs on disk, which is - # what made the sharded test jobs abort with SIGABRT before XCTest could connect (#2334). - # A release build was already self-contained: build-release.sh copies them in. - # - # codeSign matters as much as embed. create-openssl-dylibs.sh leaves them ad-hoc signed, and - # codesign refuses a bundle whose nested code carries a different identity from the app: - # "code object is not signed at all, in subcomponent libcrypto.3.dylib". A plain copy-files - # phase does not re-sign, so this built only until download-libs.sh next regenerated the - # dylibs. CI never saw it, because every CI build passes CODE_SIGNING_ALLOWED=NO. - # - # link: false keeps this to embedding and signing. A framework dependency links by default, - # which would put both into Link Binary With Libraries and change what the app links against; - # the app reaches them through @rpath from TablePro.debug.dylib and the plugins, as before. + # The app and the MySQL, PostgreSQL and Redis plugins load these through @rpath. Re-signed because + # create-openssl-dylibs.sh signs them ad hoc; link: false keeps them off the app's link line. - framework: Libs/dylibs/libssl.3.dylib embed: true codeSign: true @@ -417,6 +393,16 @@ targets: products: [TableProAnalytics, TableProConnectionLibrary, TableProGeometry, TableProGoogleCloud, TableProImport, TableProLogRedaction, TableProNumberFormatting, TableProRedisCore, TableProSQLGrammar, TableProSSHTransport, TableProSyncTransport, TableProTabular, TableProTabularIO, TableProTheme] - package: TableProMacCore products: [TableProAutocomplete, TableProMCP, TableProSSHConfig, TableProUtilities] + # Writes nothing for a version CHANGELOG.md has no section for, so a 0.0.0-dev build ships no notes. + postCompileScripts: + - name: Write WhatsNew.md + inputFiles: + - $(SRCROOT)/CHANGELOG.md + - $(SRCROOT)/scripts/release/notes.py + outputFiles: + - $(TARGET_BUILD_DIR)/$(UNLOCALIZED_RESOURCES_FOLDER_PATH)/WhatsNew.md + script: | + /usr/bin/python3 "$SCRIPT_INPUT_FILE_1" whats-new "$MARKETING_VERSION" --changelog "$SCRIPT_INPUT_FILE_0" --out "$SCRIPT_OUTPUT_FILE_0" settings: base: ASSETCATALOG_COMPILER_APPICON_NAME: AppIcon @@ -466,7 +452,6 @@ targets: Release: CODE_SIGN_ENTITLEMENTS: TablePro/TablePro.entitlements COPY_PHASE_STRIP: YES - DEPLOYMENT_POSTPROCESSING: YES # ─── Plugin ABI ─────────────────────────────────────────────────────────────── @@ -480,7 +465,6 @@ targets: base: # Swift Library Evolution. Removing this breaks every already-built plugin. BUILD_LIBRARY_FOR_DISTRIBUTION: YES - CURRENT_PROJECT_VERSION: 1 DEFINES_MODULE: YES DYLIB_COMPATIBILITY_VERSION: 1 DYLIB_CURRENT_VERSION: 1 @@ -490,7 +474,6 @@ targets: LD_RUNPATH_SEARCH_PATHS: - $(inherited) - "@executable_path/../Frameworks" - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProPluginKit PRODUCT_NAME: $(TARGET_NAME:c99extidentifier) SKIP_INSTALL: YES @@ -564,12 +547,10 @@ targets: products: [TableProMCPTestSupport] settings: base: - CURRENT_PROJECT_VERSION: 1 GENERATE_INFOPLIST_FILE: YES HEADER_SEARCH_PATHS: - $(inherited) - $(SRCROOT)/TablePro/Core/SSH/CLibSSH2/include - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProTests PRODUCT_NAME: $(TARGET_NAME) STRING_CATALOG_GENERATE_SYMBOLS: NO @@ -587,9 +568,7 @@ targets: - target: TablePro settings: base: - CURRENT_PROJECT_VERSION: 1 GENERATE_INFOPLIST_FILE: YES - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProUITests PRODUCT_NAME: $(TARGET_NAME) STRING_CATALOG_GENERATE_SYMBOLS: NO @@ -616,33 +595,18 @@ targets: templateAttributes: folder: SQLiteDriverPlugin principalClass: SQLitePlugin - settings: - base: - # Above plugin-sqlite-v1.0.1, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.2" ClickHouseDriver: templates: [DriverPlugin, ClickHouseLinkage] templateAttributes: folder: ClickHouseDriverPlugin principalClass: ClickHousePlugin - settings: - base: - # Above plugin-clickhouse-v1.0.19, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.20" RedisDriver: templates: [DriverPlugin, RedisLinkage] templateAttributes: folder: RedisDriverPlugin principalClass: RedisPlugin - settings: - base: - # Above plugin-redis-v1.0.19, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.20" CSVExport: templates: [DriverPlugin] @@ -738,10 +702,6 @@ targets: settings: base: PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.MQLExportPlugin - # Above plugin-mql-v1.0.7, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so at the template's 1.0 a retained user copy would keep - # writing server-provided names into exports unescaped. - MARKETING_VERSION: "1.0.8" SQLImport: templates: [DriverPlugin] @@ -751,10 +711,6 @@ targets: settings: base: PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.SQLImportPlugin - # Above plugin-sqlimport-v1.0.5, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so at the template's 1.0 a retained user copy would keep - # offering SQL on MongoDB. - MARKETING_VERSION: "1.0.6" JSONImport: templates: [DriverPlugin] diff --git a/scripts/build-plugin.sh b/scripts/build-plugin.sh deleted file mode 100755 index 9ff0d2d90f..0000000000 --- a/scripts/build-plugin.sh +++ /dev/null @@ -1,278 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/notarize.sh" - -# Build script for creating standalone plugin bundles -# Usage: ./scripts/build-plugin.sh [arm64|x86_64|both] [version] -# Example: ./scripts/build-plugin.sh OracleDriverPlugin arm64 1.0.0 -# -# Version (3rd arg or PLUGIN_VERSION env) is injected as MARKETING_VERSION so -# CFBundleShortVersionString in the built bundle matches the registry version. -# Required for bundled drivers that also ship via registry. Without it, the -# user copy ties with built-in v1.0 and PluginManager prunes it on load. - -PLUGIN_TARGET="${1:?Usage: $0 [arm64|x86_64|both] [version]}" -ARCH="${2:-both}" -PLUGIN_VERSION="${3:-${PLUGIN_VERSION:-}}" -PROJECT="TablePro.xcodeproj" -CONFIG="Release" -BUILD_DIR="build/Plugins" -SIGN_IDENTITY="${SIGN_IDENTITY:-}" -TEAM_ID="${TEAM_ID:-}" -NOTARIZE="${NOTARIZE:-false}" - -if [ -z "$TEAM_ID" ]; then - echo "ERROR: TEAM_ID is not set. Pass via env or set in your shell profile." >&2 - echo " Example: TEAM_ID=ABCDEFGHIJ ./scripts/build-plugin.sh $PLUGIN_TARGET" >&2 - exit 1 -fi - -if [ -z "$SIGN_IDENTITY" ]; then - # Try the canonical "Developer ID Application: ()" pattern. - # If your keychain stores the identity differently, set SIGN_IDENTITY explicitly. - SIGN_IDENTITY=$(security find-identity -v -p codesigning 2>/dev/null \ - | awk -F'"' -v team="$TEAM_ID" '$2 ~ /Developer ID Application/ && $2 ~ team {print $2; exit}') - if [ -z "$SIGN_IDENTITY" ]; then - echo "ERROR: No Developer ID Application identity found in keychain for team $TEAM_ID." >&2 - echo " Either install the cert or set SIGN_IDENTITY explicitly." >&2 - exit 1 - fi -fi - -if [ -n "$PLUGIN_VERSION" ]; then - echo "Building plugin: $PLUGIN_TARGET v$PLUGIN_VERSION for $ARCH" -else - echo "Building plugin: $PLUGIN_TARGET for $ARCH (no version override)" -fi - -is_helper_executable() { - local candidate="$1" - local main_binary="$2" - [ -f "$candidate" ] || return 1 - [ ! -L "$candidate" ] || return 1 - [ "$candidate" != "$main_binary" ] || return 1 - case "$(file -b "$candidate")" in - *Mach-O*) return 0 ;; - *) return 1 ;; - esac -} - -sign_helper_executables() { - local bundle="$1" - local main_binary="$2" - local helper - for helper in "$bundle/Contents/MacOS"/* "$bundle/Contents/Helpers"/*; do - is_helper_executable "$helper" "$main_binary" || continue - echo " Signing helper: $(basename "$helper")" >&2 - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$helper" - done -} - -# codesign --verify --deep --strict also accepts an ad-hoc signed helper, which notarization -# then rejects, so each helper is checked for what notarization needs. -verify_helper_signatures() { - local bundle="$1" - local main_binary="$2" - local helper - local details - for helper in "$bundle/Contents/MacOS"/* "$bundle/Contents/Helpers"/*; do - is_helper_executable "$helper" "$main_binary" || continue - if ! details="$(codesign -dvvv "$helper" 2>&1)"; then - echo "FATAL: $helper is not signed: $details" >&2 - exit 1 - fi - if ! grep -qE '^CodeDirectory .*flags=0x[0-9a-f]+\([^)]*runtime' <<< "$details"; then - echo "FATAL: $helper is not signed with the hardened runtime" >&2 - exit 1 - fi - if ! grep -q '^Authority=Developer ID Application: ' <<< "$details"; then - echo "FATAL: $helper is not signed with a Developer ID Application identity" >&2 - exit 1 - fi - if ! grep -q '^Timestamp=' <<< "$details"; then - echo "FATAL: $helper has no secure timestamp" >&2 - exit 1 - fi - echo " Verified helper: $(basename "$helper") (hardened runtime, Developer ID, timestamp)" >&2 - done -} - -build_plugin() { - local arch=$1 - local build_dir="$BUILD_DIR/$arch" - - echo "Building $PLUGIN_TARGET ($arch)..." >&2 - - # Use -scheme (not -target) with -derivedDataPath to ensure proper - # transitive SPM dependency resolution in explicit module builds. - # project.yml declares one shared scheme per plugin target, named after the target. - DERIVED_DATA_DIR="build/DerivedData" - - local marketing_version_arg="" - if [ -n "$PLUGIN_VERSION" ]; then - marketing_version_arg="MARKETING_VERSION=$PLUGIN_VERSION" - fi - - if ! xcodebuild \ - -project "$PROJECT" \ - -scheme "$PLUGIN_TARGET" \ - -configuration "$CONFIG" \ - -arch "$arch" \ - ONLY_ACTIVE_ARCH=YES \ - CONFIGURATION_BUILD_DIR="$(pwd)/$build_dir" \ - CODE_SIGN_IDENTITY="$SIGN_IDENTITY" \ - CODE_SIGN_STYLE=Manual \ - DEVELOPMENT_TEAM="$TEAM_ID" \ - ${marketing_version_arg:+"$marketing_version_arg"} \ - -skipPackagePluginValidation \ - -derivedDataPath "$DERIVED_DATA_DIR" \ - build > "build-plugin-${arch}.log" 2>&1; then - echo "FATAL: xcodebuild failed for $PLUGIN_TARGET ($arch)" >&2 - echo "=== Swift errors (grep error:) ===" >&2 - grep -nE "error:|cannot|undefined symbol" "build-plugin-${arch}.log" | head -40 >&2 || true - echo "=== Last 80 lines of build log ===" >&2 - tail -80 "build-plugin-${arch}.log" >&2 - exit 1 - fi - - # Find the built plugin bundle by target name - local plugin_bundle="$build_dir/${PLUGIN_TARGET}.tableplugin" - - if [ ! -d "$plugin_bundle" ]; then - echo "FATAL: Plugin bundle not found at $plugin_bundle" >&2 - exit 1 - fi - - echo "Built: $plugin_bundle" >&2 - - if [ -n "$PLUGIN_VERSION" ]; then - actual_version=$(plutil -extract CFBundleShortVersionString raw -o - "$plugin_bundle/Contents/Info.plist" 2>/dev/null || echo "") - if [ "$actual_version" != "$PLUGIN_VERSION" ]; then - echo "FATAL: Built bundle CFBundleShortVersionString='$actual_version' but expected '$PLUGIN_VERSION'" >&2 - echo " MARKETING_VERSION injection failed. Users would see 'Update to v$PLUGIN_VERSION' loops." >&2 - exit 1 - fi - echo "Bundle version verified: CFBundleShortVersionString=$actual_version" >&2 - fi - - # Strip the plugin binary to reduce size - local plugin_name - plugin_name=$(basename "$plugin_bundle" .tableplugin) - local plugin_binary="$plugin_bundle/Contents/MacOS/$plugin_name" - if [ -f "$plugin_binary" ]; then - local before after - before=$(ls -lh "$plugin_binary" | awk '{print $5}') - strip -x "$plugin_binary" - after=$(ls -lh "$plugin_binary" | awk '{print $5}') - echo "Stripped binary: $before -> $after" >&2 - fi - - # Code sign inside-out: nested frameworks/dylibs and helper executables first, then binary, then bundle - echo "Code signing with: $SIGN_IDENTITY" >&2 - - # Sign nested frameworks - if [ -d "$plugin_bundle/Contents/Frameworks" ]; then - find "$plugin_bundle/Contents/Frameworks" -name "*.framework" -o -name "*.dylib" | sort | while read -r nested; do - echo " Signing nested: $(basename "$nested")" >&2 - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$nested" - done - fi - - sign_helper_executables "$plugin_bundle" "$plugin_binary" - - # Sign the main binary - if [ -f "$plugin_binary" ]; then - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_binary" - fi - - # Sign the outer bundle - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_bundle" - - if ! codesign --verify --deep --strict "$plugin_bundle" 2>&1; then - echo "FATAL: Code signature verification failed" >&2 - exit 1 - fi - verify_helper_signatures "$plugin_bundle" "$plugin_binary" - echo "Code signature verified" >&2 - - # Only the path goes to stdout (return value) - echo "$plugin_bundle" -} - -create_zip() { - local plugin_path=$1 - local arch=$2 - local plugin_name - plugin_name=$(basename "$plugin_path" .tableplugin) - local zip_name="${plugin_name}-${arch}.zip" - local zip_path="$BUILD_DIR/$zip_name" - - echo "Creating ZIP: $zip_name" - ditto -c -k --keepParent "$plugin_path" "$zip_path" - - # Print SHA-256 for registry manifest - local sha256 - sha256=$(shasum -a 256 "$zip_path" | awk '{print $1}') - # Write SHA-256 to sidecar file for CI automation - echo "$sha256" > "${zip_path}.sha256" - - echo "ZIP created: $zip_path" - echo " SHA-256: $sha256" - echo " Size: $(ls -lh "$zip_path" | awk '{print $5}')" -} - -# Notarization has to happen BEFORE create_zip, and it has to staple. -# -# Gatekeeper refuses to load an unnotarized bundle into TablePro, and -# com.apple.security.cs.disable-library-validation does not exempt it: a quarantined -# unnotarized plugin fails with "library load disallowed by system policy", and the user -# gets a "could not verify it is free of malware" panel instead of a driver. Every plugin -# published before this ran was unnotarized, because the workflow gated the step on an -# environment variable nothing ever set. -# -# Stapling rewrites the bundle, so the distribution zip and its SHA-256 must both be produced -# after it: the registry manifest pins that checksum and PluginInstaller rejects a mismatch. -notarize_plugin() { - if [ "$NOTARIZE" != "true" ]; then - echo "Skipping notarization (set NOTARIZE=true to enable)" - return - fi - # "open", not "exec": a plugin bundle is opened by the app, not launched. - notarize_and_staple "$1" open -} - -# TablePro.xcodeproj is generated and not in git, so a fresh checkout has none. -# Generation also declares the per-plugin scheme this script builds with. -scripts/generate-project.sh - -# Clean DerivedData for fresh builds; preserve BUILD_DIR across arch invocations -rm -rf build/DerivedData -mkdir -p "$BUILD_DIR" - -case "$ARCH" in - arm64|x86_64) - plugin_path=$(build_plugin "$ARCH") - notarize_plugin "$plugin_path" - create_zip "$plugin_path" "$ARCH" - ;; - both) - arm64_path=$(build_plugin "arm64") - x86_path=$(build_plugin "x86_64") - - notarize_plugin "$arm64_path" - notarize_plugin "$x86_path" - - create_zip "$arm64_path" "arm64" - create_zip "$x86_path" "x86_64" - ;; - *) - echo "Invalid architecture: $ARCH (use arm64, x86_64, or both)" - exit 1 - ;; -esac - -echo "" -echo "Plugin build complete!" -echo "Output: $BUILD_DIR/" -ls -lh "$BUILD_DIR/"*.zip 2>/dev/null || echo "No ZIP files found" diff --git a/scripts/build-release.sh b/scripts/build-release.sh deleted file mode 100755 index 61c2b32294..0000000000 --- a/scripts/build-release.sh +++ /dev/null @@ -1,505 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# shellcheck source=lib/macos.sh -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/macos.sh" -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/notarize.sh" - -# Build script for creating architecture-specific releases -# Usage: ./build-release.sh [arm64|x86_64|both] - -ARCH="${1:-both}" -PROJECT="TablePro.xcodeproj" -SCHEME="TablePro" -CONFIG="Release" -BUILD_DIR="build/Release" -SIGN_IDENTITY="${SIGN_IDENTITY:-Developer ID Application: Dat Ngo Quoc (D7HJ5TFYCU)}" -TEAM_ID="D7HJ5TFYCU" -NOTARIZE="${NOTARIZE:-false}" - -echo "🏗️ Building TablePro for: $ARCH" - -# Bundle non-system dynamic libraries into the app bundle -# so the app runs without Homebrew on end-user machines. -bundle_dylibs() { - local app_path=$1 - local binary="$app_path/Contents/MacOS/TablePro" - local frameworks_dir="$app_path/Contents/Frameworks" - - echo "📦 Bundling dynamic libraries into app bundle..." - mkdir -p "$frameworks_dir" - - # Iteratively discover and copy all non-system dylibs. - # Each pass scans the main binary + already-copied dylibs; - # repeat until no new dylibs are found (handles transitive deps). - local changed=1 - while [ "$changed" -eq 1 ]; do - changed=0 - for target in "$binary" "$frameworks_dir"/*.dylib; do - [ -f "$target" ] || continue - - while IFS= read -r dep; do - # Keep only non-system, non-rewritten absolute paths - case "$dep" in - /usr/lib/*|/System/*|@*|"") continue ;; - esac - - local name - name=$(basename "$dep") - - # Already bundled - [ -f "$frameworks_dir/$name" ] && continue - - if [ -f "$dep" ]; then - echo " Copying $name" - cp "$dep" "$frameworks_dir/$name" - chmod 644 "$frameworks_dir/$name" - changed=1 - else - echo " ⚠️ WARNING: $dep not found on disk, skipping" - fi - done < <(otool -L "$target" 2>/dev/null | awk 'NR>1 {print $1}') - done - done - - # Count bundled dylibs - local count - count=$(find "$frameworks_dir" -name '*.dylib' 2>/dev/null | wc -l | tr -d ' ') - - if [ "$count" -eq 0 ]; then - echo " No non-system dylibs to bundle" - return 0 - fi - - # Rewrite each dylib's own install name - for fw in "$frameworks_dir"/*.dylib; do - [ -f "$fw" ] || continue - local name - name=$(basename "$fw") - install_name_tool -id "@executable_path/../Frameworks/$name" "$fw" - done - - # Rewrite all references in the main binary and every bundled dylib - for target in "$binary" "$frameworks_dir"/*.dylib; do - [ -f "$target" ] || continue - - while IFS= read -r dep; do - case "$dep" in - /usr/lib/*|/System/*|@*|"") continue ;; - esac - - local name - name=$(basename "$dep") - - if [ -f "$frameworks_dir/$name" ]; then - install_name_tool -change "$dep" "@executable_path/../Frameworks/$name" "$target" - fi - done < <(otool -L "$target" 2>/dev/null | awk 'NR>1 {print $1}') - done - - # Verify bundled dylibs are compatible with the deployment target. - # Homebrew builds libraries targeting the *host* macOS version, so if - # the build machine runs macOS 26.0, libpq will require 26.0 symbols - # (e.g. strchrnul) that don't exist on earlier OS versions → launch crash. - echo " Verifying deployment target compatibility..." - # Read the deployment target off the built app rather than the project file: the - # bundle is the only source that cannot disagree with what actually shipped. - local deploy_target - deploy_target=$(plutil -extract LSMinimumSystemVersion raw -o - "$app_path/Contents/Info.plist" 2>/dev/null || true) - if [ -n "$deploy_target" ]; then - local deploy_major - deploy_major=$(echo "$deploy_target" | cut -d. -f1) - local failed=0 - for fw in "$frameworks_dir"/*.dylib; do - [ -f "$fw" ] || continue - local name min_ver min_major - name=$(basename "$fw") - # otool -l prints LC_BUILD_VERSION with minos field, or LC_VERSION_MIN_MACOSX with version field - min_ver=$(otool -l "$fw" 2>/dev/null | awk '/LC_BUILD_VERSION/{found=1} found && /minos/{print $2; exit}') - if [ -z "$min_ver" ]; then - min_ver=$(otool -l "$fw" 2>/dev/null | awk '/LC_VERSION_MIN_MACOSX/{found=1} found && /version/{print $2; exit}') - fi - if [ -n "$min_ver" ]; then - min_major=$(echo "$min_ver" | cut -d. -f1) - if [ "$min_major" -gt "$deploy_major" ]; then - echo " ❌ FATAL: $name requires macOS $min_ver but deployment target is $deploy_target" - echo " This library was built on a newer macOS. Rebuild libpq with:" - echo " MACOSX_DEPLOYMENT_TARGET=$deploy_target brew reinstall libpq --build-from-source" - echo " Or use a CI runner on macOS $deploy_target+" - failed=1 - fi - fi - done - if [ "$failed" -eq 1 ]; then - echo "" - echo " Bundled dylibs target a newer macOS than the app's deployment target." - echo " The app will crash at launch on macOS $deploy_target with 'Symbol not found'." - exit 1 - fi - echo " ✅ All dylibs compatible with macOS $deploy_target" - else - echo " ⚠️ WARNING: Could not determine deployment target, skipping dylib version check" - fi - - echo "✅ Bundled $count dynamic libraries into Frameworks/" - ls -lh "$frameworks_dir"/*.dylib 2>/dev/null -} - -build_for_arch() { - local arch=$1 - echo "" - echo "🔨 Building for $arch..." - - # Prepare architecture-specific libraries - echo "📦 Preparing static libraries for $arch..." - prepare_arch_libs "$arch" libmariadb libpq libpgcommon libpgport libssl libcrypto libmongoc libbson libhiredis libhiredis_ssl libsqlite3_vendored - - # Create OpenSSL shared dylibs for this architecture - echo "📦 Creating OpenSSL shared dylibs for $arch..." - scripts/create-openssl-dylibs.sh "$arch" - - # TablePro.xcodeproj is generated and not in git, so a fresh checkout has none. - scripts/generate-project.sh - - # Persistent SPM package cache (speeds up CI on self-hosted runners) - SPM_CACHE_DIR="${HOME}/.spm-cache" - mkdir -p "$SPM_CACHE_DIR" - - # Resolve the provisioning profile UUID. Only the app target reads - # TABLEPRO_PROVISIONING_PROFILE (see Configs/Base.xcconfig), so passing it on the - # xcodebuild command line reaches the app and nothing else. A bare - # PROVISIONING_PROFILE_SPECIFIER override would hit every plugin and package target. - PROFILE_ARGS=() - PROFILE_PATH=$(find ~/Library/MobileDevice/Provisioning\ Profiles -name "*.provisionprofile" -print -quit 2>/dev/null) - if [ -n "${PROFILE_PATH:-}" ]; then - PROFILE_UUID=$(/usr/libexec/PlistBuddy -c "Print UUID" /dev/stdin <<< "$(security cms -D -i "$PROFILE_PATH" 2>/dev/null)" || true) - if [ -n "${PROFILE_UUID:-}" ]; then - echo "📋 Using provisioning profile: $PROFILE_UUID" - PROFILE_ARGS=("TABLEPRO_PROVISIONING_PROFILE=$PROFILE_UUID") - fi - fi - - # Build with xcodebuild - echo "Running xcodebuild..." - if ! xcodebuild \ - -project "$PROJECT" \ - -scheme "$SCHEME" \ - -configuration "$CONFIG" \ - -arch "$arch" \ - ONLY_ACTIVE_ARCH=YES \ - CODE_SIGN_IDENTITY="$SIGN_IDENTITY" \ - CODE_SIGN_STYLE=Manual \ - DEVELOPMENT_TEAM="$TEAM_ID" \ - CLANG_COVERAGE_MAPPING=NO \ - ENABLE_CODE_COVERAGE=NO \ - ${PROFILE_ARGS[@]+"${PROFILE_ARGS[@]}"} \ - -skipPackagePluginValidation \ - -clonedSourcePackagesDirPath "$SPM_CACHE_DIR" \ - -derivedDataPath build/DerivedData \ - build 2>&1 | tee "build-${arch}.log"; then - echo "❌ FATAL: xcodebuild failed for $arch" - echo "Check build-${arch}.log for details" - exit 1 - fi - echo "✅ Build succeeded for $arch" - - # Deterministic path via -derivedDataPath (no -showBuildSettings needed) - DERIVED_DATA="build/DerivedData/Build/Products" - - APP_PATH="${DERIVED_DATA}/${CONFIG}/TablePro.app" - echo "📂 Expected app path: $APP_PATH" - - # Verify app bundle exists - if [ ! -d "$APP_PATH" ]; then - echo "❌ ERROR: Built app not found at expected path: $APP_PATH" - echo "Build may have failed silently" - exit 1 - fi - - # Create release directory - mkdir -p "$BUILD_DIR" || { - echo "❌ FATAL: Failed to create release directory: $BUILD_DIR" - exit 1 - } - - # Copy and rename app. Removed first: cp -R onto an existing bundle merges into it, so a - # rebuild carried the previous build's files into the one being signed and shipped. - OUTPUT_NAME="TablePro-${arch}.app" - rm -rf "${BUILD_DIR:?}/$OUTPUT_NAME" - echo "Copying app bundle to release directory..." - if ! cp -R "$APP_PATH" "$BUILD_DIR/$OUTPUT_NAME"; then - echo "❌ FATAL: Failed to copy app bundle" - echo "Source: $APP_PATH" - echo "Destination: $BUILD_DIR/$OUTPUT_NAME" - exit 1 - fi - - # Verify the copy succeeded - if [ ! -d "$BUILD_DIR/$OUTPUT_NAME" ]; then - echo "❌ FATAL: App bundle was not copied successfully" - exit 1 - fi - - # Strip plugin binaries — removes debug symbols, code coverage (__LLVM_COV), - # and dead LINKEDIT metadata that bloat the bundle (e.g., OracleDriver 43MB → ~15MB) - echo "🔪 Stripping plugin binaries..." - PLUGINS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/PlugIns" - if [ -d "$PLUGINS_DIR" ]; then - for plugin in "$PLUGINS_DIR"/*.tableplugin; do - [ -d "$plugin" ] || continue - local plugin_name - plugin_name=$(basename "$plugin" .tableplugin) - local plugin_binary="$plugin/Contents/MacOS/$plugin_name" - if [ -f "$plugin_binary" ]; then - local before - before=$(ls -lh "$plugin_binary" | awk '{print $5}') - strip -x "$plugin_binary" - local after - after=$(ls -lh "$plugin_binary" | awk '{print $5}') - echo " $plugin_name: $before → $after" - fi - done - echo "✅ Plugin binaries stripped" - fi - - # Strip main binary - local main_binary="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/TablePro" - if [ -f "$main_binary" ]; then - local before - before=$(ls -lh "$main_binary" | awk '{print $5}') - strip -x "$main_binary" - local after - after=$(ls -lh "$main_binary" | awk '{print $5}') - echo "🔪 Main binary: $before → $after" - fi - - # Strip helper executables in Contents/MacOS - for helper in "$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS"/*; do - [ -f "$helper" ] || continue - [ "$(basename "$helper")" = "TablePro" ] && continue - local hname - hname=$(basename "$helper") - local before - before=$(ls -lh "$helper" | awk '{print $5}') - strip -x "$helper" - local after - after=$(ls -lh "$helper" | awk '{print $5}') - echo " $hname: $before → $after" - done - - # Strip PluginKit framework - local pluginkit_binary="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks/TableProPluginKit.framework/Versions/A/TableProPluginKit" - if [ -f "$pluginkit_binary" ]; then - strip -x "$pluginkit_binary" - echo " TableProPluginKit framework stripped" - fi - - # Remove development rpaths (absolute source paths) from all binaries - echo "🔧 Stripping development rpaths..." - for binary in "$main_binary" \ - "$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS"/* \ - "$PLUGINS_DIR"/*.tableplugin/Contents/MacOS/*; do - [ -f "$binary" ] || continue - otool -l "$binary" 2>/dev/null | grep "Libs/dylibs" | awk '{print $2}' | while read -r rpath; do - install_name_tool -delete_rpath "$rpath" "$binary" 2>/dev/null || true - done || true - done - - # Strip Sparkle helper binaries - local sparkle_dir="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks/Sparkle.framework/Versions/B" - for sparkle_bin in \ - "$sparkle_dir/Autoupdate" \ - "$sparkle_dir/Updater.app/Contents/MacOS/Updater"; do - if [ -f "$sparkle_bin" ]; then - strip -x "$sparkle_bin" - echo " $(basename "$sparkle_bin") (Sparkle) stripped" - fi - done - - # Remove Sparkle XPC services (not needed for non-sandboxed apps) - if [ -d "$sparkle_dir/XPCServices" ]; then - rm -rf "$sparkle_dir/XPCServices" - echo " Removed Sparkle XPC services (non-sandboxed app)" - fi - - # Copy shared OpenSSL dylibs into Frameworks - echo "📦 Copying OpenSSL shared dylibs to Frameworks/..." - FRAMEWORKS_EMBED_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks" - mkdir -p "$FRAMEWORKS_EMBED_DIR" - for lib in libcrypto.3.dylib libssl.3.dylib; do - if [ -f "Libs/dylibs/$lib" ]; then - cp -f "Libs/dylibs/$lib" "$FRAMEWORKS_EMBED_DIR/$lib" - chmod 644 "$FRAMEWORKS_EMBED_DIR/$lib" - echo " Copied $lib" - else - echo " WARNING: Libs/dylibs/$lib not found" - fi - done - - # Bundle non-system dynamic libraries (libpq, etc.) - bundle_dylibs "$BUILD_DIR/$OUTPUT_NAME" - - # Sign the entire app bundle with Developer ID. - # Sign from inside out: nested binaries → frameworks → dylibs → app. - echo "🔏 Signing app bundle with: $SIGN_IDENTITY" - FRAMEWORKS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks" - - # Sign all nested XPC services, helper apps, and executables inside frameworks - while IFS= read -r -d '' binary; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$binary" - done < <(find "$FRAMEWORKS_DIR" -type f \( -name "*.xpc" -o -perm +111 \) -not -name "*.dylib" -not -name "*.plist" -not -name "*.h" -not -name "*.strings" -not -name "*.nib" -not -name "*.png" -not -name "*.icns" -not -name "*.car" -not -name "CodeResources" -not -name "Info.plist" -print0 2>/dev/null) - - # Sign XPC service bundles - while IFS= read -r -d '' xpc; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$xpc" - done < <(find "$FRAMEWORKS_DIR" -name "*.xpc" -type d -print0 2>/dev/null) - - # Sign nested .app bundles (e.g., Sparkle's Updater.app) - while IFS= read -r -d '' app; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$app" - done < <(find "$FRAMEWORKS_DIR" -name "*.app" -type d -print0 2>/dev/null) - - # Sign top-level frameworks - for fw in "$FRAMEWORKS_DIR"/*.framework; do - [ -d "$fw" ] || continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$fw" - done - - # Sign top-level dylibs - for dylib in "$FRAMEWORKS_DIR"/*.dylib; do - [ -f "$dylib" ] || continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$dylib" - done - - # Sign plugin bundles (stripped binaries need re-signing) - # Sign binary first, then bundle — inside-out order required for valid signatures - if [ -d "$PLUGINS_DIR" ]; then - for plugin in "$PLUGINS_DIR"/*.tableplugin; do - [ -d "$plugin" ] || continue - local plugin_name - plugin_name=$(basename "$plugin" .tableplugin) - local plugin_binary="$plugin/Contents/MacOS/$plugin_name" - # Sign the binary inside the bundle first - if [ -f "$plugin_binary" ]; then - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_binary" - fi - # Then sign the bundle - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin" - done - fi - - # Sign helper executables in Contents/MacOS (e.g., mcp-server) - MACOS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS" - for helper in "$MACOS_DIR"/*; do - [ -f "$helper" ] || continue - [ "$(basename "$helper")" = "TablePro" ] && continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$helper" - done - - # Embed provisioning profile (required for iCloud entitlements) - PROFILE=$(find ~/Library/MobileDevice/Provisioning\ Profiles -name "*.provisionprofile" -print -quit 2>/dev/null) - if [ -n "$PROFILE" ]; then - echo "📋 Embedding provisioning profile: $(basename "$PROFILE")" - cp "$PROFILE" "$BUILD_DIR/$OUTPUT_NAME/Contents/embedded.provisionprofile" - fi - - # Sign the app bundle last - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp --entitlements "TablePro/TablePro.entitlements" "$BUILD_DIR/$OUTPUT_NAME" - echo "✅ Code signing complete" - - # Verify signature - if ! codesign --verify --deep --strict "$BUILD_DIR/$OUTPUT_NAME" 2>&1; then - echo "❌ FATAL: Code signature verification failed" - exit 1 - fi - echo "✅ Signature verified" - - # Verify binary exists inside the copied bundle - BINARY_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/TablePro" - if [ ! -f "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary not found in copied app bundle: $BINARY_PATH" - exit 1 - fi - - # Verify binary is not empty - if [ ! -s "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary file is empty" - exit 1 - fi - - # Verify binary is executable - if [ ! -x "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary is not executable" - exit 1 - fi - - # Verify embedded MCP stdio bridge made it into the bundle - MCP_CLI_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/tablepro-mcp" - if [ ! -x "$MCP_CLI_PATH" ]; then - echo "❌ FATAL: tablepro-mcp helper missing from $MCP_CLI_PATH" - echo "Check the mcp-server target's Copy Files build phase on the TablePro target." - exit 1 - fi - - SHIM_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/SharedSupport/bin/tablepro" - if [ ! -x "$SHIM_PATH" ]; then - echo "❌ FATAL: tablepro shim missing from $SHIM_PATH" - echo "Check the copy-files phase for TablePro/CLI/tablepro on the TablePro target." - exit 1 - fi - - # Get size - SIZE=$(ls -lh "$BINARY_PATH" 2>/dev/null | awk '{print $5}') - if [ -z "$SIZE" ]; then - echo "⚠️ WARNING: Could not determine binary size" - SIZE="unknown" - fi - - echo "✅ Built: $OUTPUT_NAME ($SIZE)" - - # Verify and display architecture - if ! lipo -info "$BINARY_PATH"; then - echo "⚠️ WARNING: Could not verify binary architecture" - fi -} - -# Main -case "$ARCH" in - arm64) - build_for_arch arm64 - ;; - x86_64) - build_for_arch x86_64 - ;; - both) - build_for_arch arm64 - echo "" - echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" - echo "" - build_for_arch x86_64 - ;; - *) - echo "Usage: $0 [arm64|x86_64|both]" - exit 1 - ;; -esac - -echo "" -echo "🎉 Build complete!" -echo "📁 Output: $BUILD_DIR/" - -if ! ls -lh "$BUILD_DIR" 2>/dev/null; then - echo "⚠️ WARNING: Could not list build directory contents" - echo "Directory may be empty or inaccessible" -fi - -# Notarization (opt-in via NOTARIZE=true) -if [ "$NOTARIZE" = "true" ]; then - echo "" - echo "📮 Notarizing..." - for app in "$BUILD_DIR"/TablePro-*.app; do - [ -d "$app" ] || continue - notarize_and_staple "$app" exec - done - echo "✅ Notarization complete" -fi diff --git a/scripts/check-pluginkit-abi.sh b/scripts/check-pluginkit-abi.sh index f1026ac933..cfd4cc50ad 100755 --- a/scripts/check-pluginkit-abi.sh +++ b/scripts/check-pluginkit-abi.sh @@ -6,16 +6,8 @@ set -euo pipefail # Builds TableProPluginKit at the current tree AND at a base ref with the SAME toolchain, then # diffs their public Swift interfaces. Comparing two builds from one compiler means Swift version # drift between a dev machine and CI cannot produce a false diff, so there is no committed baseline -# to keep in sync. A reported diff is a real ABI change to act on: -# -# Additive (a new requirement WITH a default implementation, a new field on a non-@frozen struct, -# a new case on a non-@frozen enum): bump currentPluginKitVersion in PluginManager.swift and -# TableProPluginKitVersion in every plugin Info.plist, leave minimumCompatiblePluginKitVersion -# alone, and re-publish nothing. Already-built plugins keep loading; the bump is what makes an -# older app refuse a plugin REBUILT against this API instead of failing its load. -# Breaking (changed/removed/renamed signature, a new case on a @frozen enum, a changed frozen -# layout): the same bumps, plus raise minimumCompatiblePluginKitVersion, then run -# scripts/release-all-plugins.sh . +# to keep in sync. A reported diff is a real ABI change; the rules printed at the end say what to +# raise in Configs/PluginKit.xcconfig. # # Both sides are generated from their project.yml first, so this regenerates # TablePro.xcodeproj in the working tree as well as in the base checkout. @@ -93,20 +85,16 @@ if diff -u "$work/base.txt" "$work/head.txt"; then exit 0 fi -# This is run by hand, per CLAUDE.md, not by a workflow. It used to have a branch that passed when -# ABI_ACKNOWLEDGED_ADDITIVE=1, set by a CI job that no longer exists, alongside instructions to add -# an `abi-additive` label and re-run. Nothing can set the variable and nothing reads the label, so -# both were telling the reader to do something that has no effect. cat <<'EOF' TableProPluginKit public ABI changed vs base (diff above). Decide additive vs breaking: Additive: a new requirement with a default, a reordering, or a field on a non-frozen transfer - struct. Bump currentPluginKitVersion and every plugin Info.plist - TableProPluginKitVersion; leave minimumCompatiblePluginKitVersion alone and re-publish - nothing. Shipped plugins keep loading; the bump makes an older app refuse a plugin - rebuilt against this API rather than failing its load. + struct. Raise TABLEPRO_PLUGINKIT_VERSION in Configs/PluginKit.xcconfig. Shipped plugins + keep loading; the bump makes an older app refuse a plugin rebuilt against this API + rather than failing its load. Breaking: a changed or removed requirement, a requirement without a default, a case on a @frozen - enum, or a frozen type's layout. The same bumps, plus raise - minimumCompatiblePluginKitVersion, then run scripts/release-all-plugins.sh . + enum, or a frozen type's layout. Raise TABLEPRO_PLUGINKIT_VERSION and + TABLEPRO_PLUGINKIT_MINIMUM_VERSION. + Registry plugins ship with the next app release, built against the new kit. EOF exit 1 diff --git a/scripts/check-registry-readiness.py b/scripts/check-registry-readiness.py deleted file mode 100644 index 934f5dbb5e..0000000000 --- a/scripts/check-registry-readiness.py +++ /dev/null @@ -1,81 +0,0 @@ -#!/usr/bin/env python3 -"""Fail if any registry database driver lacks a binary compatible with the app's -PluginKit range [floor, current]. - -Run this before tagging an app release so the app can never ship ahead of its -plugin binaries. With range-aware binary selection an additive PluginKit bump -needs no re-publish (an older resilient binary still serves), so this only fails -after a breaking bump that raised the floor and left the registry behind. - -The gate reads the raw GitHub origin, not the jsDelivr CDN that clients use, so -it sees the manifest the moment the registry push lands rather than waiting out -(or racing) the CDN edge cache. -""" - -import argparse -import json -import sys -import urllib.request - -DEFAULT_MANIFEST_URL = "https://raw.githubusercontent.com/TableProApp/plugins/main/plugins.json" - - -def fetch_manifest(url, retries=4): - last_error = None - for attempt in range(retries): - try: - request = urllib.request.Request(url, headers={"Cache-Control": "no-cache"}) - with urllib.request.urlopen(request, timeout=30) as response: - return json.load(response) - except Exception as error: # noqa: BLE001 - surface any fetch/parse failure - last_error = error - raise SystemExit(f"ERROR: could not fetch registry manifest from {url}: {last_error}") - - -def compatible_kits(plugin): - return sorted({ - binary.get("pluginKitVersion") - for binary in plugin.get("binaries", []) - if binary.get("pluginKitVersion") is not None - }) - - -def has_compatible_binary(plugin, floor, current): - return any(floor <= kit <= current for kit in compatible_kits(plugin)) - - -def main(): - parser = argparse.ArgumentParser(description="Check plugin registry readiness for an app release") - parser.add_argument("--manifest-url", default=DEFAULT_MANIFEST_URL) - parser.add_argument("--floor", required=True, type=int, help="minimumCompatiblePluginKitVersion") - parser.add_argument("--current", required=True, type=int, help="currentPluginKitVersion") - args = parser.parse_args() - - manifest = fetch_manifest(args.manifest_url) - plugins = manifest.get("plugins", manifest if isinstance(manifest, list) else []) - drivers = [plugin for plugin in plugins if plugin.get("category") == "database-driver"] - - if not drivers: - raise SystemExit("ERROR: no database-driver plugins found in the registry manifest") - - not_ready = [] - for plugin in drivers: - if not has_compatible_binary(plugin, args.floor, args.current): - name = plugin.get("name") or plugin.get("id") or "?" - not_ready.append(f"{name}: no binary in [{args.floor},{args.current}] (has {compatible_kits(plugin)})") - - if not_ready: - print("Registry is NOT ready for this app release:", file=sys.stderr) - for entry in not_ready: - print(f" - {entry}", file=sys.stderr) - print( - "Run scripts/release-all-plugins.sh for the new PluginKit version before tagging the app.", - file=sys.stderr, - ) - sys.exit(1) - - print(f"Registry ready: all {len(drivers)} database drivers have a binary in [{args.floor},{args.current}].") - - -if __name__ == "__main__": - main() diff --git a/scripts/ci/appcast_feed.py b/scripts/ci/appcast_feed.py deleted file mode 100644 index 0aba8980e4..0000000000 --- a/scripts/ci/appcast_feed.py +++ /dev/null @@ -1,198 +0,0 @@ -"""Text-level primitives for editing the published Sparkle feed. - -Two scripts change appcast.xml: merge-appcast.py adds a release's items, pull-release.py removes -them again when a build has to be withdrawn. They have to agree on where an item starts and ends, -so that lives here rather than in both. - -Everything works on the file's text, because a full ElementTree round trip destroys every CDATA -section. Measured on the published 695 KB feed: the round trip takes 7 ms, keeps the sparkle prefix -exactly and is semantically identical, but entity-escapes all 155 release-note blocks and grows the -file 9%, which every install then downloads and every future diff then buries. (The older claim here -that a round trip rewrites namespace prefixes was wrong; `register_namespace` preserves them.) So -items that are not being touched are moved byte for byte and only parsed to be understood. -""" - -import pathlib -import xml.etree.ElementTree as ElementTree - -SPARKLE_NS = "http://www.andymatuschak.org/xml-namespaces/sparkle" -SHORT_VERSION = f"{{{SPARKLE_NS}}}shortVersionString" -BUNDLE_VERSION = f"{{{SPARKLE_NS}}}version" -HARDWARE = f"{{{SPARKLE_NS}}}hardwareRequirements" -ED_SIGNATURE = f"{{{SPARKLE_NS}}}edSignature" - - -class FeedError(Exception): - """A violated invariant. Every one of these fails the job that raised it.""" - - -def parse_text(text): - try: - return ElementTree.fromstring(text) - except ElementTree.ParseError as error: - raise FeedError(f"not well-formed XML: {error}") from error - - -def parse(path): - try: - return ElementTree.parse(path).getroot() - except (OSError, ElementTree.ParseError) as error: - raise FeedError(f"{path}: {error}") from error - - -def read(path): - try: - return pathlib.Path(path).read_text(encoding="utf-8") - except OSError as error: - raise FeedError(f"{path}: {error}") from error - - -def channel_items(root, path): - channel = root.find("channel") - if channel is None: - raise FeedError(f"{path}: no element") - return channel.findall("item") - - -def item_short_version(item): - element = item.find(SHORT_VERSION) - return None if element is None else (element.text or "").strip() - - -def item_bundle_version(item): - element = item.find(BUNDLE_VERSION) - return None if element is None else (element.text or "").strip() - - -def item_enclosures(item): - return list(item.findall("enclosure")) - - -SKIPPABLE = (("", "CDATA section"), ("", "comment")) -ITEM_OPEN = "/ \t\r\n") - - -def skip_region(text, index): - """The offset just past a CDATA section or comment starting at `index`, else None.""" - for opener, closer, label in SKIPPABLE: - if text.startswith(opener, index): - end = text.find(closer, index + len(opener)) - if end == -1: - raise FeedError(f"unterminated {label} in the feed") - return end + len(closer) - return None - - -def opens_item(text, index): - """True only for a real `` tag. `` starts with the same five characters.""" - if not text.startswith(ITEM_OPEN, index): - return False - after = index + len(ITEM_OPEN) - return after >= len(text) or text[after] in TAG_DELIMITERS - - -def item_spans(text): - """Whole-line (start, end) offsets of every in document order. - - Release-note CDATA can hold either literal tag, and so can an XML comment, so both are skipped - whole rather than matching tags pairwise across the document. An item never nests inside another - item, so the first `` outside a skipped region closes it. Every marker begins with `<`, - so the scan hops between those rather than walking characters. - """ - spans = [] - index = 0 - length = len(text) - while index < length: - index = text.find("<", index) - if index == -1: - break - skip = skip_region(text, index) - if skip is not None: - index = skip - continue - if not opens_item(text, index): - index += 1 - continue - - open_at = index - cursor = open_at + len(ITEM_OPEN) - close_at = -1 - while cursor < length: - cursor = text.find("<", cursor) - if cursor == -1: - break - skip = skip_region(text, cursor) - if skip is not None: - cursor = skip - continue - if text.startswith(ITEM_CLOSE, cursor): - close_at = cursor - break - cursor += 1 - if close_at == -1: - raise FeedError("an is never closed in the feed") - - line_start = text.rfind("\n", 0, open_at) + 1 - line_end = text.find("\n", close_at) - line_end = length if line_end == -1 else line_end + 1 - spans.append((line_start, line_end)) - index = close_at + len(ITEM_CLOSE) - return spans - - -def sole_item_text(path, version): - """The one item's own bytes, whole lines, taken from a generated single-item feed.""" - text = read(path) - spans = item_spans(text) - if len(spans) != 1: - raise FeedError(f"{path}: expected exactly one , found {len(spans)}") - start, end = spans[0] - block = text[start:end] - return block if block.endswith("\n") else block + "\n" - - -def splice_point(text, path): - """Where new items go: before the newest existing item, else before .""" - spans = item_spans(text) - if spans: - return spans[0][0] - anchor = text.find("") - if anchor == -1: - raise FeedError(f"{path}: no and no to splice against") - return text.rfind("\n", 0, anchor) + 1 - - -def insert_items(base_text, path, items_text): - """Inserts whole lines at a line boundary. - - The old check here compared the merged text minus the inserted slice against the base, which - concatenation makes true by construction: it could never fail. The invariant worth asserting is - the one `splice_point` promises and `sole_item_text` relies on, which is that both the cut and - the inserted block land on line boundaries. - """ - at = splice_point(base_text, path) - if at != 0 and base_text[at - 1] != "\n": - raise FeedError(f"{path}: the splice point is mid-line, so the inserted item would be merged into another") - if items_text and not items_text.endswith("\n"): - raise FeedError(f"{path}: the inserted items do not end on a line boundary") - return base_text[:at] + items_text + base_text[at:] - - -def remove_version(base_text, path, version): - """Every item advertising `version`, removed whole. Returns (text, count).""" - root = parse_text(base_text) - spans = item_spans(base_text) - items = channel_items(root, path) - if len(spans) != len(items): - raise FeedError( - f"{path}: found {len(spans)} item spans in the text but {len(items)} parsed items" - ) - doomed = [span for span, item in zip(spans, items) if item_short_version(item) == version] - if not doomed: - return base_text, 0 - result = base_text - for start, end in reversed(doomed): - result = result[:start] + result[end:] - return result, len(doomed) diff --git a/scripts/ci/changelog_credits.py b/scripts/ci/changelog_credits.py deleted file mode 100644 index 0923880f4e..0000000000 --- a/scripts/ci/changelog_credits.py +++ /dev/null @@ -1,214 +0,0 @@ -#!/usr/bin/env python3 -"""Credit every CHANGELOG entry of one version to the pull request that added it. - -An outside contributor's entry ends `(#123 by @author)`, a maintainer's ends `(#123)`. An issue -reference stays first: `(#1748, #2741 by @author)`. A maintainer is a pull request author whose role -on the repository is admin or maintain. - -The credit comes from the history: `git blame` names the commit that wrote each entry line, a squash -merge ends its subject with `(#123)`, and `gh pr view` names that pull request's author. Blame -reports the last commit to touch a line, so reword a contributor's entry in their own pull request. -An entry whose commit has no pull request number, or that is not committed yet, is left alone and -listed. When an author's role cannot be read, the entry keeps the handle and the login is listed. - -Usage: - python3 scripts/ci/changelog_credits.py [--section Unreleased] [--dry-run] - -Run it from the repository root. It rewrites CHANGELOG.md in place and is idempotent. -""" - -import argparse -import json -import re -import subprocess -import sys -from dataclasses import dataclass -from pathlib import Path - -REFERENCES = re.compile(r"\s*\((#\d+(?:,\s*#\d+)*)\)\s*$") -CREDITED = re.compile(r"\(#\d+(?:,\s*#\d+)* by @[A-Za-z0-9][A-Za-z0-9-]*\)\s*$") -PULL_REQUEST_SUBJECT = re.compile(r"\(#(\d+)\)\s*$") -UNCOMMITTED = "0" * 40 -MAINTAINER_ROLES = {"admin", "maintain"} - - -def is_entry(line): - return line.startswith("- ") - - -def is_credited(line): - return CREDITED.search(line) is not None - - -def normalized_login(login): - """`gh` reports a GitHub App as `app/name`, which GitHub itself renders as `@name`.""" - return login.split("/", 1)[1] if login.startswith("app/") else login - - -def credit_entry(line, pull_request, author=None): - """`line` with the pull request folded into its reference parens, and `by @author` unless None.""" - if is_credited(line): - return line - body = line.rstrip() - references = [] - existing = REFERENCES.search(body) - if existing: - references = [reference.strip() for reference in existing.group(1).split(",")] - body = body[:existing.start()] - if f"#{pull_request}" not in references: - references.append(f"#{pull_request}") - credit = "" if author is None else f" by @{normalized_login(author)}" - return f"{body} ({', '.join(references)}{credit})" - - -def section_bounds(lines, section): - """Zero-based `[start, end)` of the entries under `## [section]`, heading excluded.""" - heading = f"## [{section}]" - start = next((index for index, line in enumerate(lines) if line.startswith(heading)), None) - if start is None: - raise ValueError(f"CHANGELOG.md has no {heading} section") - end = next( - (index for index in range(start + 1, len(lines)) if lines[index].startswith("## [")), - len(lines), - ) - return start + 1, end - - -@dataclass -class BlamedLine: - sha: str - summary: str - - -def parse_blame(porcelain): - """One `BlamedLine` per line of `git blame --line-porcelain` output, in order.""" - blamed = [] - sha = None - summary = "" - for line in porcelain.split("\n"): - header = re.match(r"^([0-9a-f]{40}) \d+ \d+", line) - if header: - sha = header.group(1) - summary = "" - elif line.startswith("summary "): - summary = line[len("summary "):] - elif line.startswith("\t") and sha is not None: - blamed.append(BlamedLine(sha=sha, summary=summary)) - return blamed - - -def pull_request_of(blamed): - if blamed.sha == UNCOMMITTED: - return None - match = PULL_REQUEST_SUBJECT.search(blamed.summary) - return match.group(1) if match else None - - -def blame_section(start, end): - result = subprocess.run( - ["git", "blame", "--line-porcelain", "-L", f"{start + 1},{end}", "--", "CHANGELOG.md"], - capture_output=True, - text=True, - check=True, - ) - return parse_blame(result.stdout) - - -def author_of(pull_request, cache): - if pull_request not in cache: - result = subprocess.run( - ["gh", "pr", "view", pull_request, "--json", "author"], - capture_output=True, - text=True, - check=True, - ) - cache[pull_request] = json.loads(result.stdout)["author"]["login"] - return cache[pull_request] - - -class Maintainers: - """Whether a login holds the admin or maintain role on the repository, asked once per login.""" - - def __init__(self, run=subprocess.run): - self._run = run - self._roles = {} - self.unknown = set() - - def includes(self, login): - if login.startswith("app/"): - return False - if login not in self._roles: - self._roles[login] = self._role_of(login) - return self._roles[login] in MAINTAINER_ROLES - - def _role_of(self, login): - result = self._run( - ["gh", "api", f"repos/{{owner}}/{{repo}}/collaborators/{login}/permission", "--jq", ".role_name"], - capture_output=True, - text=True, - ) - if result.returncode != 0: - self.unknown.add(login) - return None - return result.stdout.strip() - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--section", default="Unreleased", help="the version heading to credit") - parser.add_argument("--dry-run", action="store_true", help="print the result instead of writing it") - args = parser.parse_args(argv) - - path = Path("CHANGELOG.md") - lines = path.read_text(encoding="utf-8").split("\n") - try: - start, end = section_bounds(lines, args.section) - except ValueError as error: - sys.exit(f"ERROR: {error}") - if start == end: - print(f"[{args.section}] has no entries") - return 0 - - blamed = blame_section(start, end) - if len(blamed) != end - start: - sys.exit("ERROR: git blame did not return one record per line") - - authors = {} - maintainers = Maintainers() - credited = 0 - skipped = [] - for offset, record in enumerate(blamed): - index = start + offset - line = lines[index] - if not is_entry(line) or is_credited(line): - continue - pull_request = pull_request_of(record) - if pull_request is None: - reason = "not committed yet" if record.sha == UNCOMMITTED else f"{record.sha[:9]} has no pull request number" - skipped.append(f"line {index + 1}: {reason}: {line[:90]}") - continue - author = author_of(pull_request, authors) - handle = None if maintainers.includes(author) else author - lines[index] = credit_entry(line, pull_request, handle) - credited += 1 - - if args.dry_run: - print("\n".join(lines[start:end])) - else: - path.write_text("\n".join(lines), encoding="utf-8") - - logins = set(authors.values()) - inside = sorted(normalized_login(login) for login in logins if maintainers.includes(login)) - outside = sorted(normalized_login(login) for login in logins if not maintainers.includes(login)) - print(f"Credited {credited} entries in [{args.section}] across {len(authors)} pull requests.") - print(f"Maintainers, credited by pull request only: {', '.join('@' + login for login in inside) or 'none'}") - print(f"Contributors: {', '.join('@' + login for login in outside) or 'none'}") - for login in sorted(maintainers.unknown): - print(f"Role unknown, credited by handle: @{login}") - for line in skipped: - print(f"Left uncredited, {line}") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/check-changelog.py b/scripts/ci/check-changelog.py new file mode 100755 index 0000000000..a197d28acc --- /dev/null +++ b/scripts/ci/check-changelog.py @@ -0,0 +1,262 @@ +#!/usr/bin/env python3 + +import argparse +import datetime +import importlib.util +import json +import os +import re +import subprocess +import sys +import tempfile +from pathlib import Path +from typing import Callable, Dict, List, NamedTuple, Optional, Tuple + +ROOT = Path(__file__).resolve().parent.parent.parent +NOTES_SPEC = importlib.util.spec_from_file_location("notes", ROOT / "scripts" / "release" / "notes.py") +notes = importlib.util.module_from_spec(NOTES_SPEC) +NOTES_SPEC.loader.exec_module(notes) + +REPOSITORY = "TableProApp/TablePro" +MARKER = "" +FRAGMENTS = "changelog.d" +IGNORED = {".gitkeep"} +MAX_LINE = 120 +TYPE_DIRECTORY = re.compile(r'^\[\[tool\.towncrier\.type\]\]\ndirectory = "([a-z]+)"$', re.MULTILINE) +VERSION = r"(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)" +DATE = r"(?P[0-9]{4}-[0-9]{2}-[0-9]{2})" +OLD_HEADING = re.compile(rf"## \[(?P{VERSION})\] - {DATE}") +NEW_HEADING = re.compile( + rf"## \[(?P{VERSION})\]\(https://github\.com/{re.escape(REPOSITORY)}/releases/tag/v(?P=version)\)" + rf" - {DATE}(?: \[CRITICAL\])?" +) +# The forward-port to main keeps the release commit's subject, which ends in its (#N). +RELEASE_TITLE = re.compile(rf"chore\(release\): v{VERSION}(?: \(#[0-9]+\))?") +NEW_STATUSES = {"added", "copied"} + + +class Fragment(NamedTuple): + owner: str + kind: str + counter: Optional[str] + + +class ChangedFile(NamedTuple): + filename: str + status: str + previous_filename: Optional[str] = None + + +class CheckError(Exception): + pass + + +def fragment_types(towncrier_toml: str) -> List[str]: + return TYPE_DIRECTORY.findall(towncrier_toml) + + +def parse_fragment_name(name: str, types: List[str]) -> Optional[Fragment]: + match = re.fullmatch( + rf"(?P[0-9]+|\+[a-z0-9][a-z0-9-]*)\.(?P{'|'.join(types)})(?:\.(?P[1-9][0-9]*))?\.md", + name, + ) + if not match: + return None + return Fragment(match.group("owner"), match.group("kind"), match.group("counter")) + + +def fragment_problems(directory: Path, types: List[str]) -> List[str]: + if not directory.is_dir(): + return [] + problems = [] + for path in sorted(directory.iterdir()): + name = f"{FRAGMENTS}/{path.name}" + if path.name in IGNORED: + continue + if not path.is_file(): + problems.append(f"{name} is not a file") + continue + if parse_fragment_name(path.name, types) is None: + problems.append(f"{name} is not named ..md or ...md, with type one of " + + ", ".join(types)) + try: + text = path.read_text(encoding="utf-8") + except UnicodeDecodeError: + problems.append(f"{name} is not UTF-8") + continue + lines = [line.strip() for line in text.splitlines() if line.strip()] + if len(lines) != 1: + problems.append(f"{name} has {len(lines)} non-blank lines; a fragment is one line") + continue + line = lines[0] + if len(line) > MAX_LINE: + problems.append(f"{name} is {len(line)} characters, over the {MAX_LINE} limit") + if line.startswith("- "): + problems.append(f"{name} starts with `- `; towncrier adds the bullet") + if "by @" in line: + problems.append(f"{name} credits an author; notes.py adds the credit at release") + return problems + + +def _is_date(text: str) -> bool: + try: + datetime.date.fromisoformat(text) + except ValueError: + return False + return True + + +def changelog_problems(changelog: str) -> List[str]: + lines = changelog.split("\n") + markers = [index for index, line in enumerate(lines) if line == MARKER] + if len(markers) != 1: + return [f"CHANGELOG.md has {len(markers)} `{MARKER}` lines, expected 1"] + marker = markers[0] + problems = [] + + first = next((index for index in range(marker + 1, len(lines)) if lines[index].startswith("## [")), None) + if first is not None and lines[marker + 1:first] != [""]: + problems.append(f"CHANGELOG.md:{marker + 1}: only one blank line goes between the marker and the first ## [") + + versions: List[Tuple[int, ...]] = [] + seen: Dict[str, int] = {} + for index, line in enumerate(lines): + if not line.startswith("## "): + continue + number = index + 1 + if index < marker: + problems.append(f"CHANGELOG.md:{number}: `{line}` is above the marker") + continue + old, new = OLD_HEADING.fullmatch(line), NEW_HEADING.fullmatch(line) + heading = old or new + if heading is None: + problems.append(f"CHANGELOG.md:{number}: `{line}` is not `## [X.Y.Z]() - YYYY-MM-DD`") + continue + if not _is_date(heading.group("date")): + problems.append(f"CHANGELOG.md:{number}: {heading.group('date')} is not a date") + version = heading.group("version") + if version in seen: + problems.append(f"CHANGELOG.md:{number}: ## [{version}] repeats line {seen[version]}") + continue + seen[version] = number + parsed = tuple(int(part) for part in version.split(".")) + if versions and parsed >= versions[-1]: + problems.append(f"CHANGELOG.md:{number}: ## [{version}] is not below the version above it") + versions.append(parsed) + if new is not None: + problems += _new_section_problems(changelog, version, number) + return problems + + +def _new_section_problems(changelog: str, version: str, number: int) -> List[str]: + section = notes.find_section(changelog, version) + if section is None: + return [f"CHANGELOG.md:{number}: notes.py cannot find ## [{version}]"] + problems = [] + if len(section.lead) > notes.MAX_LEAD_LINES: + problems.append(f"CHANGELOG.md:{number}: the {version} lead block has {len(section.lead)} lines, " + f"over {notes.MAX_LEAD_LINES}") + with tempfile.TemporaryDirectory() as scratch: + source = Path(scratch, "CHANGELOG.md") + source.write_text(changelog, encoding="utf-8") + if not notes.whats_new(version, source, Path(scratch, "WhatsNew.md")): + problems.append(f"CHANGELOG.md:{number}: notes.py whats-new renders nothing for {version}") + return problems + + +def repository_problems(root: Path) -> List[str]: + types = fragment_types((root / "towncrier.toml").read_text(encoding="utf-8")) + if not types: + return ["towncrier.toml defines no [[tool.towncrier.type]]"] + return (changelog_problems((root / "CHANGELOG.md").read_text(encoding="utf-8")) + + fragment_problems(root / FRAGMENTS, types)) + + +def _fragment_in(path: Optional[str], types: List[str]) -> Optional[Fragment]: + if path is None or not path.startswith(f"{FRAGMENTS}/"): + return None + return parse_fragment_name(path[len(FRAGMENTS) + 1:], types) + + +def pull_request_problems(number: int, title: str, base: str, files: List[ChangedFile], + types: List[str]) -> List[str]: + # The pull request that creates changelog.d/ is the one that moves the old entries into it. + if any(changed.filename == f"{FRAGMENTS}/.gitkeep" and changed.status in NEW_STATUSES for changed in files): + return [] + problems = [] + expected = f"{FRAGMENTS}/{number}..md" + if any("CHANGELOG.md" in (changed.filename, changed.previous_filename) for changed in files) \ + and not RELEASE_TITLE.fullmatch(title): + problems.append(f"only a `chore(release): vX.Y.Z` pull request edits CHANGELOG.md. Put the entry in " + f"{expected}, with type one of {', '.join(types)} (.claude/rules/changelog.md)") + if base != "main": + return problems + + # A fragment deleted and added again under a new type keeps the pull request it is named for. + removed = {fragment.owner for fragment in + (_fragment_in(changed.filename, types) for changed in files if changed.status == "removed") + if fragment is not None} + for changed in files: + if changed.status not in NEW_STATUSES or not changed.filename.startswith(f"{FRAGMENTS}/"): + continue + if changed.filename[len(FRAGMENTS) + 1:] in IGNORED: + continue + fragment = _fragment_in(changed.filename, types) + if fragment is None: + problems.append(f"{changed.filename} is not a fragment name. Name it {expected}") + elif fragment.owner != str(number) and fragment.owner not in removed: + counter = f".{fragment.counter}" if fragment.counter else "" + problems.append(f"{changed.filename} is named for another pull request. " + f"Name it {FRAGMENTS}/{number}.{fragment.kind}{counter}.md") + return problems + + +def _gh(run: Callable, endpoint: str, jq: str, paginate: bool = False) -> List[str]: + command = ["gh", "api", *(["--paginate"] if paginate else []), endpoint, "--jq", jq] + result = run(command, capture_output=True, text=True) + if result.returncode != 0: + raise CheckError(f"gh api {endpoint}: {result.stderr.strip()}") + return [line for line in result.stdout.splitlines() if line.strip()] + + +def fetch_pull_request(number: int, repository: str, + run: Callable = subprocess.run) -> Tuple[str, str, List[ChangedFile]]: + endpoint = f"repos/{repository}/pulls/{number}" + details = json.loads(_gh(run, endpoint, "{title: .title, base: .base.ref} | @json")[0]) + entries = _gh(run, f"{endpoint}/files", ".[] | {filename, status, previous_filename} | @json", paginate=True) + files = [ChangedFile(entry["filename"], entry["status"], entry.get("previous_filename")) + for entry in map(json.loads, entries)] + return details["title"], details["base"], files + + +def workflow_command_data(text: str) -> str: + return text.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A") + + +def main(argv: Optional[List[str]] = None, run: Callable = subprocess.run) -> int: + parser = argparse.ArgumentParser(description="Check changelog.d/ and CHANGELOG.md.") + parser.add_argument("--pr", type=int, help="check the files this pull request changes instead") + parser.add_argument("--root", type=Path, default=ROOT) + args = parser.parse_args(argv) + + try: + if args.pr is None: + problems = repository_problems(args.root) + else: + types = fragment_types((args.root / "towncrier.toml").read_text(encoding="utf-8")) + repository = os.environ.get("GITHUB_REPOSITORY", REPOSITORY) + title, base, files = fetch_pull_request(args.pr, repository, run) + problems = pull_request_problems(args.pr, title, base, files, types) + except (CheckError, OSError, ValueError, KeyError, IndexError) as error: + problems = [str(error)] + + for problem in problems: + print(f"::error title=Changelog::{workflow_command_data(problem)}") + if problems: + return 1 + print("OK: changelog.d/ and CHANGELOG.md" if args.pr is None else f"OK: the changelog files of #{args.pr}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/ci/check-plugin-manifest.py b/scripts/ci/check-plugin-manifest.py index dca111f733..971802dce9 100755 --- a/scripts/ci/check-plugin-manifest.py +++ b/scripts/ci/check-plugin-manifest.py @@ -1,19 +1,20 @@ #!/usr/bin/env python3 """Fails when .github/plugin-registry.json, the plugin classes and the project.yml plugin aggregates disagree.""" -import json import pathlib import re import sys REPO_ROOT = pathlib.Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO_ROOT / "scripts" / "release")) + +import registry # noqa: E402 + MANIFEST = REPO_ROOT / ".github" / "plugin-registry.json" PROJECT = REPO_ROOT / "project.yml" PLUGINS_DIR = REPO_ROOT / "Plugins" ICON = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) iconName(?:\s*:\s*String)?\s*=\s*\"([^\"]+)\"", re.M) -PRIMARY_ID = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) databaseTypeId(?:\s*:\s*String)?\s*=\s*\"([^\"]+)\"", re.M) -EXTRA_IDS = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) additionalDatabaseTypeIds\s*(?::\s*\[String\]\s*)?=\s*\[([^\]]*)\]", re.M) TOP_LEVEL_KEY = re.compile(r"^([A-Za-z_][\w-]*):") ENTRY = re.compile(r"^ ([A-Za-z_][\w-]*):\s*$") @@ -72,7 +73,7 @@ def embedded_plugins(app): def publish_set(plugins): - return {entry["target"] for entry in plugins.values() if not entry.get("bundled")} + return set(plugins) def aggregate_problems(project, plugins): @@ -122,57 +123,35 @@ def normalise(name): return None -def declared(directory): - icon = ids = None - extra = [] +def declared_icon(directory): for source in sorted(directory.rglob("*.swift")): - text = source.read_text(encoding="utf-8") - if icon is None: - found = ICON.search(text) - icon = found.group(1) if found else None - if ids is None: - found = PRIMARY_ID.search(text) - ids = found.group(1) if found else None - found = EXTRA_IDS.search(text) - if found and not extra: - extra = [part.strip().strip('"') for part in found.group(1).split(",") if part.strip()] - return icon, ids, extra + found = ICON.search(source.read_text(encoding="utf-8")) + if found: + return found.group(1) + return None def declaration_problems(plugins): problems = [] - for slug, entry in sorted(plugins.items()): - directory = plugin_directory(entry["target"]) + for target, entry in sorted(plugins.items()): + directory = plugin_directory(target) if directory is None: - problems.append(f"{slug}: no plugin directory matches target {entry['target']}") + problems.append(f"{target}: no plugin directory matches it") continue - - icon, primary, extra = declared(directory) + icon = declared_icon(directory) if icon is None: - problems.append(f"{slug}: could not read iconName from {directory.name}") - elif icon != entry["icon"]: - problems.append(f"{slug}: manifest icon {entry['icon']!r} but the plugin declares {icon!r}") - - # An export or import plugin drives no database type and declares none. - if entry["databaseTypeIds"] is None: - if primary is not None: - problems.append(f"{slug}: manifest says no database types but the plugin declares {primary!r}") - continue - - if primary is None: - problems.append(f"{slug}: could not read databaseTypeId from {directory.name}") - continue - if [primary, *extra] != entry["databaseTypeIds"]: - problems.append( - f"{slug}: manifest databaseTypeIds {entry['databaseTypeIds']} " - f"but the plugin declares {[primary, *extra]}" - ) + problems.append(f"{target}: could not read iconName from {directory.name}") + elif icon != entry["iconName"]: + problems.append(f"{target}: {MANIFEST.name} says iconName {entry['iconName']!r} " + f"but the plugin declares {icon!r}") return problems def main(): - with open(MANIFEST, encoding="utf-8") as handle: - plugins = json.load(handle)["plugins"] + try: + plugins = registry.load_metadata(MANIFEST)["plugins"] + except registry.RegistryError as error: + sys.exit(f"::error::{error}") project = parse_project(PROJECT.read_text(encoding="utf-8")) problems = declaration_problems(plugins) + aggregate_problems(project, plugins) diff --git a/scripts/ci/check-pr-title.py b/scripts/ci/check-pr-title.py index 0a54f3086b..bcb7785be2 100644 --- a/scripts/ci/check-pr-title.py +++ b/scripts/ci/check-pr-title.py @@ -1,13 +1,4 @@ #!/usr/bin/env python3 -"""Check a pull request title against the commit subject rules, because a squash merge uses it. - -The title must be a Conventional Commits subject, `()!: `, with the scope -and the `!` optional, and the title must fit in 72 characters. GitHub appends ` (#123)` when it -squashes; that suffix is not counted. - -Usage: - python3 scripts/ci/check-pr-title.py TITLE -""" import re import sys @@ -15,6 +6,8 @@ TYPES = ("feat", "fix", "refactor", "perf", "test", "docs", "build", "ci", "chore", "style", "revert") MAX_TITLE = 72 SUBJECT = re.compile(rf"^(?:{'|'.join(TYPES)})(?:\([a-z0-9][a-z0-9-]*\))?!?: \S") +# A pick into a release line keeps its subject from main, whose trailing (#N) names its origin. +PROVENANCE = re.compile(r" \(#[0-9]+\)$") def problems(title): @@ -26,8 +19,9 @@ def problems(title): + ", ".join(TYPES) + " and the scope is lowercase words joined by hyphens, for example `fix(editor): keep the caret`" ) - if len(title) > MAX_TITLE: - found.append(f"the title is {len(title)} characters, over the {MAX_TITLE} limit") + length = len(PROVENANCE.sub("", title, count=1)) + if length > MAX_TITLE: + found.append(f"the title is {length} characters, over the {MAX_TITLE} limit") return found diff --git a/scripts/ci/detect-changed-paths.sh b/scripts/ci/detect-changed-paths.sh index 0062d4273b..6eab0add36 100755 --- a/scripts/ci/detect-changed-paths.sh +++ b/scripts/ci/detect-changed-paths.sh @@ -24,7 +24,7 @@ case "$SUITE" in 'TablePro\.xcodeproj/project\.xcworkspace/xcshareddata/swiftpm/Package\.resolved$' 'scripts/(generate-project|download-libs|create-openssl-dylibs|build-dameng|build-hana)\.sh$' 'scripts/lib/' 'scripts/ci/(detect-changed-paths|summarize-xcresult|verify-plugin-loads)\.sh$' - '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/macos-tests\.yml$') ;; + 'scripts/release/notes\.py$' '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/macos-tests\.yml$') ;; plugins) # Registry plugins link the TableProCore and TableProOracle products. PATTERNS=('Plugins/' 'Packages/TableProCore/' 'Packages/TableProOracle/' 'Native/' 'Libs/' @@ -75,26 +75,17 @@ if [ -z "$BASE" ]; then exit 0 fi -# A release commit changes only these, so it reuses the verdict of the code it ships. -RELEASE_ONLY='CHANGELOG\.md|appcast\.xml|TablePro/Resources/WhatsNew\.md|\.github/release-flags\.json' - -# Version.xcconfig is the TablePro target's config file, so it counts as release-only only while -# its diff touches nothing but the two version lines. -VERSION_DIFF="$(git diff -U0 "$BASE" HEAD -- Configs/Version.xcconfig)" -OTHER_EDITS="$(awk '/^@@/ { hunk = 1; next } - hunk && /^[+-]/ && !/^[+-](MARKETING_VERSION|CURRENT_PROJECT_VERSION) = [0-9.]+$/' <<< "$VERSION_DIFF")" -[ -n "$OTHER_EDITS" ] || RELEASE_ONLY+='|Configs/Version\.xcconfig' - -IGNORED="^($RELEASE_ONLY)\$" -[ -z "$SKIP" ] || IGNORED+="|^($SKIP)" - # Raw NUL-separated paths: core.quotePath would quote a non-ASCII path and defeat the ^ anchor. # Files rather than $(...), which drops NUL bytes and would hide grep's exit status. CHANGED="$(mktemp)" RELEVANT="$(mktemp)" trap 'rm -f "$CHANGED" "$RELEVANT"' EXIT git -c core.quotePath=false diff --no-renames --name-only -z "$BASE" HEAD > "$CHANGED" -LC_ALL=C grep -zvE "$IGNORED" "$CHANGED" > "$RELEVANT" || [ $? -eq 1 ] +if [ -n "$SKIP" ]; then + LC_ALL=C grep -zvE "^($SKIP)" "$CHANGED" > "$RELEVANT" || [ $? -eq 1 ] +else + cp "$CHANGED" "$RELEVANT" +fi JOINED="$(IFS='|'; echo "${PATTERNS[*]}")" status=0 diff --git a/scripts/ci/draft-release.sh b/scripts/ci/draft-release.sh deleted file mode 100755 index 87eb77ba9c..0000000000 --- a/scripts/ci/draft-release.sh +++ /dev/null @@ -1,174 +0,0 @@ -#!/usr/bin/env bash -# -# Puts every asset of a release on a draft, for the caller to publish in one last step. Once -# release immutability is on, a published release can never gain, replace or lose an asset. -# -# Usage: draft-release.sh [--target ] [--prerelease true|false] <notes-file> <asset>... -# -# Writes id=<draft id> to $GITHUB_OUTPUT, or id= when <tag> is already published with these exact -# assets. Fails when <tag> is published with anything else. Without --target the tag must exist. - -set -euo pipefail - -usage() { - sed -n 's/^# Usage: //p' "$0" >&2 - exit 2 -} - -TARGET="" -PRERELEASE="false" -while [ $# -gt 0 ]; do - case "$1" in - --target) - [ $# -ge 2 ] || usage - TARGET="$2" - shift 2 - ;; - --prerelease) - [ $# -ge 2 ] || usage - PRERELEASE="$2" - shift 2 - ;; - -*) usage ;; - *) break ;; - esac -done -[ $# -ge 4 ] || usage -case "$PRERELEASE" in - true | false) ;; - *) usage ;; -esac - -TAG="$1" -TITLE="$2" -NOTES="$3" -shift 3 - -for file in "$NOTES" "$@"; do - if [ ! -f "$file" ]; then - echo "::error::$file does not exist" - exit 1 - fi -done - -export GH_REPO="${GH_REPO:-${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is not set}}" -OUTPUT="${GITHUB_OUTPUT:-/dev/stdout}" -WORK="$(mktemp -d)" -trap 'rm -rf "$WORK"' EXIT - -sha256() { - shasum -a 256 "$1" | cut -d' ' -f1 -} - -# Only asked once no published release holds the tag: gh races its published and draft lookups. -draft_id() { - if gh release view "$TAG" --json databaseId,isDraft --jq 'select(.isDraft) | .databaseId' 2> "$WORK/view.err"; then - return 0 - fi - if grep -q "release not found" "$WORK/view.err"; then - return 0 - fi - cat "$WORK/view.err" >&2 - return 1 -} - -# 0 when release $1 holds every later argument under its own name, byte for byte; 1 when it does -# not; 2 when the release could not be read. -holds_assets() { - local id="$1" file name row state digest - shift - gh api "repos/$GH_REPO/releases/$id" \ - --jq '.assets[] | [.name, .state, (.digest // "")] | @tsv' > "$WORK/assets.tsv" || return 2 - for file in "$@"; do - name="$(basename "$file")" - row="$(awk -F'\t' -v name="$name" '$1 == name { print; exit }' "$WORK/assets.tsv")" - if [ -z "$row" ]; then - echo "$TAG has no $name" - return 1 - fi - IFS=$'\t' read -r _ state digest <<< "$row" - if [ "$state" != "uploaded" ]; then - echo "$name on $TAG is $state" - return 1 - fi - # Assets uploaded before GitHub recorded digests carry none. - if [ -z "$digest" ]; then - rm -rf "$WORK/download" - gh release download "$TAG" --pattern "$name" --dir "$WORK/download" || return 2 - digest="sha256:$(sha256 "$WORK/download/$name")" - fi - if [ "$digest" != "sha256:$(sha256 "$file")" ]; then - echo "$name on $TAG is not the file built here" - return 1 - fi - done -} - -if ! PUBLISHED="$(gh api "repos/$GH_REPO/releases/tags/$TAG" --jq .id 2> "$WORK/tag.err")"; then - if ! grep -q "HTTP 404" "$WORK/tag.err"; then - cat "$WORK/tag.err" >&2 - exit 1 - fi - PUBLISHED="" -fi - -if [ -n "$PUBLISHED" ]; then - STATUS=0 - holds_assets "$PUBLISHED" "$@" || STATUS=$? - case "$STATUS" in - 0) - echo "$TAG is already published with these assets" - echo "id=" >> "$OUTPUT" - exit 0 - ;; - 1) - echo "::error::$TAG is already published with other assets, and a published release cannot change them. Ship this build under a new tag." - exit 1 - ;; - *) exit 1 ;; - esac -fi - -DRAFT="$(draft_id)" -if [ -n "$DRAFT" ]; then - # Reused, never deleted: if someone publishes it meanwhile, GitHub refuses the asset upload, - # while a delete would take the published release with it. - echo "Reusing draft $DRAFT, left for $TAG by an earlier attempt" - EDIT=(-f "name=$TITLE" -F "body=@$NOTES" -F "prerelease=$PRERELEASE") - if [ -n "$TARGET" ]; then - EDIT+=(-f "target_commitish=$TARGET") - fi - gh api --method PATCH "repos/$GH_REPO/releases/$DRAFT" "${EDIT[@]}" > /dev/null - gh release upload "$TAG" "$@" --clobber -else - FLAGS=(--draft --title "$TITLE" --notes-file "$NOTES") - if [ -n "$TARGET" ]; then - FLAGS+=(--target "$TARGET") - else - FLAGS+=(--verify-tag) - fi - if [ "$PRERELEASE" = "true" ]; then - FLAGS+=(--prerelease) - fi - gh release create "$TAG" "${FLAGS[@]}" "$@" - - for _ in 1 2 3 4 5; do - DRAFT="$(draft_id)" - [ -z "$DRAFT" ] || break - sleep 2 - done - if [ -z "$DRAFT" ]; then - echo "::error::the draft for $TAG was created but cannot be found" - exit 1 - fi -fi - -STATUS=0 -holds_assets "$DRAFT" "$@" || STATUS=$? -if [ "$STATUS" -ne 0 ]; then - echo "::error::draft $DRAFT for $TAG does not hold this build; it stays unpublished" - exit 1 -fi - -echo "Draft $DRAFT holds every asset for $TAG" -echo "id=$DRAFT" >> "$OUTPUT" diff --git a/scripts/ci/extract-release-notes.py b/scripts/ci/extract-release-notes.py deleted file mode 100644 index 4b0036576f..0000000000 --- a/scripts/ci/extract-release-notes.py +++ /dev/null @@ -1,129 +0,0 @@ -#!/usr/bin/env python3 -"""Keep a release's complete Markdown, with its sections in Keep a Changelog order. - -With --highlights-only, emit just the lead block: the lines a version section carries before its -first `### ` heading. That is what the update window and the What's New surface read, because the -full section is not something anyone reads in a dialog. 0.73.0's appcast item ran 22,443 bytes and -231 list items, and the feed grows by about that much per release for every install that polls it. - -The lead block is optional. When a version has none, this falls back to the full notes rather than -failing: the release job runs under `set -euo pipefail` about forty minutes in, after both -notarized builds, and a forgotten heading is not worth losing that to. -""" - -import argparse -import re -import sys -from pathlib import Path - -SECTION_ORDER = { - "added": 1, - "features": 1, - "new features": 1, - "changed": 2, - "performance": 2, - "deprecated": 3, - "removed": 4, - "fixed": 5, - "security": 6, -} -UNKNOWN_SECTION_ORDER = 7 - -MAX_HIGHLIGHT_LINES = 6 - - -def extract_sections(changelog, version): - """Every section of one version, as (priority, lines). Index 0 is the lead block.""" - sections = [(0, [])] - found = False - fence = None - for line in changelog.splitlines(keepends=True): - if fence is not None: - if re.fullmatch(r" {0,3}" + re.escape(fence[0]) + "{" + str(len(fence)) + r",}\s*", line): - fence = None - if found: - sections[-1][1].append(line) - continue - - opening_fence = re.match(r"^ {0,3}(`{3,}|~{3,})", line) - if opening_fence: - fence = opening_fence[1] - if found: - sections[-1][1].append(line) - continue - - release = re.match(r"^## \[([^]]+)\]", line) - if release: - if found: - break - found = release[1] == version - continue - - if found: - heading = re.match(r"^### (.+?)\s*$", line) - if heading: - sections.append((SECTION_ORDER.get(heading[1].casefold(), UNKNOWN_SECTION_ORDER), [])) - sections[-1][1].append(line) - return sections - - -def extract_notes(changelog, version): - # Keep each section intact, including nested lists, code, and unknown headings. - # The sort is stable, so sections at the same priority keep their original order. - sections = extract_sections(changelog, version) - blocks = ["".join(lines).strip("\n") for _, lines in sorted(sections, key=lambda section: section[0])] - notes = "\n\n".join(block for block in blocks if block.strip()) - if not notes.strip(): - raise ValueError(f"No release notes found for version {version} in CHANGELOG.md") - return notes + "\n" - - -def extract_highlights(changelog, version, require_lead=False): - """The lead block, capped at MAX_HIGHLIGHT_LINES, as one Markdown paragraph per line. - - Falls back to the full notes when a version has none, unless `require_lead`. The feed wants - the fallback, because failing a release forty minutes in over a missing heading is worse than - a long dialog. A file shipped inside the app bundle wants the opposite: 270 entries is not - something to compile into the product. - """ - sections = extract_sections(changelog, version) - lead = [line.rstrip("\r\n") for line in sections[0][1] if line.strip()] - if not lead: - if require_lead: - raise ValueError(f"Version {version} has no lead block in CHANGELOG.md") - return extract_notes(changelog, version) - return "\n\n".join(lead[:MAX_HIGHLIGHT_LINES]) + "\n" - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("version") - parser.add_argument( - "--highlights-only", - action="store_true", - help="emit the lead block instead of every section, falling back to the full notes", - ) - parser.add_argument( - "--require-lead-block", - action="store_true", - help="with --highlights-only, fail instead of falling back to the full notes", - ) - parser.add_argument("--out", default="release_notes.md", help="where to write (default: release_notes.md)") - args = parser.parse_args(argv) - - try: - changelog = Path("CHANGELOG.md").read_text(encoding="utf-8") - notes = extract_highlights(changelog, args.version, require_lead=args.require_lead_block) \ - if args.highlights_only else extract_notes(changelog, args.version) - Path(args.out).write_text(notes, encoding="utf-8") - except (OSError, ValueError) as error: - sys.exit(f"ERROR: {error}") - - label = "Release highlights" if args.highlights_only else "Release notes" - print(f"{label} extracted for {args.version} into {args.out}:") - print(notes, end="") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/extract-release-notes.sh b/scripts/ci/extract-release-notes.sh deleted file mode 100755 index 77a908ec62..0000000000 --- a/scripts/ci/extract-release-notes.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Thin wrapper so callers do not repeat the python3 invocation. Every argument after the version -# is passed straight through, which is how sign-and-appcast.sh asks for --highlights-only. - -VERSION="${1:?Usage: extract-release-notes.sh <version> [--highlights-only] [--out PATH]}" -shift - -python3 "$(dirname "$0")/extract-release-notes.py" "$VERSION" "$@" diff --git a/scripts/ci/merge-appcast.py b/scripts/ci/merge-appcast.py deleted file mode 100755 index f1f6c30741..0000000000 --- a/scripts/ci/merge-appcast.py +++ /dev/null @@ -1,229 +0,0 @@ -#!/usr/bin/env python3 -"""Splices one release's two architecture items into the published Sparkle feed. - -`generate_appcast` keeps only what it is handed, so the release used to hand it the whole -published feed as a seed. That is safe only while the staging directory holds nothing but the new -archive, and it stops being true as soon as anything else is staged. Verified in the pinned -Sparkle 2.9.5 source: `FeedXML.swift:333-339` matches an existing item by version, and two writes then run outside the resulting `if createNewItem` guard. Line 603 -assigns the enclosure's attributes unconditionally, so a seeded item matched by a staged archive -has its download URL rewritten into the current release's directory, which is a permanent 404. -Lines 523-525 remove that item's `<description>` when the staged archive has no sibling notes -file. Neither is caught by the rewind guard in build.yml, which compares only version strings. - -So generate_appcast now runs against a directory holding one release's archive and nothing else, -and this script merges its output into the feed. Items already published are never handed to a -generator at all; see appcast_feed for how they are moved. - -This replaces an awk state machine that scanned for `<item>` and `</item>` line by line, on three -untested assumptions, one of which was that no release note ever contains the literal `</item>`. - -Nothing pruned the feed, so it grew by two items per release forever: 155 items and 695 KB by -0.74.0, of which 84% was embedded release-note CDATA and 92% was items no install can ever be -offered. Every install downloads the whole file on every scheduled check, daily by default. -`--keep-releases` bounds it. Dropping an old item is safe because Sparkle picks the highest version -present (`SUAppcastDriver.bestItemFromAppcastItems`) and never looks for the host's own version in -the feed; what it cannot survive is losing the newest item, which is what the caller's rewind guard -still checks. - -Run: python3 scripts/ci/merge-appcast.py --base appcast.xml --version 0.75.0 \ - --arm64 <dir>/appcast.xml --x86-64 <dir>/appcast.xml \ - --download-prefix https://.../download/v0.75.0/ --out appcast/appcast.xml -""" - -import argparse -import pathlib -import sys - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -from appcast_feed import ( # noqa: E402 - ED_SIGNATURE, - HARDWARE, - FeedError, - channel_items, - insert_items, - item_bundle_version, - item_enclosures, - item_short_version, - parse, - parse_text, - read, - remove_version, - sole_item_text, -) - -DEFAULT_KEEP_RELEASES = 5 - -MergeError = FeedError - - -def sole_item(path, version): - items = channel_items(parse(path), path) - matching = [item for item in items if item_short_version(item) == version] - if len(items) != 1 or len(matching) != 1: - raise FeedError( - f"{path}: expected exactly one item for {version}, " - f"found {len(items)} item(s) of which {len(matching)} match" - ) - return matching[0] - - -def check_new_items(arm64_item, x86_64_item, version, download_prefix): - if arm64_item.find(HARDWARE) is None: - raise FeedError( - f"the arm64 item for {version} carries no <sparkle:hardwareRequirements>; " - "every install would be offered the Apple Silicon build" - ) - hardware = (arm64_item.find(HARDWARE).text or "").strip() - if hardware != "arm64": - raise FeedError(f"the arm64 item declares hardwareRequirements '{hardware}', expected 'arm64'") - if x86_64_item.find(HARDWARE) is not None: - raise FeedError(f"the x86_64 item for {version} carries <sparkle:hardwareRequirements>") - - arm64_bundle = item_bundle_version(arm64_item) - x86_64_bundle = item_bundle_version(x86_64_item) - if not arm64_bundle or arm64_bundle != x86_64_bundle: - raise FeedError( - f"the two architectures disagree on <sparkle:version>: " - f"arm64 {arm64_bundle!r}, x86_64 {x86_64_bundle!r}" - ) - - for label, item in (("arm64", arm64_item), ("x86_64", x86_64_item)): - enclosures = item_enclosures(item) - if not enclosures: - raise FeedError(f"the {label} item for {version} advertises no enclosure") - for enclosure in enclosures: - url = enclosure.get("url", "") - if not url.startswith(download_prefix): - raise FeedError( - f"the {label} item advertises {url!r}, which is not under {download_prefix!r}" - ) - # generate_appcast signs an archive only when the app's own Info.plist declares - # SUPublicEDKey (Appcast.swift:199 in 2.9.5). Drop that key and it writes an unsigned - # enclosure, reports "Wrote 1 new update" and exits 0, and every installed updater - # then refuses the download with nothing said anywhere. Measured against the pinned - # 2.9.5 generate_appcast with a bundle that omits the key. - if not enclosure.get(ED_SIGNATURE): - raise FeedError( - f"the {label} enclosure {url!r} carries no sparkle:edSignature; " - "check that TablePro/Info.plist still declares SUPublicEDKey" - ) - return arm64_bundle - - -def check_base(base_root, base_path, version, bundle_version): - for item in channel_items(base_root, base_path): - if item_short_version(item) == version: - raise FeedError(f"{base_path} already advertises {version}") - existing = item_bundle_version(item) - if existing and int(existing) >= int(bundle_version): - raise FeedError( - f"{base_path} advertises build {existing}, which is not below the new build " - f"{bundle_version}; Sparkle compares builds, so the release would not be offered" - ) - - -def releases_in_order(text, path): - """Every distinct shortVersionString, newest first, in document order.""" - ordered = [] - for item in channel_items(parse_text(text), path): - version = item_short_version(item) - if version and version not in ordered: - ordered.append(version) - return ordered - - -def prune(text, path, keep_releases): - """Drops every item belonging to a release outside the newest `keep_releases`.""" - if keep_releases <= 0: - return text, [] - ordered = releases_in_order(text, path) - doomed = ordered[keep_releases:] - result = text - for version in doomed: - result, removed = remove_version(result, path, version) - if removed == 0: - raise FeedError(f"{path}: {version} was listed for pruning but no item carries it") - return result, doomed - - -def merge(base_path, arm64_path, x86_64_path, version, download_prefix, out_path, keep_releases): - base_text = read(base_path) - base_root = parse(base_path) - - arm64_item = sole_item(arm64_path, version) - x86_64_item = sole_item(x86_64_path, version) - bundle_version = check_new_items(arm64_item, x86_64_item, version, download_prefix) - try: - int(bundle_version) - except ValueError as error: - raise FeedError(f"<sparkle:version> {bundle_version!r} is not an integer build number") from error - check_base(base_root, base_path, version, bundle_version) - - # arm64 first: Sparkle offers the first item the host can run, so an x86_64 item ahead of it - # would hand every Apple Silicon Mac the Intel build. - insert = sole_item_text(arm64_path, version) + sole_item_text(x86_64_path, version) - merged = insert_items(base_text, base_path, insert) - - merged_items = channel_items(parse_text(merged), "merged feed") - if len(merged_items) != len(channel_items(base_root, base_path)) + 2: - raise FeedError("the merged feed does not hold exactly two more items than the base") - if [item_short_version(item) for item in merged_items[:2]] != [version, version]: - raise FeedError("the two new items are not the first two in the merged feed") - if merged_items[0].find(HARDWARE) is None: - raise FeedError("the arm64 item is not first; Sparkle offers the first item the host can run") - - merged, dropped = prune(merged, "merged feed", keep_releases) - if dropped: - pruned_items = channel_items(parse_text(merged), "pruned feed") - if not pruned_items: - raise FeedError("pruning emptied the feed") - if item_short_version(pruned_items[0]) != version: - raise FeedError(f"pruning left {item_short_version(pruned_items[0])} newest instead of {version}") - if pruned_items[0].find(HARDWARE) is None: - raise FeedError("pruning left a non-arm64 item first") - kept = releases_in_order(merged, "pruned feed") - if len(kept) != keep_releases: - raise FeedError(f"pruning left {len(kept)} releases, expected {keep_releases}") - print(f"pruned {len(dropped)} release(s) below the newest {keep_releases}: {', '.join(dropped)}") - - pathlib.Path(out_path).parent.mkdir(parents=True, exist_ok=True) - pathlib.Path(out_path).write_text(merged, encoding="utf-8") - return merged - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--base", required=True, help="the published feed to splice into") - parser.add_argument("--version", required=True, help="marketing version being released") - parser.add_argument("--arm64", required=True, help="generate_appcast output for the arm64 archive") - parser.add_argument("--x86-64", required=True, dest="x86_64", help="generate_appcast output for x86_64") - parser.add_argument("--download-prefix", required=True, help="every enclosure URL must start with this") - parser.add_argument("--out", required=True, help="where to write the merged feed") - parser.add_argument( - "--keep-releases", - type=int, - default=DEFAULT_KEEP_RELEASES, - help=f"how many releases the published feed keeps, newest first (default: {DEFAULT_KEEP_RELEASES}; 0 keeps all)", - ) - args = parser.parse_args(argv) - - try: - merge( - args.base, - args.arm64, - args.x86_64, - args.version, - args.download_prefix, - args.out, - args.keep_releases, - ) - except FeedError as error: - print(f"::error::merge-appcast: {error}", file=sys.stderr) - return 1 - print(f"merged {args.version} into {args.base} -> {args.out}") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/notify-telegram.sh b/scripts/ci/notify-telegram.sh deleted file mode 100755 index e7de51b66c..0000000000 --- a/scripts/ci/notify-telegram.sh +++ /dev/null @@ -1,71 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -VERSION="${1:?Usage: notify-telegram.sh <version>}" - -if [ -z "${TELEGRAM_BOT_TOKEN:-}" ]; then - echo "❌ ERROR: TELEGRAM_BOT_TOKEN environment variable is not set" - exit 1 -fi - -if [ -z "${TELEGRAM_CHAT_ID:-}" ]; then - echo "❌ ERROR: TELEGRAM_CHAT_ID environment variable is not set" - exit 1 -fi - -RELEASE_URL="https://github.com/TableProApp/TablePro/releases/tag/v${VERSION}" -NOTES=$(cat release_notes.md 2>/dev/null || echo "Bug fixes and improvements") - -ESCAPED=$(echo "$NOTES" | sed -e 's/&/\&/g' -e 's/</\</g' -e 's/>/\>/g') - -FORMATTED=$(echo "$ESCAPED" | sed -E \ - -e 's/^### (.+)$/<b>\1<\/b>/' \ - -e 's/^- /• /' \ - -e 's/`([^`]+)`/<code>\1<\/code>/g' \ - -e '/^[[:space:]]*$/d') - -MESSAGE_LIMIT=4096 -TRUNCATION_NOTICE="• Read the rest in the full release notes." - -build_message() { - printf '<b>TablePro v%s Released</b>\n\n%s\n\n<a href="%s">View Release</a>' "$VERSION" "$1" "$RELEASE_URL" -} - -message_length() { - printf '%s' "$1" | wc -m | tr -d ' ' -} - -TEXT=$(build_message "$FORMATTED") - -# Telegram rejects a message over MESSAGE_LIMIT outright, so keep whole entries -# until the budget runs out and point at the release for the remainder. -if [ "$(message_length "$TEXT")" -gt "$MESSAGE_LIMIT" ]; then - OVERHEAD=$(( $(message_length "$TEXT") - $(message_length "$FORMATTED") )) - BUDGET=$(( MESSAGE_LIMIT - OVERHEAD - ${#TRUNCATION_NOTICE} - 1 )) - KEPT=$(printf '%s\n' "$FORMATTED" | awk -v budget="$BUDGET" ' - { if (used + length($0) + 1 > budget) exit - used += length($0) + 1 - print }') - FORMATTED=$(printf '%s\n%s' "$KEPT" "$TRUNCATION_NOTICE") - TEXT=$(build_message "$FORMATTED") - echo "Release notes exceeded ${MESSAGE_LIMIT} characters, trimmed to $(message_length "$TEXT")" -fi - -PAYLOAD=$(jq -n \ - --arg chat_id "$TELEGRAM_CHAT_ID" \ - --arg text "$TEXT" \ - --arg topic_id "${TELEGRAM_TOPIC_ID:-}" \ - '{chat_id: $chat_id, text: $text, parse_mode: "HTML", disable_web_page_preview: true} - + (if $topic_id != "" then {message_thread_id: ($topic_id | tonumber)} else {} end)') - -RESPONSE=$(curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \ - -H "Content-Type: application/json" \ - -d "$PAYLOAD") - -if echo "$RESPONSE" | jq -e '.ok == true' > /dev/null; then - echo "Telegram notification sent for v${VERSION}" -else - echo "Telegram API rejected the message:" - echo "$RESPONSE" | jq . - exit 1 -fi diff --git a/scripts/ci/package-artifacts.sh b/scripts/ci/package-artifacts.sh deleted file mode 100755 index 2021c5ab6b..0000000000 --- a/scripts/ci/package-artifacts.sh +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -ARCH="${1:?Usage: package-artifacts.sh <arch> [staging_dir]}" -STAGING="${2:-}" - -if [[ "$ARCH" != "arm64" && "$ARCH" != "x86_64" ]]; then - echo "❌ ERROR: Invalid architecture: $ARCH (expected arm64 or x86_64)" - exit 1 -fi - -# Read from the bundle being packaged rather than inferred from git. `git describe --tags` picks -# up whatever tag is nearest, which on a release commit that also carries a plugin tag is the -# wrong one, and its "dev" fallback shipped that name to a public release rather than failing. -# Taken from the artifact, the DMG name cannot disagree with the app inside it. -APP_BUNDLE="build/Release/TablePro-${ARCH}.app" -VERSION=$(plutil -extract CFBundleShortVersionString raw -o - "$APP_BUNDLE/Contents/Info.plist") - -# --- Create DMG --- -echo "Creating DMG installer..." - -# create-dmg is pre-installed in CI dependencies step -if ! command -v create-dmg &>/dev/null; then - echo "📦 Installing create-dmg tool..." - brew install create-dmg -fi - -chmod +x scripts/create-dmg.sh - -echo "📌 Using version: $VERSION" -NOTARIZE="${NOTARIZE:-false}" scripts/create-dmg.sh "$VERSION" "$ARCH" "$APP_BUNDLE" - -# Verify DMG was created. A DMG under any other name is not the one the release will upload, so -# accepting it here only moved the failure to somewhere it was harder to read. -DMG_FILE="build/Release/TablePro-${VERSION}-${ARCH}.dmg" -if [ ! -f "$DMG_FILE" ]; then - echo "❌ ERROR: expected DMG not found at: $DMG_FILE" >&2 - ls -la build/Release/*.dmg 2>/dev/null || true - exit 1 -fi -echo "✅ DMG installer created successfully: $DMG_FILE" - -ls -lh build/Release/*.dmg - -# --- Create the Sparkle update archive --- -echo "Creating the update archive..." - -cd build/Release - -# tar keeps the framework symlinks, which codesign seals. A signed bundle needs no extended -# attributes, and the build machine's owner means nothing on a user's Mac. -if ! /usr/bin/tar --no-mac-metadata --no-xattrs --no-acls --uid 0 --gid 0 \ - --options xz:compression-level=9 -cJf "TablePro-${ARCH}.tar.xz" "TablePro-${ARCH}.app"; then - echo "❌ ERROR: Failed to create the update archive" - exit 1 -fi - -echo "✅ Update archive created" -ls -lh "TablePro-${ARCH}.tar.xz" - -cd - > /dev/null - -# --- Stage artifacts (optional, for local/self-hosted use) --- -if [ -n "$STAGING" ]; then - mkdir -p "$STAGING" - cp build/Release/*.dmg "$STAGING/" 2>/dev/null || true - cp "build/Release/TablePro-${ARCH}.tar.xz" "$STAGING/" 2>/dev/null || true - echo "Artifacts staged to $STAGING" - ls -lh "$STAGING" -fi diff --git a/scripts/ci/plugin-info.py b/scripts/ci/plugin-info.py deleted file mode 100755 index 91217a123a..0000000000 --- a/scripts/ci/plugin-info.py +++ /dev/null @@ -1,77 +0,0 @@ -#!/usr/bin/env python3 -"""Resolves a plugin tag to the release metadata the workflow needs. - -Replaces a 120-line `case` block in build-plugin.yml. That block was shell holding data, which is -how seven of its icon values drifted away from the plugin classes they describe without anything -noticing. - -Usage: plugin-info.py <tag> prints KEY=VALUE lines for $GITHUB_OUTPUT - plugin-info.py --list-slugs prints every publishable slug, one per line -""" - -import json -import pathlib -import re -import sys - -REPO_ROOT = pathlib.Path(__file__).resolve().parents[2] -MANIFEST = REPO_ROOT / ".github" / "plugin-registry.json" -VERSION_XCCONFIG = REPO_ROOT / "Configs" / "Version.xcconfig" -TAG_PATTERN = re.compile(r"^plugin-([a-z0-9-]+)-v([0-9]+\.[0-9]+\.[0-9]+)$") - - -def load_plugins(): - with open(MANIFEST, encoding="utf-8") as handle: - return json.load(handle)["plugins"] - - -def marketing_version(): - """Configs/Version.xcconfig is the single declaration of the app version.""" - for line in VERSION_XCCONFIG.read_text(encoding="utf-8").splitlines(): - if line.startswith("MARKETING_VERSION"): - return line.split("=", 1)[1].strip() - sys.exit(f"MARKETING_VERSION missing from {VERSION_XCCONFIG}") - - -def main(): - if len(sys.argv) != 2: - sys.exit(__doc__) - - plugins = load_plugins() - - if sys.argv[1] == "--list-slugs": - print("\n".join(sorted(plugins))) - return - - match = TAG_PATTERN.match(sys.argv[1]) - if not match: - sys.exit(f"::error::Malformed plugin tag: {sys.argv[1]}") - slug, version = match.groups() - - plugin = plugins.get(slug) - if plugin is None: - known = ", ".join(sorted(plugins)) - sys.exit(f"::error::Unknown plugin '{slug}'. {MANIFEST.name} knows: {known}") - - ids = plugin["databaseTypeIds"] - fields = { - "target": plugin["target"], - "bundleId": plugin["bundleId"], - "displayName": plugin["displayName"], - "summary": plugin["summary"], - "dbTypeIds": "null" if ids is None else json.dumps(ids, separators=(",", ":")), - "icon": plugin["icon"], - "bundleName": plugin["bundleName"], - "category": plugin["category"], - "homepage": plugin["homepage"], - "version": version, - "minAppVersion": marketing_version(), - } - for key, value in fields.items(): - if "\n" in value: - sys.exit(f"::error::{slug}.{key} contains a newline, which $GITHUB_OUTPUT cannot carry") - print(f"{key}={value}") - - -if __name__ == "__main__": - main() diff --git a/scripts/ci/prepare-libs.sh b/scripts/ci/prepare-libs.sh deleted file mode 100755 index df0b4c083e..0000000000 --- a/scripts/ci/prepare-libs.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Selects the per-architecture slice of every vendored static library, so an Xcode build links the -# right one. Called by the release workflow before scripts/build-release.sh. -# -# The selection lives in scripts/lib/macos.sh and is shared with build-release.sh, which used to -# carry four near-identical copies of it. This script did a bare `cp` of the per-architecture file -# with no fallback, so the two took different sources for the same library. - -# shellcheck source=../lib/macos.sh -source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/lib/macos.sh" - -ARCH="${1:-}" -if [[ "$ARCH" != "arm64" && "$ARCH" != "x86_64" ]]; then - echo "Usage: $0 <arm64|x86_64>" >&2 - exit 1 -fi - -echo "📦 Preparing static libraries for $ARCH..." -prepare_arch_libs "$ARCH" \ - libmariadb \ - libpq libpgcommon libpgport libssl libcrypto \ - libmongoc libbson \ - libhiredis libhiredis_ssl \ - libsqlite3_vendored -echo "✅ Static libraries ready for $ARCH" diff --git a/scripts/ci/pull-release.py b/scripts/ci/pull-release.py deleted file mode 100755 index 4127a1154e..0000000000 --- a/scripts/ci/pull-release.py +++ /dev/null @@ -1,158 +0,0 @@ -#!/usr/bin/env python3 -"""Withdraws a released version from the Sparkle feed. - -Removing an item from appcast.xml is how a Sparkle release is withdrawn. There is no API for it -and no other mechanism, so the only question is whether the operator has a rehearsed tool or edits -a 640 KB XML file by hand under time pressure. - -This became necessary the moment updates started installing without being accepted first. Before -that a bad build was contained by the user, who saw a dialog and could decline it. Phased rollout -bounds the first wave, but it does not stop the cohorts behind it. - -The GitHub Release is deliberately left in place: people who downloaded the DMG directly still -need their link to resolve, and the corrective release is what supersedes the build. Only the feed -entry goes, so no updater offers it again. - -The items are removed with the same primitives merge-appcast.py uses to insert them, so the two -cannot disagree about where an item begins and ends. - -The push target is checked rather than assumed. SUFeedURL names main, so a withdrawal pushed to -any other branch leaves every install downloading the bad build while this script reports success. -`git push origin HEAD` from one of this repo's worktrees did exactly that. - -Run: python3 scripts/ci/pull-release.py 0.75.0 [--dry-run] [--feed appcast.xml] [--no-push] -""" - -import argparse -import pathlib -import subprocess -import sys - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -from appcast_feed import ( # noqa: E402 - FeedError, - channel_items, - item_short_version, - parse_text, - read, - remove_version, -) - - -PUBLISHED_BRANCH = "main" - - -def git(*arguments, check=True): - return subprocess.run(["git", *arguments], check=check, capture_output=True, text=True) - - -def checked_out_branch(): - """The current branch name, or None on a detached HEAD.""" - result = git("symbolic-ref", "--quiet", "--short", "HEAD", check=False) - return result.stdout.strip() or None - - -def require_publishable_checkout(): - """The feed every install polls is `main`, so nothing else may be pushed as a withdrawal.""" - branch = checked_out_branch() - if branch is None: - raise FeedError( - "HEAD is detached, so there is no branch to push. Check out " - f"{PUBLISHED_BRANCH} and run this again" - ) - if branch != PUBLISHED_BRANCH: - raise FeedError( - f"on branch {branch!r}, but SUFeedURL serves {PUBLISHED_BRANCH}. Pushing here would " - f"report the build withdrawn while every install kept downloading it. Check out " - f"{PUBLISHED_BRANCH} and run this again" - ) - return branch - - -def require_in_step_with_origin(): - """The release key skips every check, so the push may carry the withdrawal and nothing else.""" - fetch = git("fetch", "origin", PUBLISHED_BRANCH, check=False) - if fetch.returncode != 0: - raise FeedError(f"could not fetch origin/{PUBLISHED_BRANCH}: {fetch.stderr.strip()}") - head = git("rev-parse", "HEAD").stdout.strip() - published = git("rev-parse", "FETCH_HEAD").stdout.strip() - if head != published: - raise FeedError( - f"local {PUBLISHED_BRANCH} is not origin/{PUBLISHED_BRANCH}, so the push would carry other " - f"commits past every check or be refused. Make {PUBLISHED_BRANCH} match " - f"origin/{PUBLISHED_BRANCH} and run this again" - ) - - -def withdraw(feed_path, version): - """Returns (new_text, removed_count). Raises FeedError when the result would be wrong.""" - text = read(feed_path) - before = channel_items(parse_text(text), feed_path) - if not any(item_short_version(item) == version for item in before): - raise FeedError(f"{feed_path} does not advertise {version}, so there is nothing to withdraw") - - result, removed = remove_version(text, feed_path, version) - after = channel_items(parse_text(result), feed_path) - if len(after) != len(before) - removed: - raise FeedError("removing the items changed the number of other items in the feed") - if any(item_short_version(item) == version for item in after): - raise FeedError(f"{version} is still advertised after the removal") - if not after: - raise FeedError( - f"withdrawing {version} would empty the feed, and an empty feed tells every install " - "it is up to date forever" - ) - return result, removed - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("version", help="the marketing version to withdraw, without the leading v") - parser.add_argument("--feed", default="appcast.xml", help="path to the feed (default: appcast.xml)") - parser.add_argument("--dry-run", action="store_true", help="report what would change, write nothing") - parser.add_argument("--no-push", action="store_true", help="commit but do not push") - args = parser.parse_args(argv) - - try: - if not args.dry_run and not args.no_push: - require_publishable_checkout() - require_in_step_with_origin() - result, removed = withdraw(args.feed, args.version) - except FeedError as error: - print(f"error: {error}", file=sys.stderr) - return 1 - - remaining = channel_items(parse_text(result), args.feed) - newest = item_short_version(remaining[0]) - print(f"{args.version}: {removed} item(s) to remove, {len(remaining)} left, newest becomes {newest}") - - if args.dry_run: - print("dry run, nothing written") - return 0 - - pathlib.Path(args.feed).write_text(result, encoding="utf-8") - - status = git("status", "--porcelain", "--", args.feed) - if not status.stdout.strip(): - print(f"{args.feed} is unchanged on disk, nothing to commit") - return 0 - - git("add", "--", args.feed) - git("commit", "-m", f"release: withdraw v{args.version} from the update feed") - if args.no_push: - print("committed, not pushed") - return 0 - - push = git("push", "origin", f"HEAD:{PUBLISHED_BRANCH}", check=False) - if push.returncode != 0: - print(push.stdout, file=sys.stderr) - print(push.stderr, file=sys.stderr) - print("error: the commit is local; push it before telling anyone the build is pulled", file=sys.stderr) - return 1 - print(f"withdrawn: v{args.version} is no longer offered to any install") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/sign-and-appcast.sh b/scripts/ci/sign-and-appcast.sh deleted file mode 100755 index 68cc314b89..0000000000 --- a/scripts/ci/sign-and-appcast.sh +++ /dev/null @@ -1,161 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Signs release archives and publishes them into appcast.xml using Sparkle's generate_appcast, -# the official tool for building Sparkle update feeds. -# -# generate_appcast only ever sees the release being published. It is never handed the existing -# feed, and it never rewrites an item it did not create. Everything already published is merged -# in afterwards by scripts/ci/merge-appcast.py, which splices text rather than re-serializing. -# The reason is in that script's docstring: two writes in Sparkle's FeedXML.swift run outside its -# `if createNewItem` guard, so a previously published item that a staged archive happens to match -# has its download URL rewritten and its release notes dropped. -# -# generate_appcast rejects two archives sharing a bundle version in one directory -# (Unarchive.swift:96, SUSparkleErrorDomain 1002, reproduced against 2.9.5 and 2.10.0), so each -# architecture gets its own staging directory and its own generate_appcast run. -# -# Usage: sign-and-appcast.sh <version> -# Requires: SPARKLE_PRIVATE_KEY env var, artifacts/ directory with both architectures' tar.xz -# archives. -# Optional: CRITICAL_UPDATE=1 to mark the release critical. - -BASE_APPCAST="${BASE_APPCAST:-appcast.xml}" -CRITICAL_UPDATE="${CRITICAL_UPDATE:-0}" -VERSION="${1:?Usage: sign-and-appcast.sh <version>}" - -if [ -z "${SPARKLE_PRIVATE_KEY:-}" ]; then - echo "::error::SPARKLE_PRIVATE_KEY environment variable is not set" - exit 1 -fi - -if [ ! -f "$BASE_APPCAST" ]; then - echo "::error::base appcast $BASE_APPCAST does not exist" - exit 1 -fi - -# --------------------------------------------------------------------------- -# 1. Extract the notes -# --------------------------------------------------------------------------- -# Two files, for two audiences. release_notes.md is the whole section and becomes the GitHub -# release body, where a reader has scrolled to it on purpose. release_highlights.md is the lead -# block and is what goes in the feed, because an appcast item is downloaded by every install on -# every check and read inside a dialog. 0.73.0's ran 22,443 bytes and 231 list items. -# -# A version with no lead block falls back to the full notes, so this changes nothing until the -# convention is used. -bash "$(dirname "$0")/extract-release-notes.sh" "$VERSION" -bash "$(dirname "$0")/extract-release-notes.sh" "$VERSION" --highlights-only --out release_highlights.md - -# --------------------------------------------------------------------------- -# 2. Locate Sparkle tools -# --------------------------------------------------------------------------- -# Pinned and checksum-verified rather than installed from a cask that tracks latest. This step -# holds the EdDSA private key that signs every update every user receives, so it should not run a -# binary whose contents can change between releases. The version matches the Sparkle framework -# pinned in Package.resolved, and check-sparkle-version.py fails Repo Hygiene when they drift. -SPARKLE_VERSION="2.10.0" -SPARKLE_SHA256="c2bf58aa8387266ac179357b1415d6f2635f044da8be41042af32425dae6da0c" -SPARKLE_DIR="$(mktemp -d)" -curl -sSLo "$SPARKLE_DIR/sparkle.tar.xz" \ - "https://github.com/sparkle-project/Sparkle/releases/download/$SPARKLE_VERSION/Sparkle-$SPARKLE_VERSION.tar.xz" -echo "$SPARKLE_SHA256 $SPARKLE_DIR/sparkle.tar.xz" | shasum -a 256 -c - -tar xf "$SPARKLE_DIR/sparkle.tar.xz" -C "$SPARKLE_DIR" -SPARKLE_BIN="$SPARKLE_DIR/bin" - -DOWNLOAD_PREFIX="${GITHUB_SERVER_URL:-https://github.com}/${GITHUB_REPOSITORY:-TableProApp/TablePro}/releases/download/v${VERSION}/" - -KEY_FILE=$(mktemp) -STAGING_ROOT=$(mktemp -d) -trap 'rm -rf "$KEY_FILE" "$STAGING_ROOT"' EXIT - -echo "$SPARKLE_PRIVATE_KEY" > "$KEY_FILE" - -# --------------------------------------------------------------------------- -# 3. Generate one item per architecture, each in its own directory -# --------------------------------------------------------------------------- -generate_item() { - local arch="$1" - local archive="artifacts/TablePro-${VERSION}-${arch}.tar.xz" - local staging="$STAGING_ROOT/$arch" - - if [ ! -f "$archive" ]; then - echo "::error::$archive is missing, so the $arch build produced no update archive" - return 1 - fi - - mkdir -p "$staging" - cp "$archive" "$staging/" - - # Sparkle 2.9 renders Markdown natively, including code and links. Feeding hand-built HTML - # left Markdown visible and interpreted literal SQL/XML angle brackets as HTML tags. - # - # The name matches the archive's without .tar.xz, which is how generate_appcast pairs notes to - # an archive. Without the pairing the item ships with no description at all. - { - printf "# What's New in TablePro %s\n\n" "$VERSION" - cat release_highlights.md - printf '\n[View full changelog](https://docs.tablepro.app/changelog)\n' - } > "${staging}/TablePro-${VERSION}-${arch}.md" - - # Sparkle bypasses phasing for a critical item anyway, but passing both would be a contradiction - # in the feed rather than a belt and braces. - local flags=() - if [ "$CRITICAL_UPDATE" = "1" ]; then - # An empty --critical-update-version writes <sparkle:criticalUpdate/> with no version - # attribute, which SPUAppcastItemStateResolver treats as critical for every host. - flags+=(--critical-update-version "") - else - # Seven cohorts, so the interval times six is the tail: 21600 puts the last one 36 hours - # behind, which fits inside the median gap between releases. A user-initiated check is never - # phased, so Check for Updates always offers the newest build. - flags+=(--phased-rollout-interval 21600) - fi - - # --maximum-versions 1 states the invariant merge-appcast.py checks rather than leaving it as a - # consequence of the directory holding one archive. - "$SPARKLE_BIN/generate_appcast" \ - --ed-key-file "$KEY_FILE" \ - --download-url-prefix "$DOWNLOAD_PREFIX" \ - --embed-release-notes \ - --full-release-notes-url "https://docs.tablepro.app/changelog" \ - --maximum-versions 1 \ - "${flags[@]}" \ - "$staging" - - if [ ! -f "$staging/appcast.xml" ]; then - echo "::error::generate_appcast produced no feed for $arch" - return 1 - fi -} - -for arch in arm64 x86_64; do - generate_item "$arch" -done - -# --------------------------------------------------------------------------- -# 4. Keep both signed items, and merge once here as a gate -# --------------------------------------------------------------------------- -# The items are kept because the merge that actually ships runs later, in the commit step, against -# whatever appcast.xml main holds at that moment. Merging only here meant merging into a snapshot -# taken minutes earlier and then copying the result over main, so a withdrawal pushed in between -# was silently reverted and the pulled build was offered again. -# -# This merge still runs, against the snapshot, because it is the cheapest place to refuse a release: -# it happens before the GitHub Release publishes any artifact. Every invariant worth checking lives -# in merge-appcast.py, which scripts/ci/test_merge_appcast.py covers on the Linux runner, because a -# real release is the only thing that runs this script. -mkdir -p appcast/items -cp "$STAGING_ROOT/arm64/appcast.xml" appcast/items/arm64.xml -cp "$STAGING_ROOT/x86_64/appcast.xml" appcast/items/x86_64.xml - -python3 "$(dirname "$0")/merge-appcast.py" \ - --base "$BASE_APPCAST" \ - --version "$VERSION" \ - --arm64 appcast/items/arm64.xml \ - --x86-64 appcast/items/x86_64.xml \ - --download-prefix "$DOWNLOAD_PREFIX" \ - --out appcast/appcast.xml - -echo "✅ Appcast validated for $VERSION:" -head -c 4000 appcast/appcast.xml diff --git a/scripts/ci/test_appcast_feed.py b/scripts/ci/test_appcast_feed.py deleted file mode 100644 index 76ffb10665..0000000000 --- a/scripts/ci/test_appcast_feed.py +++ /dev/null @@ -1,160 +0,0 @@ -#!/usr/bin/env python3 -"""Regression tests for the text primitives both feed scripts share. - -merge-appcast.py and pull-release.py agree about where an item begins and ends only because they -call the same scanner. When that scanner is wrong, one of them moves the wrong bytes into the file -every install polls, and the result is still well-formed XML, so nothing downstream notices. - -Run: python3 scripts/ci/test_appcast_feed.py -""" - -import pathlib -import sys -import unittest - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -import appcast_feed # noqa: E402 - - -def feed(*items): - body = "".join(items) - return ( - '<?xml version="1.0" standalone="yes"?>\n' - '<rss xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle" version="2.0">\n' - " <channel>\n" - " <title>TablePro\n" - f"{body}" - " \n" - "\n" - ) - - -def item(version, body=""): - return ( - " \n" - f" {version}\n" - f"{body}" - " \n" - ) - - -class ItemSpanTests(unittest.TestCase): - def spans(self, text): - return [text[start:end] for start, end in appcast_feed.item_spans(text)] - - def test_finds_each_item_whole(self): - text = feed(item("0.75.0"), item("0.74.0")) - found = self.spans(text) - self.assertEqual(len(found), 2) - self.assertTrue(found[0].startswith(" ")) - self.assertTrue(found[0].rstrip().endswith("")) - - def test_a_comment_holding_an_open_tag_does_not_swallow_the_next_item(self): - """`` used to open a span that ran on until the real item's close tag, so - the first real item was consumed by a comment and silently moved with it.""" - text = feed(" \n", item("0.75.0")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertIn("0.75.0", found[0]) - self.assertNotIn("a note", found[0]) - - def test_a_comment_holding_a_close_tag_does_not_truncate_an_item(self): - """A comment inside an item used to close it early, so the rest of the item stayed behind - when the item was removed.""" - text = feed(item("0.75.0", " \n Keep\n")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertIn("Keep", found[0]) - - def test_an_element_whose_name_merely_starts_with_item_is_not_an_item(self): - text = feed(" 3\n", item("0.75.0")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertNotIn("itemCount", found[0]) - - def test_cdata_holding_either_tag_is_skipped_whole(self): - body = " and in the notes]]>\n" - text = feed(item("0.75.0", body), item("0.74.0")) - found = self.spans(text) - self.assertEqual(len(found), 2) - self.assertIn("in the notes", found[0]) - self.assertIn("0.74.0", found[1]) - - def test_an_unterminated_comment_is_refused(self): - with self.assertRaises(appcast_feed.FeedError) as caught: - appcast_feed.item_spans(feed(" \n\n" +NEW = f"## [0.82.0]({LINK}/v0.82.0) - 2026-10-25\n\nOne lead line.\n\n### Added\n\n- A thing. ([#1](x))\n\n" +OLD = "## [0.81.0] - 2026-10-16\n\n### Fixed\n\n- A bug.\n\n## [0.80.0] - 2026-10-11\n\n### Added\n\n- Another.\n" + + +class TypesTests(unittest.TestCase): + def test_reads_the_fragment_types_and_not_the_directory(self): + self.assertEqual(check.fragment_types(TOWNCRIER), TYPES) + + +class FragmentTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.directory = Path(self.scratch.name, "changelog.d") + self.directory.mkdir() + (self.directory / ".gitkeep").write_text("", encoding="utf-8") + + def problems(self, name, text): + (self.directory / name).write_text(text, encoding="utf-8") + return check.fragment_problems(self.directory, TYPES) + + def test_a_one_line_fragment_passes(self): + self.assertEqual(self.problems("3446.added.md", "Generate UUID in Set Value.\n"), []) + + def test_counters_and_orphans_pass(self): + self.assertEqual(self.problems("3452.fixed.2.md", "A bug.\n"), []) + self.assertEqual(self.problems("+theme-import.changed.md", "A change.\n"), []) + + def test_an_empty_directory_and_a_missing_one_pass(self): + self.assertEqual(check.fragment_problems(self.directory, TYPES), []) + self.assertEqual(check.fragment_problems(self.directory / "missing", TYPES), []) + + def test_a_bad_name_is_refused(self): + for name in ("3446.feature.md", "3446.added.txt", "3446.added", "abc.added.md", "3446.md", + "3446.added.0.md", "+Theme.added.md"): + with self.subTest(name=name): + problems = self.problems(name, "A thing.\n") + (self.directory / name).unlink() + self.assertEqual(len(problems), 1) + self.assertIn("is not named", problems[0]) + + def test_more_than_one_line_is_refused(self): + self.assertIn("2 non-blank lines", self.problems("1.added.md", "One.\nTwo.\n")[0]) + + def test_a_blank_fragment_is_refused(self): + self.assertIn("0 non-blank lines", self.problems("1.added.md", "\n\n")[0]) + + def test_blank_lines_around_the_entry_pass(self): + self.assertEqual(self.problems("1.added.md", "\nA thing.\n\n"), []) + + def test_the_line_limit_is_120_characters(self): + self.assertEqual(self.problems("1.added.md", "x" * 120 + "\n"), []) + self.assertIn("121 characters", self.problems("1.added.md", "x" * 121 + "\n")[0]) + + def test_a_bullet_is_refused(self): + self.assertIn("`- `", self.problems("1.added.md", "- A thing.\n")[0]) + + def test_a_credit_is_refused(self): + self.assertIn("credit", self.problems("1.added.md", "A thing by @someone.\n")[0]) + + def test_a_directory_inside_is_refused(self): + (self.directory / "added").mkdir() + self.assertEqual(check.fragment_problems(self.directory, TYPES), ["changelog.d/added is not a file"]) + + +class ChangelogTests(unittest.TestCase): + def test_old_and_new_headings_pass(self): + self.assertEqual(check.changelog_problems(HEADER + NEW + OLD), []) + + def test_a_critical_new_heading_passes(self): + text = HEADER + NEW.replace("2026-10-25\n", "2026-10-25 [CRITICAL]\n", 1) + OLD + self.assertEqual(check.changelog_problems(text), []) + + def test_only_old_headings_pass(self): + self.assertEqual(check.changelog_problems(HEADER + OLD), []) + + def test_a_missing_or_repeated_marker_is_refused(self): + self.assertIn("0 `", check.changelog_problems(OLD)[0]) + self.assertIn("2 `", check.changelog_problems(HEADER + "\n" + OLD)[0]) + + def test_text_between_the_marker_and_the_first_version_is_refused(self): + text = HEADER + "- A stray entry.\n\n" + OLD + self.assertEqual(len(check.changelog_problems(text)), 1) + self.assertEqual(len(check.changelog_problems(HEADER + "\n" + OLD)), 1) + self.assertEqual(len(check.changelog_problems(HEADER.rstrip("\n") + "\n" + OLD)), 1) + + def test_a_heading_above_the_marker_is_refused(self): + text = "# Changelog\n\n## [Unreleased]\n\n" + HEADER.split("\n", 2)[2] + OLD + problems = check.changelog_problems(text) + self.assertEqual(len(problems), 1) + self.assertIn("above the marker", problems[0]) + + def test_a_heading_in_neither_grammar_is_refused(self): + for heading in ("## [Unreleased]", "## [0.82.0] 2026-10-25", "## [0.82.0-beta.1] - 2026-10-25", + f"## [0.82.0]({LINK}/v0.82.1) - 2026-10-25", + f"## [0.82.0](https://example.com/v0.82.0) - 2026-10-25", + "## [0.82.0] - 2026-10-25 [CRITICAL]"): + with self.subTest(heading=heading): + problems = check.changelog_problems(HEADER + heading + "\n\n" + OLD) + self.assertEqual(len(problems), 1) + self.assertIn("is not `## [X.Y.Z]", problems[0]) + + def test_a_heading_without_a_version_is_refused(self): + problems = check.changelog_problems(HEADER + OLD + "\n## Notes\n") + self.assertEqual(len(problems), 1) + self.assertIn("`## Notes` is not", problems[0]) + + def test_an_impossible_date_is_refused(self): + problems = check.changelog_problems(HEADER + OLD.replace("2026-10-16", "2026-02-30")) + self.assertEqual(problems, ["CHANGELOG.md:7: 2026-02-30 is not a date"]) + + def test_a_repeated_version_is_refused(self): + problems = check.changelog_problems(HEADER + OLD + "\n## [0.80.0] - 2026-10-01\n") + self.assertEqual(len(problems), 1) + self.assertIn("repeats", problems[0]) + + def test_versions_must_descend(self): + text = HEADER + "## [0.80.0] - 2026-10-11\n\n## [0.81.0] - 2026-10-16\n" + problems = check.changelog_problems(text) + self.assertEqual(len(problems), 1) + self.assertIn("not below", problems[0]) + + def test_versions_compare_as_numbers(self): + text = HEADER + "## [0.10.0] - 2026-10-11\n\n## [0.9.0] - 2026-10-01\n" + self.assertEqual(check.changelog_problems(text), []) + + def test_a_new_lead_block_takes_at_most_six_lines(self): + six = "\n".join(f"Lead {index}." for index in range(6)) + text = HEADER + NEW.replace("One lead line.", six) + OLD + self.assertEqual(check.changelog_problems(text), []) + problems = check.changelog_problems(HEADER + NEW.replace("One lead line.", six + "\nLead 6.") + OLD) + self.assertEqual(len(problems), 1) + self.assertIn("7 lines", problems[0]) + + def test_an_old_lead_block_is_not_limited(self): + lead = "\n".join(f"Lead {index}." for index in range(9)) + self.assertEqual(check.changelog_problems(HEADER + OLD.replace("### Fixed", lead + "\n\n### Fixed")), []) + + +class RepositoryTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.root = Path(self.scratch.name) + (self.root / "towncrier.toml").write_text(TOWNCRIER, encoding="utf-8") + (self.root / "CHANGELOG.md").write_text(HEADER + NEW + OLD, encoding="utf-8") + (self.root / "changelog.d").mkdir() + (self.root / "changelog.d" / "3446.added.md").write_text("A thing.\n", encoding="utf-8") + + def run_main(self, *argv): + captured = io.StringIO() + with redirect_stdout(captured): + status = check.main(["--root", str(self.root), *argv]) + return status, captured.getvalue() + + def test_a_good_tree_passes(self): + self.assertEqual(self.run_main(), (0, "OK: changelog.d/ and CHANGELOG.md\n")) + + def test_problems_from_both_files_are_annotations(self): + (self.root / "changelog.d" / "3446.added.md").write_text("- A thing.\n", encoding="utf-8") + (self.root / "CHANGELOG.md").write_text(HEADER + "## [Unreleased]\n\n" + OLD, encoding="utf-8") + status, out = self.run_main() + self.assertEqual(status, 1) + self.assertEqual(out.count("::error title=Changelog::"), 2) + + def test_a_config_with_no_types_fails(self): + (self.root / "towncrier.toml").write_text("[tool.towncrier]\n", encoding="utf-8") + status, out = self.run_main() + self.assertEqual(status, 1) + self.assertIn("no [[tool.towncrier.type]]", out) + + +def changed(filename, status="added", previous=None): + return check.ChangedFile(filename, status, previous) + + +class PullRequestTests(unittest.TestCase): + def problems(self, files, title="fix(editor): keep the caret", base="main", number=3600): + return check.pull_request_problems(number, title, base, files, TYPES) + + def test_a_fragment_named_for_the_pull_request_passes(self): + files = [changed("changelog.d/3600.fixed.md"), changed("changelog.d/3600.fixed.2.md"), + changed("TablePro/Editor.swift", "modified")] + self.assertEqual(self.problems(files), []) + + def test_editing_changelog_outside_a_release_names_the_fragment_to_add(self): + problems = self.problems([changed("CHANGELOG.md", "modified")]) + self.assertEqual(len(problems), 1) + self.assertIn("changelog.d/3600..md", problems[0]) + + def test_a_release_pull_request_edits_changelog(self): + files = [changed("CHANGELOG.md", "modified"), changed("changelog.d/3446.added.md", "removed")] + self.assertEqual(self.problems(files, "chore(release): v0.82.0", "release/0.82"), []) + self.assertEqual(self.problems(files, "chore(release): v0.82.1 (#3601)"), []) + + def test_titles_that_are_not_a_stable_release_cannot_edit_changelog(self): + for title in ("chore(release): v0.82.0-beta.1", "chore(release): 0.82.0", "chore: v0.82.0", + "chore(release): v0.82.0 and more", "chore(release): v0.82"): + with self.subTest(title=title): + self.assertEqual(len(self.problems([changed("CHANGELOG.md", "modified")], title)), 1) + + def test_renaming_changelog_away_counts_as_editing_it(self): + self.assertEqual(len(self.problems([changed("CHANGES.md", "renamed", "CHANGELOG.md")])), 1) + + def test_a_fragment_named_for_another_pull_request_prints_the_expected_name(self): + problems = self.problems([changed("changelog.d/3599.fixed.2.md")]) + self.assertEqual(len(problems), 1) + self.assertIn("Name it changelog.d/3600.fixed.2.md", problems[0]) + + def test_an_orphan_fragment_on_main_is_refused(self): + problems = self.problems([changed("changelog.d/+theme.fixed.md")]) + self.assertIn("Name it changelog.d/3600.fixed.md", problems[0]) + + def test_a_file_that_is_not_a_fragment_is_refused(self): + problems = self.problems([changed("changelog.d/notes.txt")]) + self.assertIn("Name it changelog.d/3600..md", problems[0]) + + def test_gitkeep_is_not_a_fragment(self): + self.assertEqual(self.problems([changed("changelog.d/.gitkeep")]), []) + + def test_the_pull_request_that_creates_the_directory_moves_the_old_entries(self): + files = [changed("changelog.d/.gitkeep"), changed("changelog.d/3446.fixed.md"), + changed("CHANGELOG.md", "modified")] + self.assertEqual(self.problems(files, "ci(release): release from tags through release.yml"), []) + self.assertEqual(len(self.problems(files[1:], "ci(release): release from tags through release.yml")), 2) + + def test_a_pick_into_a_release_line_keeps_the_original_name(self): + self.assertEqual(self.problems([changed("changelog.d/3446.fixed.md")], "fix: keep it (#3446)", + "release/0.82"), []) + + def test_editing_or_renaming_another_pull_requests_fragment_passes(self): + files = [changed("changelog.d/3446.fixed.md", "modified"), + changed("changelog.d/3452.changed.md", "renamed", "changelog.d/3452.fixed.md")] + self.assertEqual(self.problems(files), []) + + def test_retyping_another_pull_requests_fragment_keeps_its_number(self): + files = [changed("changelog.d/3446.fixed.md", "removed"), changed("changelog.d/3446.changed.md")] + self.assertEqual(self.problems(files), []) + + +class FakeRun: + def __init__(self, *results): + self.results = list(results) + self.commands = [] + + def __call__(self, command, **_): + self.commands.append(command) + return self.results.pop(0) + + +def completed(stdout="", returncode=0, stderr=""): + return subprocess.CompletedProcess([], returncode, stdout, stderr) + + +class FetchTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.root = Path(self.scratch.name) + (self.root / "towncrier.toml").write_text(TOWNCRIER, encoding="utf-8") + + def run_main(self, run): + captured = io.StringIO() + with redirect_stdout(captured): + status = check.main(["--pr", "3600", "--root", str(self.root)], run) + return status, captured.getvalue() + + def test_reads_the_title_base_and_every_page_of_files(self): + files = "\n".join(json.dumps(entry) for entry in ( + {"filename": "CHANGELOG.md", "status": "modified", "previous_filename": None}, + {"filename": "changelog.d/1.added.md", "status": "renamed", "previous_filename": "changelog.d/1.fixed.md"}, + )) + run = FakeRun(completed(json.dumps({"title": "chore(release): v0.82.0", "base": "release/0.82"}) + "\n"), + completed(files + "\n")) + title, base, found = check.fetch_pull_request(3600, "TableProApp/TablePro", run) + self.assertEqual((title, base), ("chore(release): v0.82.0", "release/0.82")) + self.assertEqual(found, [changed("CHANGELOG.md", "modified"), + changed("changelog.d/1.added.md", "renamed", "changelog.d/1.fixed.md")]) + self.assertEqual(run.commands[0][:3], ["gh", "api", "repos/TableProApp/TablePro/pulls/3600"]) + self.assertEqual(run.commands[1][:4], ["gh", "api", "--paginate", "repos/TableProApp/TablePro/pulls/3600/files"]) + + def test_a_gh_failure_fails_the_check(self): + status, out = self.run_main(FakeRun(completed(returncode=1, stderr="HTTP 404"))) + self.assertEqual(status, 1) + self.assertIn("HTTP 404", out) + + def test_a_clean_pull_request_passes(self): + files = json.dumps({"filename": "changelog.d/3600.fixed.md", "status": "added"}) + run = FakeRun(completed(json.dumps({"title": "fix: x", "base": "main"})), completed(files)) + self.assertEqual(self.run_main(run), (0, "OK: the changelog files of #3600\n")) + + def test_an_annotation_cannot_inject_a_workflow_command(self): + files = json.dumps({"filename": "changelog.d/x\n::add-mask::secret", "status": "added"}) + run = FakeRun(completed(json.dumps({"title": "fix: x", "base": "main"})), completed(files)) + status, out = self.run_main(run) + self.assertEqual(status, 1) + self.assertEqual(len(out.splitlines()), 1) + self.assertIn("%0A::add-mask::secret", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/ci/test_check_plugin_manifest.py b/scripts/ci/test_check_plugin_manifest.py index 2e7d504fe9..7f370e53fb 100644 --- a/scripts/ci/test_check_plugin_manifest.py +++ b/scripts/ci/test_check_plugin_manifest.py @@ -1,7 +1,8 @@ #!/usr/bin/env python3 -"""Fixture tests for the project.yml aggregate checks in check-plugin-manifest.py.""" +"""Fixture tests for check-plugin-manifest.py.""" import importlib.util +import tempfile import unittest from pathlib import Path @@ -68,10 +69,7 @@ App: all """ -PLUGINS = { - "oracle": {"target": "OracleDriver", "bundled": False}, - "redis": {"target": "RedisDriver", "bundled": True}, -} +PLUGINS = {"OracleDriver": {"name": "Oracle Driver", "iconName": "oracle-icon"}} def problems(project=PROJECT, plugins=None): @@ -105,7 +103,7 @@ def test_ignores_template_dependencies(self): class PublishSetTests(unittest.TestCase): - def test_bundled_entries_are_not_published(self): + def test_every_target_in_the_metadata_is_published(self): self.assertEqual(check.publish_set(PLUGINS), {"OracleDriver"}) @@ -119,12 +117,18 @@ def test_published_plugin_missing_from_registry_aggregate(self): self.assertIn("OracleDriver is in 0 of BundledPlugins and RegistryPlugins, expected exactly one", found) def test_registry_aggregate_target_that_is_not_published(self): - plugins = {"redis": PLUGINS["redis"]} self.assertEqual( - problems(plugins=plugins), + problems(plugins={}), ["RegistryPlugins builds OracleDriver but plugin-registry.json does not publish it"], ) + def test_bundled_plugin_in_the_metadata(self): + plugins = {**PLUGINS, "RedisDriver": {"name": "Redis Driver", "iconName": "redis-icon"}} + self.assertEqual( + problems(plugins=plugins), + ["plugin-registry.json publishes RedisDriver but RegistryPlugins does not build it"], + ) + def test_embedded_plugin_missing_from_bundled_aggregate(self): found = problems(edited(" - MySQLDriver\n - RedisDriver\n", " - RedisDriver\n")) self.assertIn("the TablePro target embeds MySQLDriver but BundledPlugins does not list it", found) @@ -153,5 +157,31 @@ def test_missing_aggregate(self): self.assertEqual(problems(project), ["project.yml has no RegistryPlugins aggregate target"]) +class DeclarationProblemsTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.addCleanup(self.temp.cleanup) + source = Path(self.temp.name) / "OracleDriverPlugin" / "OraclePlugin.swift" + source.parent.mkdir() + source.write_text('final class OraclePlugin {\n static let iconName = "oracle-icon"\n}\n') + self.saved = check.PLUGINS_DIR + check.PLUGINS_DIR = Path(self.temp.name) + self.addCleanup(setattr, check, "PLUGINS_DIR", self.saved) + + def test_matching_icon_passes(self): + self.assertEqual(check.declaration_problems(PLUGINS), []) + + def test_icon_that_differs_from_the_plugin(self): + plugins = {"OracleDriver": {"name": "Oracle Driver", "iconName": "cylinder"}} + self.assertEqual( + check.declaration_problems(plugins), + ["OracleDriver: plugin-registry.json says iconName 'cylinder' but the plugin declares 'oracle-icon'"], + ) + + def test_target_with_no_plugin_directory(self): + plugins = {"TeradataDriver": {"name": "Teradata Driver", "iconName": "teradata-icon"}} + self.assertEqual(check.declaration_problems(plugins), ["TeradataDriver: no plugin directory matches it"]) + + if __name__ == "__main__": unittest.main() diff --git a/scripts/ci/test_check_pr_title.py b/scripts/ci/test_check_pr_title.py index dc176cfe43..f80ce884ca 100644 --- a/scripts/ci/test_check_pr_title.py +++ b/scripts/ci/test_check_pr_title.py @@ -1,8 +1,4 @@ #!/usr/bin/env python3 -"""Tests for check-pr-title.py. - -Run: python3 scripts/ci/test_check_pr_title.py -""" import importlib.util import io @@ -53,6 +49,24 @@ def test_the_title_alone_is_measured(self): def test_a_long_title_in_the_wrong_shape_reports_both(self): self.assertEqual(len(check.problems("Fix " + "x" * 100)), 2) + def test_a_picks_trailing_pull_request_number_is_not_counted(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ")) + self.assertEqual(check.problems(title + " (#3443)"), []) + self.assertEqual(check.problems("chore(release): v0.82.0 (#3601)"), []) + problems = check.problems(title + "x (#3443)") + self.assertEqual(len(problems), 1) + self.assertIn("73 characters", problems[0]) + + def test_only_one_trailing_pull_request_number_is_skipped(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ")) + self.assertIn("77 characters", check.problems(title + " (#1) (#2)")[0]) + + def test_a_number_that_is_not_at_the_end_is_counted(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ") - len(" (#3443)")) + self.assertEqual(check.problems(title + " (#3443)"), []) + self.assertEqual(len(check.problems(title + " (#3443)x")), 1) + self.assertEqual(len(check.problems(title + "x (#3443) ")), 1) + class OutputTests(unittest.TestCase): def run_main(self, *argv): diff --git a/scripts/ci/test_create_dmg_args.sh b/scripts/ci/test_create_dmg_args.sh deleted file mode 100755 index 34d36585f8..0000000000 --- a/scripts/ci/test_create_dmg_args.sh +++ /dev/null @@ -1,93 +0,0 @@ -#!/usr/bin/env bash -# -# Checks how create-dmg.sh reads its arguments, with every tool that would touch the system stubbed. -# -# build-release.sh writes one bundle per architecture, build/Release/TablePro-.app, and -# nothing builds a universal TablePro.app, so the architecture picks the bundle and has no default. -# -# Each case runs in a scratch directory on a PATH holding only stubs and the plain file tools the -# script needs. SetFile is kept off it on purpose: create-dmg.sh falls back to an Applications -# symlink when the Finder alias fails, and with SetFile present it copies an icon through it. -# -# Usage: test_create_dmg_args.sh - -set -euo pipefail - -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -SCRIPT="$REPO_ROOT/scripts/create-dmg.sh" -BASH_BIN="$(command -v bash)" - -SCRATCH="$(mktemp -d)" -trap 'rm -rf "$SCRATCH"' EXIT - -TOOLS="$SCRATCH/tools" -mkdir -p "$TOOLS" -for tool in awk cp dirname du ln mkdir rm sed tr; do - ln -s "$(command -v "$tool")" "$TOOLS/$tool" -done - -write_stub() { - { printf '#!/bin/sh\n'; cat; } > "$TOOLS/$1" - chmod +x "$TOOLS/$1" -} -write_stub osascript <<< 'exit 0' -write_stub codesign <<< 'exit 0' -write_stub hdiutil <<< 'exit 1' -write_stub create-dmg <<'STUB' -eval "dmg=\${$(($# - 1))}" -: > "$dmg" -STUB - -FAILURES=0 -OUTPUT="" -STATUS=0 - -run_case() { - local work="$SCRATCH/work-$1" - shift - mkdir -p "$work/build/Release/TablePro-arm64.app" "$work/build/Release/TablePro-x86_64.app" - mkdir -p "$work/custom/Built.app" - STATUS=0 - OUTPUT="$(cd "$work" && NOTARIZE=false PATH="$TOOLS" "$BASH_BIN" "$SCRIPT" "$@" 2>&1)" || STATUS=$? - WORK="$work" -} - -fail() { - echo "FAIL: $1" >&2 - printf ' %s\n' "${OUTPUT//$'\n'/$'\n '}" >&2 - FAILURES=$((FAILURES + 1)) -} - -expect_output() { - case "$OUTPUT" in - *"$2"*) ;; - *) fail "$1: output lacks '$2'" ;; - esac -} - -run_case arm64 0.0.0 arm64 -[ "$STATUS" -eq 0 ] || fail "arm64: exited $STATUS" -expect_output arm64 "Source: build/Release/TablePro-arm64.app" -[ -f "$WORK/build/Release/TablePro-0.0.0-arm64.dmg" ] || fail "arm64: no TablePro-0.0.0-arm64.dmg" - -run_case x86_64 0.0.0 x86_64 -[ "$STATUS" -eq 0 ] || fail "x86_64: exited $STATUS" -expect_output x86_64 "Source: build/Release/TablePro-x86_64.app" - -run_case explicit 0.0.0 arm64 custom/Built.app -[ "$STATUS" -eq 0 ] || fail "explicit source: exited $STATUS" -expect_output "explicit source" "Source: custom/Built.app" - -for arguments in "" "0.0.0" "0.0.0 universal"; do - read -r -a argv <<< "$arguments" - run_case "usage-${arguments// /-}" ${argv[@]+"${argv[@]}"} - [ "$STATUS" -ne 0 ] || fail "'$arguments': exited 0" - expect_output "'$arguments'" "Usage: create-dmg.sh [source_app]" -done - -if [ "$FAILURES" -ne 0 ]; then - echo "$FAILURES create-dmg.sh argument check(s) failed." >&2 - exit 1 -fi - -echo "create-dmg.sh takes its architecture and bundle as build-release.sh writes them." diff --git a/scripts/ci/test_detect_changed_paths.py b/scripts/ci/test_detect_changed_paths.py index 07e98fa0ef..66a2725d05 100644 --- a/scripts/ci/test_detect_changed_paths.py +++ b/scripts/ci/test_detect_changed_paths.py @@ -119,6 +119,21 @@ def test_macos_keeps_its_test_folders_and_ignores_the_release_workflow(self): self.assertEqual(self.verdict_after("macos", ".github/workflows/release-build.yml"), "false") self.assertEqual(self.verdict_after("macos", "scripts/release/build.sh"), "false") + def test_macos_runs_on_the_notes_script_its_build_phase_reads(self): + self.assertEqual(self.verdict_after("macos", "scripts/release/notes.py"), "true") + self.assertEqual(self.verdict_after("macos", "scripts/release/feed.py"), "false") + + def test_changelog_files_run_no_suite(self): + for suite in ["macos", "plugins", "oracle", "hana", "release"]: + for path in ["CHANGELOG.md", "changelog.d/3500.added.md", "towncrier.toml", "appcast.xml"]: + with self.subTest(suite=suite, path=path): + self.assertEqual(self.verdict_after(suite, path), "false") + + def test_a_change_beside_the_changelog_still_runs(self): + self.git("checkout", "-q", "-B", "case", self.seed) + self.commit(["CHANGELOG.md", "TablePro/App.swift"]) + self.assertEqual(self.detect("macos", self.seed), (0, "true")) + def test_new_branch_at_main_skips(self): self.git("update-ref", "refs/remotes/origin/main", self.seed) self.assertEqual(self.detect("release", ZERO), (0, "false")) diff --git a/scripts/ci/test_draft_release.sh b/scripts/ci/test_draft_release.sh deleted file mode 100755 index e1f4e3ed37..0000000000 --- a/scripts/ci/test_draft_release.sh +++ /dev/null @@ -1,258 +0,0 @@ -#!/usr/bin/env bash -# -# Checks which release draft-release.sh creates, deletes or refuses, against a stub gh that keeps -# the repository's releases in a scratch directory. -# -# Usage: test_draft_release.sh - -set -euo pipefail - -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -SCRIPT="$REPO_ROOT/scripts/ci/draft-release.sh" - -SCRATCH="$(mktemp -d)" -trap 'rm -rf "$SCRATCH"' EXIT - -mkdir -p "$SCRATCH/bin" -cat > "$SCRATCH/bin/gh" <<'STUB' -#!/usr/bin/env bash -set -euo pipefail -echo "$*" >> "$STATE/log" -digest_of() { - if [ -f "$STATE/corrupt" ]; then - echo "sha256:0000" - else - echo "sha256:$(shasum -a 256 "$1" | cut -d' ' -f1)" - fi -} -case "$*" in - "api repos/test/repo/releases/tags/"*) - if [ -f "$STATE/fail-lookup" ]; then - echo "gh: Server Error (HTTP 500)" >&2 - exit 1 - fi - if [ ! -f "$STATE/published" ]; then - echo "gh: Not Found (HTTP 404)" >&2 - exit 1 - fi - cat "$STATE/published" - ;; - "api --method PATCH repos/test/repo/releases/"*) ;; - "api repos/test/repo/releases/"*) - id="${2##*/}" - cat "$STATE/assets-$id.tsv" - ;; - "release view "*) - if [ ! -s "$STATE/drafts" ]; then - echo "release not found" >&2 - exit 1 - fi - head -1 "$STATE/drafts" - ;; - "release create "* | "release upload "*) - if [ "$2" = "create" ]; then - echo 900 >> "$STATE/drafts" - fi - id="$(head -1 "$STATE/drafts")" - if [ -f "$STATE/published-meanwhile-$id" ]; then - echo "HTTP 422: Cannot delete asset from an immutable release." >&2 - exit 1 - fi - : > "$STATE/assets-$id.tsv" - for arg in "$@"; do - case "$arg" in - *.zip) printf '%s\tuploaded\t%s\n' "$(basename "$arg")" "$(digest_of "$arg")" >> "$STATE/assets-$id.tsv" ;; - esac - done - ;; - "release download "*) - pattern="" - dir="" - while [ $# -gt 0 ]; do - case "$1" in - --pattern) pattern="$2"; shift 2 ;; - --dir) dir="$2"; shift 2 ;; - *) shift ;; - esac - done - mkdir -p "$dir" - cp "$STATE/served/$pattern" "$dir/$pattern" - ;; - *) - echo "stub gh: unexpected call: $*" >&2 - exit 99 - ;; -esac -STUB -chmod +x "$SCRATCH/bin/gh" - -FAILURES=0 -STATUS=0 -OUTPUT="" -STATE="" - -sha() { - shasum -a 256 "$1" | cut -d' ' -f1 -} - -new_case() { - STATE="$SCRATCH/$1" - mkdir -p "$STATE/build" "$STATE/served" - : > "$STATE/log" - : > "$STATE/drafts" - : > "$STATE/output" - printf 'notes\n' > "$STATE/notes.md" - printf 'arm64 build\n' > "$STATE/build/Plugin-arm64.zip" - printf 'x86_64 build\n' > "$STATE/build/Plugin-x86_64.zip" -} - -publish_as_built() { - echo 500 > "$STATE/published" - : > "$STATE/assets-500.tsv" - for arch in arm64 x86_64; do - printf 'Plugin-%s.zip\tuploaded\tsha256:%s\n' "$arch" "$(sha "$STATE/build/Plugin-$arch.zip")" >> "$STATE/assets-500.tsv" - done -} - -run() { - STATUS=0 - OUTPUT="$(cd "$STATE" && export STATE && PATH="$SCRATCH/bin:$PATH" GH_REPO=test/repo \ - GITHUB_OUTPUT="$STATE/output" "$SCRIPT" "$@" 2>&1)" || STATUS=$? -} - -run_default() { - run "$@" v1.0.0 "v1.0.0" notes.md build/Plugin-arm64.zip build/Plugin-x86_64.zip -} - -fail() { - echo "FAIL: $1" >&2 - printf ' %s\n' "${OUTPUT//$'\n'/$'\n '}" >&2 - FAILURES=$((FAILURES + 1)) -} - -expect_status() { - [ "$STATUS" -eq "$2" ] || fail "$1: exited $STATUS, expected $2" -} - -expect_output_file() { - [ "$(cat "$STATE/output")" = "$2" ] || fail "$1: GITHUB_OUTPUT holds '$(cat "$STATE/output")', expected '$2'" -} - -expect_output() { - case "$OUTPUT" in - *"$2"*) ;; - *) fail "$1: output lacks '$2'" ;; - esac -} - -expect_log() { - grep -qF -- "$2" "$STATE/log" || fail "$1: no gh call matching '$2'" -} - -refute_log() { - if grep -qF -- "$2" "$STATE/log"; then - fail "$1: unexpected gh call matching '$2'" - fi -} - -new_case fresh -run_default -expect_status fresh 0 -expect_output_file fresh "id=900" -expect_log fresh "release create v1.0.0 --draft --title v1.0.0 --notes-file notes.md --verify-tag build/Plugin-arm64.zip build/Plugin-x86_64.zip" -refute_log fresh "--prerelease" - -new_case target -run_default --target abc123 --prerelease true -expect_status target 0 -expect_log target "--target abc123 --prerelease" -refute_log target "--verify-tag" - -new_case stale -printf '101\n' > "$STATE/drafts" -run_default --target abc123 -expect_status stale 0 -expect_log stale "api --method PATCH repos/test/repo/releases/101 -f name=v1.0.0 -F body=@notes.md -F prerelease=false -f target_commitish=abc123" -expect_log stale "release upload v1.0.0 build/Plugin-arm64.zip build/Plugin-x86_64.zip --clobber" -refute_log stale "release create" -refute_log stale "DELETE" -expect_output_file stale "id=101" - -new_case race -printf '101\n' > "$STATE/drafts" -touch "$STATE/published-meanwhile-101" -run_default -expect_status race 1 -expect_output race "immutable release" -refute_log race "release create" -refute_log race "DELETE" -expect_output_file race "" - -new_case published -publish_as_built -run_default -expect_status published 0 -expect_output_file published "id=" -refute_log published "release create" - -new_case rebuilt -publish_as_built -printf 'a different arm64 build\n' > "$STATE/build/Plugin-arm64.zip" -run_default -expect_status rebuilt 1 -expect_output rebuilt "v1.0.0 is already published with other assets" -refute_log rebuilt "release create" -refute_log rebuilt "DELETE" -expect_output_file rebuilt "" - -new_case missing -publish_as_built -sed -i.bak '/x86_64/d' "$STATE/assets-500.tsv" -run_default -expect_status missing 1 -expect_output missing "v1.0.0 has no Plugin-x86_64.zip" -refute_log missing "release create" - -new_case partial -publish_as_built -sed -i.bak 's/uploaded/starter/' "$STATE/assets-500.tsv" -run_default -expect_status partial 1 -expect_output partial "Plugin-arm64.zip on v1.0.0 is starter" - -new_case undigested -publish_as_built -sed -i.bak 's/sha256:[0-9a-f]*$//' "$STATE/assets-500.tsv" -cp "$STATE/build/Plugin-arm64.zip" "$STATE/build/Plugin-x86_64.zip" "$STATE/served/" -run_default -expect_status undigested 0 -expect_output_file undigested "id=" -expect_log undigested "release download v1.0.0 --pattern Plugin-arm64.zip" - -new_case corrupt -touch "$STATE/corrupt" -run_default -expect_status corrupt 1 -expect_output_file corrupt "" -expect_output corrupt "stays unpublished" - -new_case outage -touch "$STATE/fail-lookup" -run_default -expect_status outage 1 -refute_log outage "release create" -refute_log outage "release view" - -new_case usage -run v1.0.0 "v1.0.0" notes.md -expect_status usage 2 -run --prerelease maybe v1.0.0 "v1.0.0" notes.md build/Plugin-arm64.zip -expect_status "usage prerelease" 2 -[ ! -s "$STATE/log" ] || fail "usage: called gh" - -if [ "$FAILURES" -ne 0 ]; then - echo "$FAILURES draft-release.sh check(s) failed." >&2 - exit 1 -fi - -echo "draft-release.sh drafts, resumes and refuses as expected." diff --git a/scripts/ci/test_merge_appcast.py b/scripts/ci/test_merge_appcast.py deleted file mode 100755 index 5f04348fb7..0000000000 --- a/scripts/ci/test_merge_appcast.py +++ /dev/null @@ -1,233 +0,0 @@ -#!/usr/bin/env python3 -"""Regression tests for the Sparkle feed merge. - -This script edits the file every install polls, and the release job is the only thing that runs -it. So the invariants are tested here, on the free Linux runner, rather than discovered by a user -whose updater has gone quiet. - -Run: python3 scripts/ci/test_merge_appcast.py -""" - -import importlib.util -import pathlib -import sys -import tempfile -import unittest - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) -SPEC = importlib.util.spec_from_file_location( - "merge_appcast", pathlib.Path(__file__).with_name("merge-appcast.py") -) -merge_appcast = importlib.util.module_from_spec(SPEC) -SPEC.loader.exec_module(merge_appcast) - -import appcast_feed # noqa: E402 - -PREFIX = "https://github.com/TableProApp/TablePro/releases/download/v0.75.0/" - - -def item(version, build, arch, notes="Fixed a thing.", prefix=None): - prefix = PREFIX if prefix is None else prefix - hardware = ( - f" arch\n".replace( - "arch", arch - ) - if arch - else "" - ) - suffix = f"-{arch}" if arch else "-x86_64" - return ( - " \n" - f" {version}\n" - " Sat, 13 Sep 2026 10:00:00 +0000\n" - f" {build}\n" - f" {version}\n" - " 14.0\n" - f"{hardware}" - f" {notes}

]]>\n" - f' \n' - " \n" - ) - - -def feed(*items): - return ( - '\n' - '\n' - " \n" - " TablePro\n" - + "".join(items) - + " \n" - "\n" - ) - - -BASE = feed(item("0.74.0", 130, "arm64"), item("0.74.0", 130, None), item("0.73.0", 129, "arm64")) - - -PUBLISHED = pathlib.Path(__file__).resolve().parents[2] / "appcast.xml" - - -def succeeding(base): - """The version and build a release cut right now would carry, read from the feed itself. - - Pinning them to the version in Configs/Version.xcconfig makes every test using the published - feed fail the moment that release is published, because the feed then advertises it and the - merge refuses a version it already carries. - """ - items = merge_appcast.channel_items(merge_appcast.parse_text(base), "base") - newest = merge_appcast.item_short_version(items[0]).split(".") - newest[-1] = str(int(newest[-1]) + 1) - builds = [int(b) for b in (merge_appcast.item_bundle_version(i) for i in items) if b] - return ".".join(newest), max(builds) + 1 - - -class MergeAppcastTests(unittest.TestCase): - def merge(self, base=BASE, arm64=None, x86_64=None, version="0.75.0", prefix=PREFIX, keep_releases=0, build=131): - arm64 = feed(item(version, build, "arm64")) if arm64 is None else arm64 - x86_64 = feed(item(version, build, None)) if x86_64 is None else x86_64 - with tempfile.TemporaryDirectory() as directory: - work = pathlib.Path(directory) - (work / "base.xml").write_text(base, encoding="utf-8") - (work / "arm64.xml").write_text(arm64, encoding="utf-8") - (work / "x86_64.xml").write_text(x86_64, encoding="utf-8") - return merge_appcast.merge( - work / "base.xml", - work / "arm64.xml", - work / "x86_64.xml", - version, - prefix, - work / "out.xml", - keep_releases, - ) - - def versions(self, merged): - items = merge_appcast.channel_items(merge_appcast.parse_text(merged), "merged") - return [merge_appcast.item_short_version(i) for i in items] - - def assertRefused(self, fragment, **kwargs): - with self.assertRaises(merge_appcast.MergeError) as caught: - self.merge(**kwargs) - self.assertIn(fragment, str(caught.exception)) - - def test_new_items_lead_the_feed_arm64_first(self): - merged = self.merge() - items = merge_appcast.channel_items(merge_appcast.parse_text(merged), "merged") - self.assertEqual([merge_appcast.item_short_version(i) for i in items[:2]], ["0.75.0", "0.75.0"]) - self.assertIsNotNone(items[0].find(merge_appcast.HARDWARE)) - self.assertIsNone(items[1].find(merge_appcast.HARDWARE)) - self.assertEqual(len(items), 5) - - def test_every_other_item_survives_byte_for_byte(self): - merged = self.merge() - inserted = item("0.75.0", 131, "arm64") + item("0.75.0", 131, None) - self.assertIn(inserted, merged) - self.assertEqual(merged.replace(inserted, "", 1), BASE) - - def test_release_notes_containing_the_closing_tag_survive(self): - """The awk merge this replaced scanned for the literal, so a changelog entry broke the feed.""" - notes = "Fixed the parser on a literal </item> in a cell." - merged = self.merge(arm64=feed(item("0.75.0", 131, "arm64", notes=notes))) - items = merge_appcast.channel_items(merge_appcast.parse_text(merged), "merged") - self.assertEqual(len(items), 5) - self.assertIn(notes, merged) - - def test_refuses_a_version_the_base_already_advertises(self): - older = PREFIX.replace("v0.75.0", "v0.74.0") - self.assertRefused( - "already advertises", - version="0.74.0", - prefix=older, - arm64=feed(item("0.74.0", 131, "arm64", prefix=older)), - x86_64=feed(item("0.74.0", 131, None, prefix=older)), - ) - - def test_refuses_a_build_number_that_did_not_increase(self): - self.assertRefused( - "not below the new build", - arm64=feed(item("0.75.0", 130, "arm64")), - x86_64=feed(item("0.75.0", 130, None)), - ) - - def test_refuses_an_arm64_item_with_no_hardware_requirement(self): - self.assertRefused("no ", arm64=feed(item("0.75.0", 131, None))) - - def test_refuses_an_x86_64_item_carrying_a_hardware_requirement(self): - self.assertRefused( - "carries ", x86_64=feed(item("0.75.0", 131, "arm64")) - ) - - def test_refuses_architectures_that_disagree_on_the_build(self): - self.assertRefused("disagree on ", x86_64=feed(item("0.75.0", 132, None))) - - def test_refuses_an_enclosure_outside_the_release_directory(self): - stray = feed(item("0.75.0", 131, "arm64")).replace(PREFIX, PREFIX.replace("v0.75.0", "v0.74.0")) - self.assertRefused("is not under", arm64=stray) - - def test_refuses_an_unsigned_enclosure(self): - """generate_appcast drops the signature and still exits 0 when SUPublicEDKey is absent.""" - unsigned = feed(item("0.75.0", 131, "arm64")).replace(' sparkle:edSignature="sig=="', "") - self.assertRefused("carries no sparkle:edSignature", arm64=unsigned) - - def test_refuses_more_than_one_item_per_architecture(self): - two = feed(item("0.75.0", 131, "arm64"), item("0.74.0", 130, "arm64")) - self.assertRefused("expected exactly one item", arm64=two) - - def test_refuses_a_generated_feed_holding_a_different_version(self): - self.assertRefused("expected exactly one item", arm64=feed(item("0.74.9", 131, "arm64"))) - - def test_splices_into_a_feed_that_holds_no_items_yet(self): - merged = self.merge(base=feed()) - items = merge_appcast.channel_items(merge_appcast.parse_text(merged), "merged") - self.assertEqual(len(items), 2) - - def test_the_published_feed_is_a_valid_base(self): - base = PUBLISHED.read_text(encoding="utf-8") - version, build = succeeding(base) - merged = self.merge(base=base, version=version, build=build) - versions = self.versions(merged) - self.assertEqual(versions[:2], [version, version]) - published = merge_appcast.channel_items(merge_appcast.parse_text(base), "base") - self.assertEqual(len(versions), len(published) + 2) - - def test_keeps_every_release_when_pruning_is_off(self): - self.assertEqual(self.versions(self.merge(keep_releases=0)), ["0.75.0", "0.75.0", "0.74.0", "0.74.0", "0.73.0"]) - - def test_prunes_to_the_newest_releases(self): - merged = self.merge(keep_releases=2) - self.assertEqual(self.versions(merged), ["0.75.0", "0.75.0", "0.74.0", "0.74.0"]) - - def test_pruning_keeps_the_arm64_item_first(self): - merged = self.merge(keep_releases=1) - items = merge_appcast.channel_items(merge_appcast.parse_text(merged), "merged") - self.assertEqual(self.versions(merged), ["0.75.0", "0.75.0"]) - self.assertIsNotNone(items[0].find(merge_appcast.HARDWARE)) - self.assertIsNone(items[1].find(merge_appcast.HARDWARE)) - - def test_pruning_a_feed_shorter_than_the_limit_drops_nothing(self): - self.assertEqual(self.versions(self.merge(keep_releases=99)), ["0.75.0", "0.75.0", "0.74.0", "0.74.0", "0.73.0"]) - - def test_pruning_keeps_the_retained_items_byte_for_byte(self): - """The kept items are moved, never re-serialized: an entity-escaped CDATA block would grow - the feed by 9% and make every future diff unreadable.""" - merged = self.merge(keep_releases=2) - kept = appcast_feed.item_spans(merged) - survivor = merged[kept[2][0]:kept[3][1]] - original = appcast_feed.item_spans(BASE) - self.assertEqual(survivor, BASE[original[0][0]:original[1][1]]) - - def test_pruning_the_published_feed_bounds_its_size(self): - base = PUBLISHED.read_text(encoding="utf-8") - version, build = succeeding(base) - merged = self.merge(base=base, version=version, build=build, keep_releases=merge_appcast.DEFAULT_KEEP_RELEASES) - versions = self.versions(merged) - self.assertEqual(versions[:2], [version, version]) - self.assertEqual(len(dict.fromkeys(versions)), merge_appcast.DEFAULT_KEEP_RELEASES) - kept = self.merge(base=base, version=version, build=build) - self.assertLess(len(merged), len(kept)) - self.assertNotIn(merge_appcast.releases_in_order(base, "base")[-1], versions) - - -if __name__ == "__main__": - unittest.main() diff --git a/scripts/ci/test_pull_release.py b/scripts/ci/test_pull_release.py deleted file mode 100755 index 2c251e3fca..0000000000 --- a/scripts/ci/test_pull_release.py +++ /dev/null @@ -1,224 +0,0 @@ -#!/usr/bin/env python3 -"""Regression tests for withdrawing a release from the Sparkle feed. - -This is the rollback path. It runs once every few years, under time pressure, on the file every -install polls, so it is tested here rather than rehearsed on the day. - -Run: python3 scripts/ci/test_pull_release.py -""" - -import contextlib -import importlib.util -import os -import pathlib -import subprocess -import sys -import tempfile -import unittest - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) -SPEC = importlib.util.spec_from_file_location( - "pull_release", pathlib.Path(__file__).with_name("pull-release.py") -) -pull_release = importlib.util.module_from_spec(SPEC) -SPEC.loader.exec_module(pull_release) - -TEST_MERGE = importlib.util.spec_from_file_location( - "test_merge_appcast", pathlib.Path(__file__).with_name("test_merge_appcast.py") -) -fixtures = importlib.util.module_from_spec(TEST_MERGE) -TEST_MERGE.loader.exec_module(fixtures) - -item = fixtures.item -feed = fixtures.feed - -BASE = feed( - item("0.75.0", 131, "arm64"), - item("0.75.0", 131, None), - item("0.74.0", 130, "arm64"), - item("0.74.0", 130, None), - item("0.73.0", 129, "arm64"), -) - - -class PullReleaseTests(unittest.TestCase): - def withdraw(self, version, text=BASE): - with tempfile.TemporaryDirectory() as directory: - path = pathlib.Path(directory) / "appcast.xml" - path.write_text(text, encoding="utf-8") - return pull_release.withdraw(path, version) - - def versions(self, text): - root = pull_release.parse_text(text) - return [pull_release.item_short_version(i) for i in pull_release.channel_items(root, "t")] - - def test_removes_both_architecture_items(self): - result, removed = self.withdraw("0.75.0") - self.assertEqual(removed, 2) - self.assertEqual(self.versions(result), ["0.74.0", "0.74.0", "0.73.0"]) - - def test_every_surviving_item_is_byte_identical(self): - result, _ = self.withdraw("0.75.0") - self.assertEqual( - result, - feed(item("0.74.0", 130, "arm64"), item("0.74.0", 130, None), item("0.73.0", 129, "arm64")), - ) - - def test_removes_a_version_from_the_middle_of_the_feed(self): - result, removed = self.withdraw("0.74.0") - self.assertEqual(removed, 2) - self.assertEqual(self.versions(result), ["0.75.0", "0.75.0", "0.73.0"]) - - def test_refuses_a_version_the_feed_does_not_carry(self): - with self.assertRaises(pull_release.FeedError) as caught: - self.withdraw("9.9.9") - self.assertIn("nothing to withdraw", str(caught.exception)) - - def test_refuses_to_empty_the_feed(self): - only = feed(item("0.75.0", 131, "arm64")) - with self.assertRaises(pull_release.FeedError) as caught: - self.withdraw("0.75.0", text=only) - self.assertIn("would empty the feed", str(caught.exception)) - - def test_release_notes_containing_the_closing_tag_do_not_widen_the_cut(self): - text = feed( - item("0.75.0", 131, "arm64", notes="a literal </item> in a cell"), - item("0.74.0", 130, "arm64"), - ) - result, removed = self.withdraw("0.75.0", text=text) - self.assertEqual(removed, 1) - self.assertEqual(self.versions(result), ["0.74.0"]) - - def test_withdraws_the_newest_version_from_the_published_feed(self): - published = pathlib.Path(__file__).resolve().parents[2] / "appcast.xml" - text = published.read_text(encoding="utf-8") - newest = self.versions(text)[0] - result, removed = self.withdraw(newest, text=text) - self.assertGreaterEqual(removed, 1) - self.assertNotIn(newest, self.versions(result)) - self.assertEqual(len(self.versions(result)), len(self.versions(text)) - removed) - - -@contextlib.contextmanager -def repository_on(branch): - """A throwaway git repo checked out on `branch`, or detached when branch is None.""" - with tempfile.TemporaryDirectory() as directory: - def run(*args): - subprocess.run(["git", *args], cwd=directory, check=True, capture_output=True) - - run("init", "--quiet", "--initial-branch", "main") - run("config", "user.email", "test@example.com") - run("config", "user.name", "Test") - pathlib.Path(directory, "appcast.xml").write_text(BASE, encoding="utf-8") - run("add", "appcast.xml") - run("commit", "--quiet", "-m", "seed") - if branch is None: - run("checkout", "--quiet", "--detach", "HEAD") - elif branch != "main": - run("checkout", "--quiet", "-b", branch) - previous = os.getcwd() - os.chdir(directory) - try: - yield directory - finally: - os.chdir(previous) - - -class PublishTargetTests(unittest.TestCase): - """SUFeedURL serves main. A withdrawal pushed anywhere else leaves every install downloading the - bad build while the script prints that it is no longer offered, which `git push origin HEAD` from - one of this repo's worktrees did.""" - - def test_accepts_main(self): - with repository_on("main"): - self.assertEqual(pull_release.require_publishable_checkout(), "main") - - def test_refuses_a_feature_branch(self): - with repository_on("fix/some-feature"): - with self.assertRaises(pull_release.FeedError) as caught: - pull_release.require_publishable_checkout() - self.assertIn("fix/some-feature", str(caught.exception)) - self.assertIn("main", str(caught.exception)) - - def test_refuses_a_detached_head(self): - with repository_on(None): - with self.assertRaises(pull_release.FeedError) as caught: - pull_release.require_publishable_checkout() - self.assertIn("detached", str(caught.exception)) - - def test_a_wrong_branch_writes_nothing(self): - with repository_on("fix/some-feature") as directory: - feed_path = pathlib.Path(directory, "appcast.xml") - before = feed_path.read_text(encoding="utf-8") - self.assertEqual(pull_release.main(["0.75.0"]), 1) - self.assertEqual(feed_path.read_text(encoding="utf-8"), before) - - def test_a_dry_run_is_allowed_anywhere(self): - with repository_on("fix/some-feature"): - self.assertEqual(pull_release.main(["0.75.0", "--dry-run"]), 0) - - -@contextlib.contextmanager -def clone_of_main(ahead=False, behind=False): - """A checkout of main with an origin, one unpushed commit when ahead, one missing when behind.""" - with tempfile.TemporaryDirectory() as directory: - origin = pathlib.Path(directory, "origin.git") - work = pathlib.Path(directory, "work") - - def run(cwd, *args): - subprocess.run(["git", *args], cwd=cwd, check=True, capture_output=True) - - def commit(cwd, message): - run(cwd, "-c", "user.email=test@example.com", "-c", "user.name=Test", - "commit", "--quiet", "--allow-empty", "-m", message) - - run(directory, "init", "--quiet", "--bare", "--initial-branch", "main", str(origin)) - run(directory, "clone", "--quiet", str(origin), str(work)) - run(work, "checkout", "--quiet", "-B", "main") - pathlib.Path(work, "appcast.xml").write_text(BASE, encoding="utf-8") - run(work, "add", "appcast.xml") - commit(work, "seed") - run(work, "push", "--quiet", "origin", "main") - if behind: - other = pathlib.Path(directory, "other") - run(directory, "clone", "--quiet", str(origin), str(other)) - commit(other, "merged elsewhere") - run(other, "push", "--quiet", "origin", "main") - if ahead: - commit(work, "not pushed") - previous = os.getcwd() - os.chdir(work) - try: - yield work - finally: - os.chdir(previous) - - -class PushContentTests(unittest.TestCase): - """The release key bypasses every check on main, so a withdrawal push must hold that commit alone.""" - - def test_accepts_main_at_origin(self): - with clone_of_main(): - pull_release.require_in_step_with_origin() - - def test_refuses_an_unpushed_commit(self): - with clone_of_main(ahead=True): - with self.assertRaises(pull_release.FeedError) as caught: - pull_release.require_in_step_with_origin() - self.assertIn("origin/main", str(caught.exception)) - - def test_refuses_a_main_behind_origin(self): - with clone_of_main(behind=True): - with self.assertRaises(pull_release.FeedError): - pull_release.require_in_step_with_origin() - - def test_an_unpushed_commit_writes_nothing(self): - with clone_of_main(ahead=True) as work: - feed_path = pathlib.Path(work, "appcast.xml") - before = feed_path.read_text(encoding="utf-8") - self.assertEqual(pull_release.main(["0.75.0"]), 1) - self.assertEqual(feed_path.read_text(encoding="utf-8"), before) - - -if __name__ == "__main__": - unittest.main() diff --git a/scripts/ci/test_release_notes.py b/scripts/ci/test_release_notes.py deleted file mode 100644 index 63aa72d188..0000000000 --- a/scripts/ci/test_release_notes.py +++ /dev/null @@ -1,239 +0,0 @@ -#!/usr/bin/env python3 -"""Regression tests for the notes shared by Sparkle and GitHub releases. - -Run: python3 scripts/ci/test_release_notes.py -""" - -from pathlib import Path -import re -import subprocess -import tempfile -import unittest - - -SCRIPT = Path(__file__).with_name("extract-release-notes.sh").resolve() - - -class ReleaseNotesTests(unittest.TestCase): - def extract(self, changelog, version="1.2.3", stale_notes=None): - with tempfile.TemporaryDirectory() as directory: - work = Path(directory) - (work / "CHANGELOG.md").write_text(changelog, encoding="utf-8") - notes = work / "release_notes.md" - if stale_notes is not None: - notes.write_text(stale_notes, encoding="utf-8") - result = subprocess.run( - ["bash", str(SCRIPT), version], - cwd=work, - capture_output=True, - text=True, - timeout=10, - ) - return result, notes.read_text(encoding="utf-8") if notes.exists() else None - - def test_only_requested_version_is_extracted(self): - result, notes = self.extract( - "## [Unreleased]\n- Not released yet\n\n" - "## [1.2.3] - 2026-09-10\n\n### Fixed\n\n- Current fix\n\n" - "## [1.2.2]\n- Old fix\n" - ) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, "### Fixed\n\n- Current fix\n") - - def test_version_is_literal_not_a_regular_expression(self): - result, notes = self.extract( - "## [1x2x3]\n- Wrong version\n" - "## [1.2.3-beta.1]\n- Prerelease\n" - "## [1.2.3]\n- Stable\n" - ) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, "- Stable\n") - - def test_prerelease_version_at_end_of_file(self): - result, notes = self.extract( - "## [1.2.3-beta.1]\n- Prerelease", version="1.2.3-beta.1" - ) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, "- Prerelease\n") - - def test_preserves_markdown_and_literal_code(self): - body = ( - "### Fixed\n\n" - "- `SELECT * FROM users WHERE id < 10` & `` stay visible.\n" - "- **Important**: [Details](https://example.com/fix?a=1&b=2).\n" - "- Phím tắt `Shift+Space`.\n\n" - "```sql\nSELECT '';\n```\n" - ) - result, notes = self.extract("## [1.2.3]\n" + body) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, body) - - def test_missing_or_blank_notes_fail_without_a_generic_fallback(self): - for changelog in ( - "## [1.2.2]\n- Old fix\n", - "## [1.2.3]\n\n## [1.2.2]\n- Old fix\n", - "## [1.2.3]\n \t\n\n", - ): - with self.subTest(changelog=changelog): - result, notes = self.extract(changelog) - self.assertNotEqual(result.returncode, 0) - self.assertIn("No release notes found for version 1.2.3", result.stderr) - self.assertIsNone(notes) - - def test_existing_notes_do_not_override_the_current_changelog(self): - result, notes = self.extract("## [1.2.3]\n- Current fix\n", stale_notes="- Old fix\n") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, "- Current fix\n") - - def test_large_release_is_not_truncated(self): - body = "- A detailed release note with `code` and a fix.\n" * 4000 - result, notes = self.extract("## [1.2.3]\n" + body) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, body) - - def test_sections_follow_keep_a_changelog_order_without_dropping_any(self): - intro = "All changes in this release." - fixed = "### Fixed\n\n- Fix one\n- Fix two" - security = "### Security\n\n- Security fix" - features = "### Added\n\n- Feature one\n - Nested detail\n- Feature two" - changed = "### Changed\n\n- Improvement" - deprecated = "### Deprecated\n\n- Deprecated behavior" - removed = "### Removed\n\n- Removed behavior" - unknown = "### Notes\n\n- Unlisted heading" - result, notes = self.extract( - "## [1.2.3]\n\n" - + "\n\n".join([intro, unknown, fixed, security, removed, features, deprecated, changed]) - ) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual( - notes, - "\n\n".join([intro, features, changed, deprecated, removed, fixed, security, unknown]) + "\n", - ) - - def test_features_heading_aliases_are_first(self): - for heading in ("Features", "New Features"): - with self.subTest(heading=heading): - result, notes = self.extract(f"## [1.2.3]\n### Fixed\n- Fix\n\n### {heading}\n- Feature\n") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, f"### {heading}\n- Feature\n\n### Fixed\n- Fix\n") - - def test_headings_inside_code_are_not_release_or_section_boundaries(self): - for fence in ("```", "~~~~"): - with self.subTest(fence=fence): - fixed = f"### Fixed\n\n- Example:\n\n{fence}markdown\n## [1.2.2]\n### Added\n{fence}\n" - result, notes = self.extract("## [1.2.3]\n" + fixed + "\n### Added\n\n- Actual feature\n") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, "### Added\n\n- Actual feature\n\n" + fixed) - - def test_latest_real_release_keeps_every_change(self): - changelog = (SCRIPT.parents[2] / "CHANGELOG.md").read_text(encoding="utf-8") - releases = list(re.finditer(r"^## \[([^]]+)\].*$", changelog, re.MULTILINE)) - index = next(index for index, release in enumerate(releases) if release[1] != "Unreleased") - release = releases[index] - end = releases[index + 1].start() if index + 1 < len(releases) else len(changelog) - body = changelog[release.end():end] - result, notes = self.extract(changelog, version=release[1]) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertCountEqual( - [line for line in notes.splitlines() if line.strip()], - [line for line in body.splitlines() if line.strip()], - ) - if "### Added\n" in notes and "### Fixed\n" in notes: - self.assertLess(notes.index("### Added\n"), notes.index("### Fixed\n")) - - -class ReleaseHighlightsTests(unittest.TestCase): - """The lead block is what the update dialog and the feed read, so its fallback is load-bearing. - - A version with no lead block must not fail the release: the script runs under `set -euo - pipefail` about forty minutes in, after both notarized builds. - """ - - def extract(self, changelog, version="1.2.3", highlights=True): - with tempfile.TemporaryDirectory() as directory: - work = Path(directory) - (work / "CHANGELOG.md").write_text(changelog, encoding="utf-8") - out = work / "out.md" - args = ["bash", str(SCRIPT), version, "--out", str(out)] - if highlights: - args.append("--highlights-only") - result = subprocess.run(args, cwd=work, capture_output=True, text=True, timeout=10) - text = out.read_text(encoding="utf-8") if out.exists() else "" - return result, text - - WITH_LEAD = ( - "# Changelog\n\n" - "## [1.2.3] - 2026-01-01\n\n" - "Faster grid scrolling on wide results.\n" - "Map view for geometry columns.\n\n" - "### Added\n\n" - "- One.\n- Two.\n\n" - "### Fixed\n\n" - "- Three.\n" - ) - - WITHOUT_LEAD = ( - "# Changelog\n\n" - "## [1.2.3] - 2026-01-01\n\n" - "### Added\n\n" - "- One.\n- Two.\n\n" - "### Fixed\n\n" - "- Three.\n" - ) - - def test_emits_only_the_lead_block(self): - result, notes = self.extract(self.WITH_LEAD) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn("Faster grid scrolling", notes) - self.assertIn("Map view", notes) - self.assertNotIn("### Added", notes) - self.assertNotIn("- One.", notes) - - def test_highlights_are_separate_markdown_paragraphs(self): - expected = "Faster grid scrolling on wide results.\n\nMap view for geometry columns.\n" - spaced = self.WITH_LEAD.replace("results.\nMap", "results.\n\n\nMap") - for changelog in (self.WITH_LEAD, spaced): - with self.subTest(changelog=changelog): - result, notes = self.extract(changelog) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(notes, expected) - - def test_falls_back_to_the_full_notes_when_there_is_no_lead_block(self): - result, notes = self.extract(self.WITHOUT_LEAD) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn("### Added", notes) - self.assertIn("- Three.", notes) - - def test_the_full_notes_are_unchanged_by_the_lead_block(self): - result, notes = self.extract(self.WITH_LEAD, highlights=False) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn("Faster grid scrolling", notes) - self.assertIn("### Added", notes) - self.assertIn("- Three.", notes) - - def test_caps_a_long_lead_block(self): - lead = "".join(f"Line {n}.\n" for n in range(1, 12)) - changelog = f"# Changelog\n\n## [1.2.3] - 2026-01-01\n\n{lead}\n### Added\n\n- One.\n" - result, notes = self.extract(changelog) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(len([line for line in notes.splitlines() if line.strip()]), 6) - self.assertIn("Line 6.", notes) - self.assertNotIn("Line 7.", notes) - - def test_a_missing_version_still_fails(self): - result, _ = self.extract(self.WITH_LEAD, version="9.9.9") - self.assertNotEqual(result.returncode, 0) - - def test_every_shipped_version_produces_highlights(self): - """No version has a lead block today, so every one must take the fallback cleanly.""" - changelog = Path(__file__).resolve().parents[2].joinpath("CHANGELOG.md").read_text(encoding="utf-8") - versions = re.findall(r"^## \[(\d+\.\d+\.\d+)\]", changelog, re.M) - self.assertGreater(len(versions), 5) - for version in versions[:5]: - result, notes = self.extract(changelog, version=version) - self.assertEqual(result.returncode, 0, f"{version}: {result.stderr}") - self.assertTrue(notes.strip(), f"{version} produced empty highlights") - - -if __name__ == "__main__": - unittest.main() diff --git a/scripts/ci/verify-build.sh b/scripts/ci/verify-build.sh deleted file mode 100755 index 806ec39239..0000000000 --- a/scripts/ci/verify-build.sh +++ /dev/null @@ -1,191 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -ARCH="${1:?Usage: verify-build.sh }" - -if [[ "$ARCH" != "arm64" && "$ARCH" != "x86_64" ]]; then - echo "❌ ERROR: Invalid architecture: $ARCH (expected arm64 or x86_64)" - exit 1 -fi - -if [[ "$ARCH" == "arm64" ]]; then - OPPOSITE_ARCH="x86_64" -else - OPPOSITE_ARCH="arm64" -fi - -echo "Verifying build output..." - -BINARY_PATH="build/Release/TablePro-${ARCH}.app/Contents/MacOS/TablePro" - -# Check binary exists -if [ ! -f "$BINARY_PATH" ]; then - echo "❌ ERROR: Built binary not found at: $BINARY_PATH" - echo "Build may have failed silently" - exit 1 -fi - -# Check it's not empty -if [ ! -s "$BINARY_PATH" ]; then - echo "❌ ERROR: Binary file is empty" - exit 1 -fi - -# Check architecture -ARCH_INFO=$(lipo -info "$BINARY_PATH") -echo "Architecture: $ARCH_INFO" - -if ! echo "$ARCH_INFO" | grep -q "$ARCH"; then - echo "❌ ERROR: Binary does not contain $ARCH architecture" - echo "Expected: $ARCH only" - echo "Got: $ARCH_INFO" - exit 1 -fi - -if echo "$ARCH_INFO" | grep -q "$OPPOSITE_ARCH"; then - echo "❌ ERROR: Binary contains $OPPOSITE_ARCH but should be $ARCH only" - exit 1 -fi - -# Check it's executable -if [ ! -x "$BINARY_PATH" ]; then - echo "❌ ERROR: Binary is not executable" - exit 1 -fi - -# Verify bundled dylibs -FRAMEWORKS_DIR="build/Release/TablePro-${ARCH}.app/Contents/Frameworks" -if [ -d "$FRAMEWORKS_DIR" ]; then - echo "Bundled dynamic libraries:" - ls -lh "$FRAMEWORKS_DIR"/*.dylib 2>/dev/null || echo " (none)" - - # Verify no Homebrew paths remain in the binary - if otool -L "$BINARY_PATH" | grep -q '/opt/homebrew/\|/usr/local/opt/'; then - echo "❌ ERROR: Binary still references Homebrew paths:" - otool -L "$BINARY_PATH" | grep '/opt/homebrew/\|/usr/local/opt/' - exit 1 - fi - echo "✅ No Homebrew path references in binary" -else - echo "⚠️ WARNING: No Frameworks directory found — dylibs may not be bundled" -fi - -# Verify plugins -APP_BUNDLE="build/Release/TablePro-${ARCH}.app" -PLUGINS_DIR="$APP_BUNDLE/Contents/PlugIns" - -echo "Verifying plugins..." - -if [ ! -d "$PLUGINS_DIR" ]; then - echo "❌ ERROR: PlugIns directory not found at: $PLUGINS_DIR" - exit 1 -fi -echo "✅ PlugIns directory exists" - -# Derived from project.yml's copy phase, which is what decides what ends up in the bundle. -# This used to be a hardcoded list of three, so a release could ship missing eleven of the -# fourteen embedded plugins and still print "All bundled plugin bundles present". -REQUIRED_PLUGINS=() -while IFS= read -r plugin; do - REQUIRED_PLUGINS+=("${plugin}.tableplugin") -done < <(python3 -c ' -import re, sys - -# Parsed without PyYAML on purpose: this runs on the release path and the module is not part of -# a stock runner image. The shape being read is fixed and small: -# -# - target: MySQLDriver -# copy: -# destination: plugins -# -text = open("project.yml").read() -names = re.findall( - r"-\s*target:\s*(\S+)[^\n]*\n(?:\s+\w+:[^\n]*\n)*?\s*copy:\s*\{\s*destination:\s*plugins\s*\}", - text, -) -if not names: - sys.exit("project.yml declares no plugins copied into the app bundle") -print("\n".join(names)) -') - -if [ "${#REQUIRED_PLUGINS[@]}" -eq 0 ]; then - echo "❌ ERROR: could not read the bundled plugin list from project.yml" - exit 1 -fi -echo "project.yml embeds ${#REQUIRED_PLUGINS[@]} plugins" - -MISSING_PLUGINS=0 -for PLUGIN in "${REQUIRED_PLUGINS[@]}"; do - if [ ! -d "$PLUGINS_DIR/$PLUGIN" ]; then - echo "❌ ERROR: Missing plugin bundle: $PLUGIN" - MISSING_PLUGINS=1 - else - echo " ✅ $PLUGIN" - fi -done - -if [ "$MISSING_PLUGINS" -eq 1 ]; then - echo "❌ ERROR: One or more plugin bundles are missing" - exit 1 -fi -echo "✅ All bundled plugin bundles present" - -# Verify each plugin has a valid binary -MISSING_BINARIES=0 -for PLUGIN in "${REQUIRED_PLUGINS[@]}"; do - PLUGIN_NAME="${PLUGIN%.tableplugin}" - PLUGIN_BINARY="$PLUGINS_DIR/$PLUGIN/Contents/MacOS/$PLUGIN_NAME" - if [ ! -f "$PLUGIN_BINARY" ]; then - echo "❌ ERROR: Missing binary for plugin: $PLUGIN (expected $PLUGIN_BINARY)" - MISSING_BINARIES=1 - fi -done - -if [ "$MISSING_BINARIES" -eq 1 ]; then - echo "❌ ERROR: One or more plugin binaries are missing" - exit 1 -fi -echo "✅ All plugin binaries present" - -# Verify TableProPluginKit framework -PLUGINKIT_FRAMEWORK="$APP_BUNDLE/Contents/Frameworks/TableProPluginKit.framework" -if [ ! -d "$PLUGINKIT_FRAMEWORK" ]; then - echo "❌ ERROR: TableProPluginKit.framework not found at: $PLUGINKIT_FRAMEWORK" - exit 1 -fi -echo "✅ TableProPluginKit.framework present" - -# Verify code signature -echo "Verifying code signature..." -if codesign --verify --deep --strict "$APP_BUNDLE" 2>&1; then - SIGN_INFO=$(codesign -dvv "$APP_BUNDLE" 2>&1 | grep "Authority=" | head -1) - echo "✅ Code signature valid: $SIGN_INFO" -else - echo "❌ ERROR: Code signature verification failed" - codesign -dvv "$APP_BUNDLE" 2>&1 || true - exit 1 -fi - -# The release path always notarizes, so there is no legitimate "not yet" case: a warning here -# meant an unnotarized build passed verification and shipped. Checked by exit code rather than by -# grepping for a message, matching scripts/build-plugin.sh. -if ! xcrun stapler validate "$APP_BUNDLE"; then - echo "❌ ERROR: No notarization ticket stapled to $APP_BUNDLE" - exit 1 -fi -echo "✅ Notarization ticket stapled" - -# spctl is what a user's Mac runs. It accepts an unstapled but notarized bundle over the network, -# so it proves the app will launch but never that it will launch offline. The staple check above -# is what covers that, and both have to pass. -if ! spctl -a -vvv -t exec "$APP_BUNDLE" 2>&1 | grep -q "accepted"; then - echo "❌ ERROR: Gatekeeper rejects $APP_BUNDLE" - spctl -a -vvv -t exec "$APP_BUNDLE" 2>&1 || true - exit 1 -fi -echo "✅ Accepted by Gatekeeper" - -# Display info -echo "✅ Build verified successfully" -echo "Binary size: $(ls -lh "$BINARY_PATH" | awk '{print $5}')" -echo "App bundle size: $(du -sh "$APP_BUNDLE" | awk '{print $1}')" diff --git a/scripts/ci/verify-update-archive.sh b/scripts/ci/verify-update-archive.sh deleted file mode 100755 index 33ab4cd93d..0000000000 --- a/scripts/ci/verify-update-archive.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env bash -# -# Extracts an update archive the way Sparkle's installer does and checks the app inside still -# passes code signing, its stapled ticket and Gatekeeper. -# -# Usage: verify-update-archive.sh - -set -euo pipefail - -ARCHIVE="${1:?Usage: verify-update-archive.sh }" - -if [ ! -f "$ARCHIVE" ]; then - echo "::error::$ARCHIVE does not exist" - exit 1 -fi - -WORK="$(mktemp -d)" -trap 'rm -rf "$WORK"' EXIT - -# SUPipedUnarchiver pipes a .tar.xz into exactly this command, from Sparkle 2.8.1 through 2.10.0. -/usr/bin/tar -zxC "$WORK" < "$ARCHIVE" - -ENTRIES="$(find "$WORK" -mindepth 1 -maxdepth 1 | wc -l | tr -d ' ')" -APP="$(find "$WORK" -mindepth 1 -maxdepth 1 -name '*.app' -type d)" -if [ "$ENTRIES" -ne 1 ] || [ -z "$APP" ]; then - echo "::error::$ARCHIVE must hold one app bundle and nothing else" - ls -la "$WORK" - exit 1 -fi - -codesign --verify --deep --strict --verbose=2 "$APP" -xcrun stapler validate "$APP" - -if ! ASSESSMENT="$(spctl -a -vvv -t exec "$APP" 2>&1)"; then - echo "::error::Gatekeeper rejects the app extracted from $ARCHIVE" - echo "$ASSESSMENT" - exit 1 -fi -echo "$ASSESSMENT" -echo "✅ $(basename "$ARCHIVE") extracts to an app that passes codesign, stapler and spctl" diff --git a/scripts/create-dmg.sh b/scripts/create-dmg.sh deleted file mode 100755 index 97140fd075..0000000000 --- a/scripts/create-dmg.sh +++ /dev/null @@ -1,285 +0,0 @@ -#!/usr/bin/env bash -# Create a DMG installer with drag-and-drop installation window -# Uses create-dmg tool for reliable CI builds - -set -e - -# shellcheck source=lib/notarize.sh -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/notarize.sh" - -# Configuration -APP_NAME="TablePro" -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -ARCH="${2:-}" -case "$ARCH" in - arm64 | x86_64) ;; - *) - echo "Usage: create-dmg.sh [source_app]" >&2 - echo " source_app defaults to build/Release/${APP_NAME}-.app, which build-release.sh writes" >&2 - exit 1 - ;; -esac -VERSION="${1:-}" -if [ -z "$VERSION" ]; then - # Configs/Version.xcconfig is the single declaration of the app version. The literal that - # used to sit here as a default was 0.1.13, 66 releases behind, so a run without an explicit - # version produced a correctly built DMG under a badly wrong name. - VERSION=$(sed -n 's/^MARKETING_VERSION[[:space:]]*=[[:space:]]*//p' \ - "$REPO_ROOT/Configs/Version.xcconfig" | tr -d ' ') -fi -if [ -z "$VERSION" ]; then - # sed exits 0 when it matches nothing, so set -e does not cover this. - echo "❌ ERROR: MARKETING_VERSION missing from Configs/Version.xcconfig" >&2 - exit 1 -fi -SOURCE_APP="${3:-build/Release/${APP_NAME}-${ARCH}.app}" -DMG_NAME="${APP_NAME}-${VERSION}-${ARCH}.dmg" -VOLUME_NAME="${APP_NAME} ${VERSION}" -FINAL_DMG="build/Release/$DMG_NAME" - -# The hdiutil fallback below attaches a volume and writes a temp image. Without this, a failure -# anywhere between the attach and the detach leaves both behind, and the next run then fails on a -# volume name that is already mounted. -TEMP_DMG="" -MOUNT_DIR="" -cleanup_dmg() { - [ -n "$MOUNT_DIR" ] && [ -d "$MOUNT_DIR" ] && hdiutil detach "$MOUNT_DIR" -quiet 2> /dev/null || true - [ -n "$TEMP_DMG" ] && rm -f "$TEMP_DMG" || true -} -trap cleanup_dmg EXIT - -SIGN_IDENTITY="${SIGN_IDENTITY:-Developer ID Application: Dat Ngo Quoc (D7HJ5TFYCU)}" -NOTARIZE="${NOTARIZE:-false}" - -echo "📦 Creating DMG installer for $APP_NAME..." -echo " Version: $VERSION" -echo " Architecture: $ARCH" -echo " Source: $SOURCE_APP" - -# Verify source app exists -if [ ! -d "$SOURCE_APP" ]; then - echo "❌ ERROR: Source app not found: $SOURCE_APP" - exit 1 -fi - -# Ensure output directory exists -mkdir -p "build/Release" - -# Create a staging copy of the app with the correct name (TablePro.app) -# This ensures the DMG shows "TablePro.app" regardless of the source name -STAGING_APP="build/Release/${APP_NAME}.app" -if [ "$SOURCE_APP" != "$STAGING_APP" ]; then - echo "📋 Preparing $APP_NAME.app for DMG..." - rm -rf "$STAGING_APP" - cp -R "$SOURCE_APP" "$STAGING_APP" -fi - -# Get the app icon from the built app -APP_ICON="" -if [ -f "$STAGING_APP/Contents/Resources/AppIcon.icns" ]; then - APP_ICON="$STAGING_APP/Contents/Resources/AppIcon.icns" - echo " Using app icon: $APP_ICON" -fi - -# Create a temporary directory for DMG staging -DMG_STAGING="build/dmg-staging" -rm -rf "$DMG_STAGING" -mkdir -p "$DMG_STAGING" - -# Copy app to staging directory -cp -R "$STAGING_APP" "$DMG_STAGING/$APP_NAME.app" - -# Create an Applications alias (not symlink) with proper icon -# Using osascript to create a proper Finder alias -echo "📁 Creating Applications alias..." -osascript < /dev/null; then - # Create an Icon\r file with the icon data - cp "$APPS_ICON" "$DMG_STAGING/Applications/Icon"$'\r' 2>/dev/null || true - # Set custom icon flag - SetFile -a C "$DMG_STAGING/Applications" 2>/dev/null || true - fi -fi - -# Check if create-dmg tool is available (brew install create-dmg) -if command -v create-dmg &> /dev/null; then - echo "🔨 Using create-dmg tool..." - - # Remove existing DMG if present - rm -f "$FINAL_DMG" - - # Build create-dmg command with options - CREATE_DMG_ARGS=( - --volname "$VOLUME_NAME" - --window-pos 200 120 - --window-size 600 400 - --icon-size 80 - --icon "$APP_NAME.app" 150 190 - --icon "Applications" 450 190 - --hide-extension "$APP_NAME.app" - --no-internet-enable - # LZMA over the zlib default. Measured on 0.73.0: 27.4 MB -> 20.3 MB. - # hdiutil has read ULMO since macOS 10.15, five releases below the app's floor. - --format ULMO - ) - - # Add volume icon if available - if [ -n "$APP_ICON" ] && [ -f "$APP_ICON" ]; then - CREATE_DMG_ARGS+=(--volicon "$APP_ICON") - fi - - # Add background if exists - if [ -f ".dmg-assets/dmg-background.png" ]; then - CREATE_DMG_ARGS+=(--background ".dmg-assets/dmg-background.png") - echo " Using custom background" - fi - - # Create DMG from staging directory (which has both the app and Applications alias) - if ! create-dmg "${CREATE_DMG_ARGS[@]}" "$FINAL_DMG" "$DMG_STAGING"; then - echo "⚠️ create-dmg exited with non-zero (may be expected in CI due to AppleScript)" - # Check if DMG was still created - if [ -f "$FINAL_DMG" ]; then - echo " DMG was created despite exit code" - else - echo "❌ ERROR: DMG was not created" - rm -rf "$DMG_STAGING" - exit 1 - fi - fi - -else - echo "⚠️ create-dmg tool not found, using basic hdiutil method..." - echo " Install with: brew install create-dmg" - - # Calculate size needed for DMG - SIZE_MB=$(du -sm "$DMG_STAGING" | awk '{print $1}') - SIZE_MB=$((SIZE_MB + 50)) - - TEMP_DMG="build/Release/temp.dmg" - - echo "🔨 Creating temporary DMG ($SIZE_MB MB)..." - - # Create temporary DMG - hdiutil create -srcfolder "$DMG_STAGING" \ - -volname "$VOLUME_NAME" \ - -fs HFS+ \ - -fsargs "-c c=64,a=16,e=16" \ - -format UDRW \ - -size ${SIZE_MB}m \ - "$TEMP_DMG" - - # Mount the temporary DMG for customization - MOUNT_DIR="/Volumes/$VOLUME_NAME" - hdiutil attach "$TEMP_DMG" -readwrite -noverify -noautoopen - - # Wait for mount - sleep 2 - - # Set volume icon if available - if [ -n "$APP_ICON" ] && [ -f "$APP_ICON" ]; then - cp "$APP_ICON" "$MOUNT_DIR/.VolumeIcon.icns" - SetFile -a C "$MOUNT_DIR" 2>/dev/null || true - fi - - # Try AppleScript to set icon positions (may fail in CI, that's OK) - osascript </dev/null || echo " ⚠️ AppleScript layout skipped (headless environment)" -tell application "Finder" - tell disk "$VOLUME_NAME" - open - set current view of container window to icon view - set toolbar visible of container window to false - set statusbar visible of container window to false - set the bounds of container window to {100, 100, 700, 500} - set viewOptions to the icon view options of container window - set arrangement of viewOptions to not arranged - set icon size of viewOptions to 80 - set shows item info of viewOptions to false - set shows icon preview of viewOptions to true - - -- Position icons - delay 1 - set position of item "$APP_NAME.app" of container window to {150, 200} - set position of item "Applications" of container window to {450, 200} - - -- Force update - close - open - update without registering applications - delay 1 - close - end tell -end tell -EOF - - # Sync and unmount - sync - sleep 1 - hdiutil detach "$MOUNT_DIR" -force - - # Convert to compressed read-only DMG - hdiutil convert "$TEMP_DMG" \ - -format ULMO \ - -o "$FINAL_DMG" - - # Clean up temp DMG - rm -f "$TEMP_DMG" -fi - -# Clean up staging directories -rm -rf "$DMG_STAGING" -if [ "$SOURCE_APP" != "$STAGING_APP" ] && [ -d "$STAGING_APP" ]; then - rm -rf "$STAGING_APP" -fi - -# Verify final DMG -if [ ! -f "$FINAL_DMG" ]; then - echo "❌ ERROR: Failed to create DMG" - exit 1 -fi - -# Sign the DMG -echo "🔏 Signing DMG with: $SIGN_IDENTITY" -codesign -fs "$SIGN_IDENTITY" --timestamp "$FINAL_DMG" -if ! codesign --verify "$FINAL_DMG" 2>&1; then - echo "❌ ERROR: DMG signature verification failed" - exit 1 -fi -echo "✅ DMG signed" - -# Notarize the DMG (opt-in via NOTARIZE=true) -if [ "$NOTARIZE" = "true" ]; then - # "open", not "exec": a user opens a disk image, they do not launch it. - notarize_and_staple "$FINAL_DMG" open -fi - -# Get final size -FINAL_SIZE=$(du -h "$FINAL_DMG" | awk '{print $1}') - -echo "" -echo "✅ DMG created successfully!" -echo " 📍 Location: $FINAL_DMG" -echo " 📊 Size: $FINAL_SIZE" -echo "" -echo "🧪 Test the DMG:" -echo " open \"$FINAL_DMG\"" diff --git a/scripts/generate-whats-new.sh b/scripts/generate-whats-new.sh deleted file mode 100755 index fd22089612..0000000000 --- a/scripts/generate-whats-new.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Writes TablePro/Resources/WhatsNew.md from a version's CHANGELOG lead block, which is the same -# text the Sparkle feed carries. Generated rather than hand-written so the window and the update -# dialog cannot disagree, and so shipping a release does not depend on anyone remembering to edit -# a second file. -# -# Run from the release runbook, after finalizing CHANGELOG.md and before committing. -# -# Usage: scripts/generate-whats-new.sh - -VERSION="${1:?Usage: generate-whats-new.sh }" -REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" -OUTPUT="$REPO_ROOT/TablePro/Resources/WhatsNew.md" - -cd "$REPO_ROOT" - -TEMP=$(mktemp) -trap 'rm -f "$TEMP"' EXIT - -# --require-lead-block, not the feed's fallback. Without a lead block the fallback is the whole -# section, and 0.74.0's is 270 entries: not something to compile into the app bundle. A release -# with no highlights ships a short pointer instead. -if bash scripts/ci/extract-release-notes.sh "$VERSION" \ - --highlights-only --require-lead-block --out "$TEMP" >/dev/null 2>&1; then - { - printf '# TablePro %s\n\n' "$VERSION" - cat "$TEMP" - } > "$OUTPUT" -else - echo "No lead block for $VERSION; writing a pointer to the changelog instead" - printf '# TablePro %s\n\nThis release is fixes and refinements. Every entry is in the changelog.\n' \ - "$VERSION" > "$OUTPUT" -fi - -echo "Wrote $OUTPUT:" -cat "$OUTPUT" diff --git a/scripts/install-plugin-dev.sh b/scripts/install-plugin-dev.sh index 1056c7f18d..2517927ade 100755 --- a/scripts/install-plugin-dev.sh +++ b/scripts/install-plugin-dev.sh @@ -22,9 +22,7 @@ TARGET="${1:-ElasticsearchDriverPlugin}" BUNDLE="${TARGET}.tableplugin" DEST_DIR="${HOME}/Library/Application Support/TablePro/Plugins" -# The newest build, not whichever one the filesystem happened to list first. This repo carries -# several worktrees and each gets its own DerivedData, so `head -n 1` routinely installed a stale -# bundle from somebody else's checkout and the mismatch only showed up as a load failure later. +# Each worktree has its own DerivedData, so take the newest build. CANDIDATES="$(find "${HOME}/Library/Developer/Xcode/DerivedData/TablePro-"*/Build/Products/Debug \ -maxdepth 1 -name "${BUNDLE}" -print0 2>/dev/null | xargs -0 stat -f '%m %N' 2>/dev/null | sort -rn)" SRC="$(printf '%s' "${CANDIDATES}" | head -n 1 | cut -d' ' -f2-)" diff --git a/scripts/lib/macos.sh b/scripts/lib/macos.sh index c8254a1027..4fec055602 100644 --- a/scripts/lib/macos.sh +++ b/scripts/lib/macos.sh @@ -113,46 +113,6 @@ build_openssl() { echo "✅ OpenSSL $arch: $(du -h "$prefix/lib/libssl.a" | cut -f1) (libssl) $(du -h "$prefix/lib/libcrypto.a" | cut -f1) (libcrypto)" } -current_arch() { - [ -f "$1" ] || return 0 - lipo -info "$1" 2> /dev/null | sed 's/.*: //' | tr -s ' ' | sed 's/ $//' -} - -# Leaves Libs/.a at the requested architecture, which is what project.yml links. -prepare_arch_libs() { - local target_arch="$1" - shift - local lib unresolved=0 - - for lib in "$@"; do - # An exact match: a universal also contains the slice, and must not be linked as one. - if [ "$(current_arch "$LIBS_DIR/${lib}.a")" = "$target_arch" ]; then - continue - fi - - if [ -f "$LIBS_DIR/${lib}_${target_arch}.a" ]; then - cp "$LIBS_DIR/${lib}_${target_arch}.a" "$LIBS_DIR/${lib}.a" - continue - fi - - if [ ! -f "$LIBS_DIR/${lib}_universal.a" ]; then - echo "ERROR: neither $LIBS_DIR/${lib}_${target_arch}.a nor ${lib}_universal.a exists" >&2 - unresolved=1 - continue - fi - - if ! lipo "$LIBS_DIR/${lib}_universal.a" -thin "$target_arch" -output "$LIBS_DIR/${lib}.a"; then - echo "ERROR: could not extract the $target_arch slice from ${lib}_universal.a" >&2 - unresolved=1 - fi - done - - if [ "$unresolved" -ne 0 ]; then - echo " run scripts/download-libs.sh, or rebuild the library it names" >&2 - return 1 - fi -} - # The trap body is single-quoted so it reads BUILD_DIR when it fires, not when it is installed. # It replaces any EXIT trap already in place, so a script that needs to remove more than this # directory installs its own instead (build-freetds.sh does, for its tarball cache). diff --git a/scripts/lib/notarize.sh b/scripts/lib/notarize.sh deleted file mode 100644 index 76ebb9b65c..0000000000 --- a/scripts/lib/notarize.sh +++ /dev/null @@ -1,82 +0,0 @@ -#!/usr/bin/env bash -# Submit an artifact to Apple's notary service, staple the ticket, and prove it will open. -# Source this; it defines notarize_and_staple and sets nothing else. -# -# This existed three times at three rigor levels. build-plugin.sh stapled, validated and asked -# Gatekeeper; create-dmg.sh stapled and validated; build-release.sh only stapled, and it was the -# only one that fetched the notary log when a submission failed. So the app, the artifact users -# actually download, got the weakest checks and the DMG and plugins got the best diagnostics -# withheld. One implementation, at the strictest level all three reached between them. -# -# The two post-staple checks are not redundant. spctl asks Gatekeeper, which will fetch the -# ticket from Apple over the network, so an artifact that was notarized but never successfully -# stapled still passes: /Applications/Ghostty.app has no ticket and spctl accepts it. Only -# `stapler validate` proves the ticket is in the artifact, which is what an offline Mac needs. - -# The apple-signing action's "Configure notarization" step stores the Apple ID, team and -# app-specific password under this profile name, so --keychain-profile is the whole credential -# and passing --apple-id or --team-id alongside it is redundant. A local build with its own -# stored profile can override the name. -NOTARY_PROFILE="${NOTARY_PROFILE:-TablePro}" - -# Usage: notarize_and_staple [exec|open] -# -# The second argument is the Gatekeeper assessment type: "exec" for an application, "open" for a -# disk image or a plugin bundle, which a user opens rather than launches. -notarize_and_staple() { - local path="${1:?notarize_and_staple needs a path}" - local assessment="${2:-exec}" - local name - name="$(basename "$path")" - - [ -e "$path" ] || { echo "FATAL: nothing to notarize at $path" >&2; return 1; } - - # notarytool only accepts an archive, so a bundle is zipped to a throwaway path. The ticket - # is then stapled into the bundle itself, not the zip: without that every user needs a live - # round trip to Apple on first load and an offline Mac never gets one. Stapling rewrites the - # artifact, so any distribution zip and its checksum must be produced after this returns. - local submission="$path" scratch="" - if [ -d "$path" ]; then - scratch="$(mktemp -d)" - submission="$scratch/${name}.zip" - ditto -c -k --keepParent "$path" "$submission" - fi - - echo "Submitting $name for notarization..." - # Assigned before the substitution, then overwritten on failure. Written the other way round, - # `status=$?` reads the exit status of the assignment, which is always 0, so every failure - # took the success branch and the log fetch below was unreachable. - local submit_status=0 output="" - output="$(xcrun notarytool submit "$submission" --keychain-profile "$NOTARY_PROFILE" --wait 2>&1)" || submit_status=$? - echo "$output" - [ -z "$scratch" ] || rm -rf "$scratch" - - # Both checks, because notarytool has shipped versions that exit 0 on a rejected submission. - if [ "$submit_status" -ne 0 ] || ! echo "$output" | grep -q "status: Accepted"; then - echo "FATAL: notarization failed for $name" >&2 - local submission_id - submission_id="$(echo "$output" | grep "id:" | head -1 | awk '{print $2}')" - if [ -n "$submission_id" ]; then - echo "Notary log for $submission_id:" >&2 - xcrun notarytool log "$submission_id" --keychain-profile "$NOTARY_PROFILE" >&2 2>&1 || true - fi - return 1 - fi - - echo "Stapling the ticket into $name..." - xcrun stapler staple "$path" || { echo "FATAL: stapling failed for $name" >&2; return 1; } - - # Stapling can report success and still leave no usable ticket. - xcrun stapler validate "$path" || { echo "FATAL: the stapled ticket did not validate for $name" >&2; return 1; } - - # spctl is what a user's Mac runs. A pass here is the only proof the artifact will open. - local assessment_output - if ! assessment_output="$(spctl -a -vvv -t "$assessment" --context context:primary-signature "$path" 2>&1)" \ - || ! echo "$assessment_output" | grep -q "accepted"; then - echo "FATAL: Gatekeeper still rejects $name after notarization" >&2 - echo "$assessment_output" >&2 - return 1 - fi - - echo "$name notarized, stapled and accepted by Gatekeeper" -} diff --git a/scripts/release-all-plugins.sh b/scripts/release-all-plugins.sh deleted file mode 100755 index f35a323662..0000000000 --- a/scripts/release-all-plugins.sh +++ /dev/null @@ -1,83 +0,0 @@ -#!/usr/bin/env bash -# Trigger a bulk re-release of all registry plugins for a given PluginKit version. -# -# Usage: ./scripts/release-all-plugins.sh -# Example: ./scripts/release-all-plugins.sh 14 -# -# Reads the latest tag for each plugin, bumps the patch version, and pairs the -# NEW version with the given pluginKitVersion, then fires one workflow_dispatch -# on build-plugin.yml so all plugins build in parallel as a single matrix run. -# -# An ABI bump must publish fresh binaries at a NEW release tag. Reusing the -# existing tag overwrites that release's assets, which breaks the previous ABI's -# consumers and serves stale copies from the GitHub release CDN. -# -# Prerequisites: gh CLI authenticated, run from repo root. - -set -euo pipefail - -if [ $# -lt 1 ]; then - echo "Usage: $0 " >&2 - exit 1 -fi - -PKV="$1" - -# Derived from .github/plugin-registry.json rather than restated here. This used to be two -# hand-maintained arrays that had to agree with the workflow, the registry and each other, which is -# three copies of one list and no way to tell when they drift apart. -# -# A bulk ABI re-release covers the registry-only plugins. The ones marked `bundled` ship inside the -# app and their binaries ride with the next app release, so re-publishing them here would put a -# second copy in the registry for no one. -PLUGINS=() -while IFS= read -r PLUGIN; do - PLUGINS+=("$PLUGIN") -done < <(python3 -c ' -import json, pathlib -manifest = pathlib.Path(".github/plugin-registry.json") -for slug, entry in sorted(json.loads(manifest.read_text())["plugins"].items()): - if not entry["bundled"]: - print(slug) -') - -if [ "${#PLUGINS[@]}" -eq 0 ]; then - echo "ERROR: no registry-only plugins found in .github/plugin-registry.json" >&2 - exit 1 -fi - -TAG_LIST="" -FIRST=true -echo "Resolving next release version for each plugin (PluginKit $PKV):" -for PLUGIN in "${PLUGINS[@]}"; do - LATEST_TAG=$(git ls-remote --tags --refs origin "plugin-${PLUGIN}-v*" \ - | sed 's#.*/##' | sort -V | tail -1) - if [ -z "$LATEST_TAG" ]; then - echo " WARNING: No remote tag found for plugin-${PLUGIN}-v*. Skipping." - continue - fi - LATEST_VER="${LATEST_TAG#plugin-"${PLUGIN}"-v}" - NEW_TAG="plugin-${PLUGIN}-v${LATEST_VER%.*}.$(( ${LATEST_VER##*.} + 1 ))" - PAIR="${NEW_TAG}:${PKV}" - if [ "$FIRST" = true ]; then - TAG_LIST="$PAIR" - FIRST=false - else - TAG_LIST="${TAG_LIST},${PAIR}" - fi - echo " $PAIR" -done - -if [ -z "$TAG_LIST" ]; then - echo "ERROR: No plugin tags found." >&2 - exit 1 -fi - -echo "" -echo "Dispatching build-plugin.yml with PluginKit version $PKV" -echo "" - -gh workflow run build-plugin.yml --field "tags=$TAG_LIST" - -REPO=$(gh repo view --json nameWithOwner -q .nameWithOwner) -echo "Dispatched. Monitor at: https://github.com/${REPO}/actions/workflows/build-plugin.yml" diff --git a/scripts/release-plugin-for-shipped-app.sh b/scripts/release-plugin-for-shipped-app.sh deleted file mode 100755 index 930b97c849..0000000000 --- a/scripts/release-plugin-for-shipped-app.sh +++ /dev/null @@ -1,87 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Publishes one registry plugin built against a SHIPPED app release rather than against main. -# -# Users do not all run the newest app. The registry keeps binaries for the most recent PluginKit -# versions per plugin, and PluginManager only installs a binary whose kit is at or below the app's -# own, so a plugin published only from main is uninstallable for everyone who has not updated yet. -# That is not hypothetical: as of 2026-09-13 the oldest kit published anywhere in the registry is -# 21, while v0.65.0 through v0.70.0 ship kit 19 and v0.71.0 ships kit 20, so a user on any of them -# resolves no binary for any of the 23 registry plugins. -# -# This builds the plugin at the app tag, so the binary links against that release's PluginKit and -# its Info.plist declares that release's TableProPluginKitVersion. The two have to agree: stamping -# a binary built from main with an older number makes the older app accept it and then fail -# Bundle.loadAndReturnError, which is what #1917 and 0.49.0 were. -# -# Usage: -# scripts/release-plugin-for-shipped-app.sh [appTag] -# -# pluginTag a tag from .github/plugin-registry.json, e.g. plugin-mongodb-v1.0.45 -# appTag the shipped release to build against, e.g. v0.73.0 -# Defaults to the newest published, non-prerelease v* GitHub release. - -PLUGIN_TAG="${1:?Usage: release-plugin-for-shipped-app.sh [appTag]}" -APP_TAG="${2:-}" - -if ! command -v gh >/dev/null 2>&1; then - echo "error: the GitHub CLI (gh) is required" >&2 - exit 1 -fi - -REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" - -# The slug is what the registry keys on: plugin--v. -SLUG="${PLUGIN_TAG#plugin-}" -SLUG="${SLUG%-v*}" -if ! python3 -c " -import json, sys -manifest = json.load(open('$REPO_ROOT/.github/plugin-registry.json'))['plugins'] -sys.exit(0 if '$SLUG' in manifest else 1) -"; then - echo "error: '$SLUG' is not a key in .github/plugin-registry.json" >&2 - echo " known slugs:" >&2 - python3 -c " -import json -for slug in sorted(json.load(open('$REPO_ROOT/.github/plugin-registry.json'))['plugins']): - print(' ', slug) -" >&2 - exit 1 -fi - -if [ -z "$APP_TAG" ]; then - APP_TAG=$(gh release list --limit 100 --json tagName,isPrerelease,publishedAt \ - -q '[.[] | select(.tagName|startswith("v")) | select(.isPrerelease==false)] - | sort_by(.publishedAt) | last | .tagName') - if [ -z "$APP_TAG" ] || [ "$APP_TAG" = "null" ]; then - echo "error: could not resolve the newest published app release; pass one explicitly" >&2 - exit 1 - fi - echo "No app tag given, using the newest published release: $APP_TAG" -fi - -if ! git -C "$REPO_ROOT" rev-parse --verify "$APP_TAG^{commit}" >/dev/null 2>&1; then - echo "error: $APP_TAG is not a ref in this checkout; fetch it first" >&2 - exit 1 -fi - -if git -C "$REPO_ROOT" cat-file -e "$APP_TAG:Configs/PluginKit.xcconfig" 2>/dev/null; then - KIT=$(git -C "$REPO_ROOT" show "$APP_TAG:Configs/PluginKit.xcconfig" \ - | sed -n 's/^TABLEPRO_PLUGINKIT_VERSION *= *\([0-9][0-9]*\) *$/\1/p') -else - # A tag cut before Configs/PluginKit.xcconfig existed declares the kit in Swift. - KIT=$(git -C "$REPO_ROOT" show "$APP_TAG:TablePro/Core/Plugins/PluginManager.swift" \ - | sed -n 's/.*currentPluginKitVersion = \([0-9]*\).*/\1/p' | head -1) -fi -if [ -z "$KIT" ]; then - echo "error: could not read the PluginKit version out of $APP_TAG" >&2 - exit 1 -fi - -echo "Publishing $PLUGIN_TAG built at $APP_TAG (PluginKit $KIT)" -gh workflow run build-plugin.yml \ - -f "tags=${PLUGIN_TAG}:${KIT}" \ - -f "baseRef=${APP_TAG}" - -echo "Dispatched. Watch it with: gh run watch \$(gh run list --workflow=build-plugin.yml --limit 1 --json databaseId -q '.[0].databaseId')" diff --git a/scripts/release/ExportOptions.plist b/scripts/release/ExportOptions.plist new file mode 100644 index 0000000000..204ba1bd5c --- /dev/null +++ b/scripts/release/ExportOptions.plist @@ -0,0 +1,19 @@ + + + + + method + developer-id + signingStyle + manual + signingCertificate + Developer ID Application + iCloudContainerEnvironment + Production + provisioningProfiles + + com.TablePro + TablePro Developer ID + + + diff --git a/scripts/release/build.sh b/scripts/release/build.sh index 373437f252..702e8265ee 100755 --- a/scripts/release/build.sh +++ b/scripts/release/build.sh @@ -139,20 +139,10 @@ build_plugins() { local products="$DERIVED/Build/Products/Release" targets="$WORK/targets" kit="$WORK/kit" target plugin extra=() rm -rf "$OUT/plugins" "$OUT/plugins-dSYMs" [ "$SIGNED" = false ] || extra=(OTHER_CODE_SIGN_FLAGS=--timestamp) + python3 scripts/release/registry.py targets > "$targets" xcodebuild_project build-plugins build -scheme RegistryPlugins ${extra[@]+"${extra[@]}"} - # The publish set: the registry entries the app does not bundle. - python3 - .github/plugin-registry.json > "$targets" << 'PYTHON' -import json -import sys - -with open(sys.argv[1], encoding="utf-8") as handle: - plugins = json.load(handle)["plugins"] -print("\n".join(sorted(entry["target"] for entry in plugins.values() if not entry.get("bundled")))) -PYTHON - [ -s "$targets" ] || die ".github/plugin-registry.json publishes no plugin" - mkdir -p "$OUT/plugins" "$OUT/plugins-dSYMs" while IFS= read -r target; do plugin="$products/$target.tableplugin" diff --git a/scripts/release/notarize.sh b/scripts/release/notarize.sh new file mode 100755 index 0000000000..3ed7c70426 --- /dev/null +++ b/scripts/release/notarize.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Usage: notarize.sh ... +# One notary submission for every path, then on each: a stapled ticket, stapler validate and spctl. +# A path is a bundle, or a single disk image on its own. Credentials come from the TablePro keychain +# profile that .github/actions/apple-signing stores. + +PROFILE="TablePro" + +usage() { + echo "Usage: notarize.sh ..." >&2 + exit 2 +} + +die() { + echo "notarize.sh: $*" >&2 + exit 1 +} + +[ $# -gt 0 ] || usage +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +PATHS=() +for path in "$@"; do + path="${path%/}" + [ -e "$path" ] || die "nothing to notarize at $path" + if [ -f "$path" ] && [ $# -gt 1 ]; then + die "$path is a file; a file is notarized on its own" + fi + PATHS+=("$path") +done + +if [ -f "${PATHS[0]}" ]; then + submission="${PATHS[0]}" +else + mkdir "$WORK/submission" + for path in "${PATHS[@]}"; do + name="$(basename "$path")" + [ ! -e "$WORK/submission/$name" ] || die "two paths are named $name" + ditto "$path" "$WORK/submission/$name" + done + submission="$WORK/notarization.zip" + ditto -c -k --keepParent "$WORK/submission" "$submission" +fi + +echo "Submitting $(basename "$submission") (${#PATHS[@]} item(s)) for notarization" +result="$WORK/result.json" +status=0 +xcrun notarytool submit "$submission" --keychain-profile "$PROFILE" --wait --timeout 30m \ + --output-format json > "$result" || status=$? +cat "$result" +echo +id="$(plutil -extract id raw -o - "$result" 2> /dev/null || true)" +state="$(plutil -extract status raw -o - "$result" 2> /dev/null || true)" +# notarytool has exited 0 on a rejected submission, so the status is read as well. +if [ "$status" -ne 0 ] || [ "$state" != "Accepted" ]; then + if [ -n "$id" ]; then + echo "Notary log for $id:" >&2 + xcrun notarytool log "$id" --keychain-profile "$PROFILE" >&2 || true + fi + die "the submission ended '${state:-without a status}' (notarytool exit $status)" +fi + +for path in "${PATHS[@]}"; do + name="$(basename "$path")" + xcrun stapler staple "$path" || die "stapling $name failed" + xcrun stapler validate "$path" || die "the ticket stapled to $name does not validate" + case "$path" in + *.app) assessment=(-t exec) ;; + *) assessment=(-t open --context context:primary-signature) ;; + esac + output="$(spctl -a -vvv "${assessment[@]}" "$path" 2>&1)" || die "Gatekeeper rejects $name: $output" + grep -q 'source=Notarized Developer ID' <<< "$output" \ + || die "Gatekeeper accepts $name, but not as notarized Developer ID code: $output" + echo "$name: notarized, stapled and accepted by Gatekeeper" +done diff --git a/scripts/release/package.sh b/scripts/release/package.sh index 6c01ba5ac0..dd93b90ed8 100755 --- a/scripts/release/package.sh +++ b/scripts/release/package.sh @@ -3,6 +3,7 @@ set -euo pipefail # Usage: package.sh app|plugins [--unsigned] # Turns what scripts/release/build.sh wrote to build/release into the release assets in dist/. +# BUILD is the build number build.sh got; unset, it is 1, the CURRENT_PROJECT_VERSION project.yml sets. usage() { echo "Usage: package.sh app|plugins [--unsigned]" >&2 @@ -34,6 +35,8 @@ for arg in "$@"; do done case "$KIND" in app | plugins) ;; *) usage ;; esac case "$VERSION" in '' | *[!0-9A-Za-z.-]*) usage ;; esac +BUILD="${BUILD:-1}" +case "$BUILD" in '' | *[!0-9]*) echo "package.sh: BUILD must be an integer, got '$BUILD'" >&2; exit 2 ;; esac cd "$(dirname "${BASH_SOURCE[0]}")/../.." @@ -80,7 +83,7 @@ package_app() { } package_plugins() { - local plugin target symbols="$DIST/TablePro-$VERSION-Plugins-dSYMs.zip" count=0 + local plugin target symbols="$DIST/TablePro-$VERSION-Plugins-dSYMs.zip" count=0 kit [ -d "$OUT/plugins" ] || die "no $OUT/plugins; run scripts/release/build.sh plugins first" [ -d "$OUT/plugins-dSYMs" ] || die "no $OUT/plugins-dSYMs; run scripts/release/build.sh plugins first" mkdir -p "$DIST" @@ -94,6 +97,13 @@ package_plugins() { done [ "$count" -gt 0 ] || die "no plugins in $OUT/plugins" + # The feed job describes these zips again after download; this fails the build that made them instead. + kit="$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' \ + Configs/PluginKit.xcconfig)" + [ -n "$kit" ] || die "Configs/PluginKit.xcconfig sets no TABLEPRO_PLUGINKIT_VERSION" + python3 scripts/release/registry.py describe --zips "$DIST" --version "$VERSION" --build "$BUILD" \ + --kit "$kit" --metadata .github/plugin-registry.json --out "$WORK/entries.json" + ditto -c -k --keepParent "$OUT/plugins-dSYMs" "$symbols" echo "Packaged $count plugins as $DIST/-$VERSION.zip" ls -lh "$symbols"