Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c4c8227
fix: keep TEXT containing NUL intact in the WebAssembly engine
zknpr Oct 23, 2026
cd895d4
feat(native): snapshot through the runtime's statement-scoped vacuumInto
zknpr Oct 23, 2026
e893ace
docs(changelog): 1.8.3 notes for the scoped snapshot and NUL TEXT
zknpr Oct 23, 2026
cf64899
refactor: accept any decoder with decode() in the NUL containment helper
zknpr Oct 23, 2026
23ad138
docs(changelog): file the unreleased notes under Unreleased
zknpr Oct 23, 2026
e13819b
test(native): run the ATTACH fuzz on all five native targets
zknpr Oct 23, 2026
d733e63
fix: store TEXT with unpaired surrogates byte for byte in WebAssembly
zknpr Oct 23, 2026
c6f49f8
fix: give WITHOUT ROWID keys containing NUL the right read-only reason
zknpr Oct 23, 2026
12f2572
docs(changelog): note the five-target native fuzz lane
zknpr Oct 23, 2026
2d65ac9
chore(natives): re-pin the runtime to txiki.js b45ac83
zknpr Oct 24, 2026
b916ece
chore: prepare SQLite Explorer 1.8.3
zknpr Oct 24, 2026
8cbdef3
fix(security): show database text in notifications without links
zknpr Oct 24, 2026
a958ace
fix: send undo and redo errors through plainErrors
zknpr Oct 24, 2026
79063ca
fix: keep a DOMException intact when disarming its message
zknpr Oct 24, 2026
92a436c
test(native): bind every placeholder in the fuzz's script attaches
zknpr Oct 24, 2026
9825dd1
fix: break every "]" in notification text and guard the remaining sur…
zknpr Oct 24, 2026
20ff91a
refactor: type the provider method lists and restore a doc comment
zknpr Oct 24, 2026
e02caf9
ci: run the native fuzz when its worker client or toolchain changes
zknpr Oct 24, 2026
4bc59a8
docs(changelog): correct the 1.8.3 notes
zknpr Oct 24, 2026
448f5aa
fix(vfs): keep TEXT containing NUL out of the external text editor
zknpr Oct 24, 2026
52fa39f
fix: disarm undo labels and input validation messages
zknpr Oct 24, 2026
6357e4f
style: tidy a doc comment and a test message
zknpr Oct 24, 2026
1acd052
ci: run the native fuzz for its remaining inputs and pull requests to…
zknpr Oct 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions .github/workflows/native-fuzz.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: Native fuzz

# Differential ATTACH/DETACH/VACUUM INTO fuzz (tests/fuzz/) against the
# bundled native worker on all five shipped txiki.js targets (CI's native lanes
# run on Linux x64 only). Opt-in: manual, or on dev pushes and pull requests to
# dev that change the harness, the runtimes, the worker client it drives, the
# toolchain or this workflow.
on:
workflow_dispatch:
push:
branches: [dev]
paths:
- 'tests/fuzz/**'
- 'tests/unit/vscode_mock_setup.ts'
- 'tests/unit/mocks/**'
- 'natives/**'
- 'src/nativeWorker.ts'
- 'package.json'
- 'package-lock.json'
- '.nvmrc'
- '.github/workflows/native-fuzz.yml'
pull_request:
branches: [dev]
paths:
- 'tests/fuzz/**'
- 'tests/unit/vscode_mock_setup.ts'
- 'tests/unit/mocks/**'
- 'natives/**'
- 'src/nativeWorker.ts'
- 'package.json'
- 'package-lock.json'
- '.nvmrc'
- '.github/workflows/native-fuzz.yml'

concurrency:
group: native-fuzz-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
native-fuzz:
name: Native fuzz (${{ matrix.target }})
runs-on: ${{ matrix.runner }}
# Above the harness's own 60-minute test timeout, so a slow runner fails
# with the test's report rather than a bare cancellation.
timeout-minutes: 75
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-24.04
target: x86_64-linux-gnu
- runner: ubuntu-24.04-arm
target: aarch64-linux-gnu
- runner: windows-latest
target: x86_64-windows
# macos-13 is retired; macos-15-intel matches the arm64 lane's OS.
- runner: macos-15-intel
target: x86_64-macos
- runner: macos-15
target: aarch64-macos
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false

- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version-file: .nvmrc
cache: npm

- name: Install dependencies
run: npm ci

# FUZZ_EXPECT_TARGET makes the harness fail unless it resolved and ran
# natives/<target> (no fallback to another runtime or arch). The worker
# logs every refusal, so the summary lines are copied to the job summary.
- name: Native ATTACH fuzz
shell: bash
env:
FUZZ_EXPECT_TARGET: ${{ matrix.target }}
run: |
set -o pipefail
npm run test:native-fuzz 2>&1 | tee native-fuzz.log

- name: Summary
if: always()
shell: bash
env:
RUNNER_LABEL: ${{ matrix.runner }}
FUZZ_EXPECT_TARGET: ${{ matrix.target }}
run: |
{
echo "### ${RUNNER_LABEL} (expected ${FUZZ_EXPECT_TARGET})"
echo '```'
grep -F '[native fuzz]' native-fuzz.log || echo 'no [native fuzz] lines: the harness did not reach the worker'
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
# Changelog

## 1.8.3

### Security

- Show text that comes from a database file without links in VS Code notifications. VS Code turns `[label](command:…)` in an extension's notification or progress title into a link that runs that command, with arguments taken from the link, when clicked. A crafted database could put such a link into what SQLite Explorer showed: an error message, for example from exporting a view whose column name is a link, or a table, view or file name. These texts are now shown without links. The same applies to errors that VS Code shows itself, such as a failed save, which VS Code before 1.141 also turns into links.
- Narrow the native runtime's exception for export snapshots to the one statement that writes the snapshot, on the thread running it. In 1.8.2 the exception covered the connection while an export ran; the worker's statement check and its one-request-at-a-time order kept other SQL from using it.

### Fixes

- Keep TEXT values that contain a NUL character intact. The WebAssembly engine cut such a value short at the first NUL when writing it, for example from an import, and when reading it in the query console; the native engine did the same on some reads. Both engines now read and write the whole value. The grid still shows these cells as bytes on both engines; in tables with more than 999 columns it checks only the key columns, so other columns show such a value as text. A row whose WITHOUT ROWID primary key contains NUL stays read-only, and the reason now says so.
- Store TEXT that contains an unpaired surrogate, such as half of an emoji left by a truncated string, whole in the WebAssembly engine. When a multibyte character followed it, the end of the value was cut off. The stored bytes now match the native engine's.
- When the native engine fails to write a copy of the database, for Save As or an export, the error now gives SQLite's message instead of the generic text for its error code, such as "SQL logic error".

### Maintenance

- Update the native runtime and the WebAssembly build to the fork commits with these fixes (txiki.js `b45ac83`, sql.js `35ac205`). The runtime also resets a statement whose rows fail to convert, instead of leaving it running, and checks every step of that conversion; SQLite Explorer's worker finalizes its statements, so it was not affected.
- Run the native ATTACH, DETACH and VACUUM INTO fuzz on all five native targets in CI (`native-fuzz.yml`). Its run against this release's runtime generated 40,000 statements, 39,990 of them distinct, and found no attached database or file write on any target.

## 1.8.2

### Security
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,7 @@ Settings in `package.json` → `contributes.configuration`:
### Build
- **WASM not found**: Run `node scripts/build.mjs` if `assets/sqlite3.wasm` is missing
- **Browser vs Node**: Check `import.meta.env.VSCODE_BROWSER_EXT` for environment-specific code
- **Query-plan runtime**: `src/runtime/query-plan.c` bounds plan compilation and extraction on the primary connection. `node scripts/build-query-plan.mjs` builds the five native libraries and the WASM runtime from that file and pinned sql.js commit `3181cc5ee888d3251228d3995c32bf9ea03c3240` and SQLite amalgamation 3.54.1. It requires Emscripten 5.0.0 and the official Zig 0.16.0 release (`zig cc --version` must report `clang version 21.1.0`; distribution builds such as Homebrew's are refused); use `--emcc PATH` and optionally `--sqljs-source GIT_CHECKOUT`. The source checkout is read through `git archive` at the pin. `scripts/refresh-sqljs.mjs` delegates to this builder. Native Explain uses the primary connection's signalled `Database.allAsync` so cancel RPCs remain deliverable without losing TEMP or pending transaction state. `scripts/refresh-natives.mjs` pins the tested `master` commit, Actions run, and hashes for all five native targets.
- **Query-plan runtime**: `src/runtime/query-plan.c` bounds plan compilation and extraction on the primary connection. `node scripts/build-query-plan.mjs` builds the five native libraries and the WASM runtime from that file and pinned sql.js commit `35ac205b7f9470d9aa7696ceb9cdab5dec4b6ed3` and SQLite amalgamation 3.54.1. It requires Emscripten 5.0.0 and the official Zig 0.16.0 release (`zig cc --version` must report `clang version 21.1.0`; distribution builds such as Homebrew's are refused); use `--emcc PATH` and optionally `--sqljs-source GIT_CHECKOUT`. The source checkout is read through `git archive` at the pin. `scripts/refresh-sqljs.mjs` delegates to this builder. Native Explain uses the primary connection's signalled `Database.allAsync` so cancel RPCs remain deliverable without losing TEMP or pending transaction state. `scripts/refresh-natives.mjs` pins the tested `master` commit, Actions run, and hashes for all five native targets.
- **Runtime artifact gate**: `vendor/query-plan-manifest.json` records source and output hashes. Normal builds verify it before bundling. Regenerate runtime changes with the builder; do not edit native libraries, WASM/glue, or the manifest by hand. `node scripts/build-query-plan.mjs --check` verifies all copies.

## Extension Identifiers
Expand Down
Binary file modified natives/aarch64-linux-gnu/tjs
Binary file not shown.
Binary file modified natives/aarch64-macos/tjs
Binary file not shown.
76 changes: 10 additions & 66 deletions natives/native-worker.js
Original file line number Diff line number Diff line change
Expand Up @@ -147,8 +147,8 @@ function leadingStatementVerb(sql) {
* ToString: an array ['ATTACH ...'] would otherwise compile unchecked.
*
* The bundled runtime enforces the same rule in C: every connection installs
* an authorizer at open that refuses ATTACH and DETACH (see runVacuumInto for
* the one permitted path), so a future SQLite tokenizer change that this
* an authorizer at open that refuses ATTACH and DETACH (runVacuumInto uses
* the runtime's one exception), so a future SQLite tokenizer change that this
* mirror misses still cannot attach a file. This check stays in front for its
* specific error code.
*/
Expand Down Expand Up @@ -199,7 +199,7 @@ function isPathAuthorityRefusal(error) {

/**
* Error text for the IPC reply. The authorizer's refusal gets its code, and
* every other failure it was wrapped with (a failed permit withdrawal, a
* every other failure it was wrapped with (a failed savepoint cleanup, a
* failed deadline cleanup, a cancellation) is kept rather than replaced.
*/
function describeWorkerError(error) {
Expand Down Expand Up @@ -674,13 +674,13 @@ function executeInterruptibleAsyncOperation(requestId, timeoutMs, operation) {
* VACUUM must run on the interruptible connection and outside transactions.
*
* VACUUM INTO attaches its destination, and the runtime's authorizer refuses
* every ATTACH except the one path a connection permits. The permit names
* exactly this host-chosen snapshot and is withdrawn when the VACUUM settles,
* so SQL-level VACUUM INTO any other path (which would create an arbitrary
* file holding a copy of the database) stays refused.
* every ATTACH, so SQL-level VACUUM INTO (which would create an arbitrary file
* holding a copy of the database) is refused. vacuumInto is the runtime's one
* exception: it runs a fixed `VACUUM main INTO ?` and accepts this host-chosen
* destination only for that statement, on the thread running it.
*/
function runVacuumInto(asyncDatabase, database, snapshotPath, signal) {
if (!asyncDatabase || typeof asyncDatabase.run !== 'function') {
if (!asyncDatabase || typeof asyncDatabase.vacuumInto !== 'function') {
return Promise.reject(
new Error('Interruptible native database snapshots are unavailable in this runtime')
);
Expand All @@ -693,47 +693,8 @@ function runVacuumInto(asyncDatabase, database, snapshotPath, signal) {
if (typeof snapshotPath !== 'string' || snapshotPath.length === 0) {
return Promise.reject(new Error('A native export snapshot path is required'));
}
try {
asyncDatabase.permitAttach(snapshotPath);
} catch (error) {
return Promise.reject(error);
}
const withdrawPermit = () => {
try {
asyncDatabase.permitAttach(null);
} catch (error) {
// A closing connection ('Invalid DB') takes its permit with it. Any
// other failure leaves the permit installed, so the connection is
// closed (no handle keeps authorizing the path, even if a cancellation
// wrapper later hides this error) and the failure is reported. A
// reworded closing message lands here too, which fails closed.
if (error?.message === 'Invalid DB') return;
releaseAsyncDatabase(asyncDatabase);
throw error;
}
};
// Through a promise, so a synchronous throw from run also withdraws the permit.
return new Promise(resolve => resolve(
asyncDatabase.run('VACUUM main INTO ?', [snapshotPath], { signal })
)).then(
result => {
withdrawPermit();
return result;
},
error => {
try {
withdrawPermit();
} catch (withdrawError) {
throw new AggregateError(
[error, withdrawError],
`${describeNativeError(error, 'VACUUM INTO failed')}; ` +
`withdrawing the ATTACH permit also failed: ` +
`${describeNativeError(withdrawError, 'Unknown permit error')}`
);
}
throw error;
}
);
// Through a promise, so a synchronous throw is reported as a rejection.
return new Promise(resolve => resolve(asyncDatabase.vacuumInto(snapshotPath, { signal })));
}

/** Abort one active async operation without affecting queued or future work. */
Expand All @@ -747,23 +708,6 @@ function cancelOperation(correlationId) {
return true;
}

/**
* Take an AsyncDatabase out of service now. For the shared connection this
* goes through closeAsyncDatabase, which clears `asyncDb` synchronously, so
* later reads fall back to the primary connection instead of a closed handle.
*/
function releaseAsyncDatabase(connection) {
// Any other connection's close may be synchronous or throw; it is reported
// here rather than replacing the failure the caller is about to throw.
const closing = connection === asyncDb
? closeAsyncDatabase()
: new Promise(resolve => resolve(connection.close()));
closing.catch(closeError => {
console.error('[native-worker] closing the snapshot connection failed:',
describeNativeError(closeError, 'Unknown close error'));
});
}

function closeAsyncDatabase() {
return (async () => {
const connection = asyncDb;
Expand Down
Binary file modified natives/x86_64-linux-gnu/tjs
Binary file not shown.
Binary file modified natives/x86_64-macos/tjs
Binary file not shown.
Binary file modified natives/x86_64-windows/tjs.exe
Binary file not shown.
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"name": "sqlite-explorer",
"displayName": "SQLite Explorer",
"description": "A powerful SQLite database viewer and editor for VS Code",
"version": "1.8.2",
"version": "1.8.3",
"publisher": "zknpr",
"license": "MIT",
"repository": {
Expand Down Expand Up @@ -282,7 +282,8 @@
"test:gui:interactions": "node tests/gui/interaction-details.mjs",
"test:gui:contention": "node tests/gui/native-contention.mjs",
"test:gui:uploads": "node tests/gui/drop-upload.mjs",
"test:native-smoke": "node --import tsx --test tests/unit/native_real_smoke.test.ts"
"test:native-smoke": "node --import tsx --test tests/unit/native_real_smoke.test.ts",
"test:native-fuzz": "node --import tsx --test tests/fuzz/native_attach_fuzz.test.ts"
},
"devDependencies": {
"@types/node": "^26.6.5",
Expand Down
2 changes: 1 addition & 1 deletion scripts/build-query-plan.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { fileURLToPath } from 'node:url';
import { createPinnedArtifactPolicy } from './lib/pinned-artifacts.mjs';

const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const sourceCommit = '3181cc5ee888d3251228d3995c32bf9ea03c3240';
const sourceCommit = '35ac205b7f9470d9aa7696ceb9cdab5dec4b6ed3';
const manifestPath = 'vendor/query-plan-manifest.json';
const sourceFiles = ['src/runtime/query-plan.c', 'scripts/build-query-plan.mjs'];
const nativeTargets = [
Expand Down
18 changes: 9 additions & 9 deletions scripts/refresh-natives.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
* An already-downloaded artifact can be verified without network access only
* when its complete pinned provenance is supplied explicitly:
* node scripts/refresh-natives.mjs --from /path/to/run-artifacts \
* --run 39983760340 --branch master \
* --commit 0353db5719c80ee69e0f400f4dfcd2256a034ee0
* --run 40086152581 --branch master \
* --commit b45ac83ec0d7809c413afcf4091d7ecf12a46452
*/

import { execFileSync } from 'node:child_process';
Expand All @@ -20,14 +20,14 @@ import { createPinnedArtifactPolicy } from './lib/pinned-artifacts.mjs';

const REPOSITORY = 'zknpr/txiki.js';
const SOURCE_BRANCH = 'master';
const SOURCE_COMMIT = '0353db5719c80ee69e0f400f4dfcd2256a034ee0';
const PINNED_RUN_ID = '39983760340';
const SOURCE_COMMIT = 'b45ac83ec0d7809c413afcf4091d7ecf12a46452';
const PINNED_RUN_ID = '40086152581';
const PINNED_SHA256 = Object.freeze({
'aarch64-linux-gnu/tjs': '6ec923d4ade3dc0d0e3d00224db52f4e39c9bfbce771a9b61b8843a90954ab96',
'aarch64-macos/tjs': '155a7abdfbbbad0b2610b6ca6e6a6d10aa83bd610d4be0f6a15d352bf3f6969a',
'x86_64-linux-gnu/tjs': 'ec13344d64c8506c1821d4006999e4d089b05fe2499cd13f103e700d9a06b395',
'x86_64-macos/tjs': '6b9537e33cb5180501e5ef9e735b8126a72998e5fb4a17519722c2ec72457978',
'x86_64-windows/tjs.exe': '83b757b6f9698c669b26aead2bf795d0e2800d1cb9da85513881dc21630d2a4b'
'aarch64-linux-gnu/tjs': 'd316445ec08f39baf0bdcdf7624df73e17f9b166a1365283b396e6afd4ee778a',
'aarch64-macos/tjs': 'eabb77e38e127d0cd6276f12195524981ac0a8362ce58b56757ce74d660175c8',
'x86_64-linux-gnu/tjs': 'b437ea6321c5a1a281a9711990eb0c72cfba054cda9daff88145ee44d00a8d11',
'x86_64-macos/tjs': 'f0daa1b39b6f081053f1a41cfb4c8ef28872d2c3512a837b5f1c0bbfced126a7',
'x86_64-windows/tjs.exe': 'ae95f8c909a0b00fe5a35417ccae36541a5bea87b04f07dd203e59c77dd6150c'
});

const scriptDirectory = path.dirname(fileURLToPath(import.meta.url));
Expand Down
Loading
Loading