In Positron, use virtual notebook (in memory) for LSP features instead of vdoc (on disk)

[juliasilge]

Addresses posit-dev/positron#14540

When Positron serves embedded language features for code cells itself (the new quarto.embeddedLanguageFeatures.native setting, currently verified for R and Python), this extension now stands down and stops serving those features from virtual document temp files. Vanilla VS Code is untouched and every gate falls back to the vdoc path.

Per feature:

• Completion, hover, signature help, definition (lsp/client.ts): gated per language. In a cell of a natively served language the middleware returns undefined instead of consulting a vdoc. Two orderings are preserved deliberately: #| option lines still fall through to the Quarto language server (chunk-option completions come from there), and image hover in prose runs before the gate.
• Semantic tokens (providers/semantic-tokens.ts): gated per language, returning undefined and NOT next() in this case. The Quarto server's semantic-tokens capability is kind of a fake thing that answers { data: [] }, and an empty stream counts as an answer that can suppress Positron's provider in the shared score group.
• Document symbols (lsp/client.ts, new lsp/cell-symbols.ts): on the native path the middleware fetches positron.executeQuartoCellSymbolProvider once per request and nests each cell's symbols under its chunk symbol by range containment. The 500 ms retry loop does not run on this path; when a language server registers, the editor re-requests symbols on its own. Headings always come from the server and next() is never gated here.
• Formatting (providers/format.ts): document and range formatting call positron.executeQuartoCellFormattingProvider / positron.executeQuartoCellRangeFormattingProvider. A veto (vetoedCells > 0) abandons the format with the existing aggregate message, matching the vdoc path's all-or-nothing semantics. quarto.formatCell stays vdoc-backed (this will be a follow-up).
• Diagnostics (providers/diagnostics.ts): no per-language vdoc sessions are created while native is on; toggling the setting disposes and re-arms the manager live.
• Statement range / help topic (lsp/client.ts): the Positron registrations follow the setting live via an onDidChangeConfiguration listener. It's disposed when native turns on, re-registered when it turns off. Last registration wins these single-answer features, so both toggle directions land correctly.

Also I'll highlight that vdoc temp file creation and deletion are now logged at debug level to the Quarto output channel ([vdoc] Created/Deleted <path>). This makes it a whole lot easier to do the validation below, as the vdoc files are transient dotfiles that live for milliseconds on a fast computer.

How the gate works

host/native-features.ts has three parts:

1. Capability detection is connected to command presence. At activation the extension probes for the internal _executeQuartoCell* command ids with getCommands(false). It must be the internal ids, because the public positron.executeQuartoCell* commands are API commands, which are never mirrored into the registry getCommands reads, and getCommands(true) filters underscore-prefixed ids. Positron registers these commands unconditionally, so presence exactly tracks "this build can serve this command natively"; vanilla VS Code and older Positron builds have no such commands.
2. The setting is read live on every call, so toggling takes effect without a window reload for every gate.
3. The language set is extension-side (r, python), matched against EmbeddedLanguage.ids so aliases count. A language not in the set keeps its vdoc. This does mean we are hard-coding which languages can use these virtual notebooks but it's the safe direction.

Both skew directions are meant to be conservative and fail safe, so old extension + new Positron and new extension + old Positron both stay on vdocs.

Behavior changes

• Format Document now formats every language's cells, not just the cursor's, and every Python cell in the document, where the vdoc path deliberately formatted only the cell under the cursor. Format-on-save does the same. This is the change most likely to surprise a Python user, I think, but probably in a good way.
• quarto.cells.hoverHelp.enabled / signatureHelp.enabled can no longer suppress anything once native is on. Those settings gate the extension's middleware, which is no longer in the loop; Positron serves hover and signature help regardless. If we get comlaints about this, we'll need to figure out how to respect this setting in Positron.

Languages outside the verified set

Three gates are whole-document (symbols, formatting, statement range / help topic), so a language we don't yet support, e.g. the {sql} chunk a knitr document can carry, is handed to Positron too. I validated against a setting-off control window, where the {sql} chunk's outline symbols match the control, Format Document leaves the sql chunk unchanged without blocking the R and Python cells, and Cmd+Enter/F1 behave the same in both windows. IIUC the residual risk is selector scheme where a provider registered for { language, scheme: 'file' } answers a vdoc temp file but not a vscode-notebook-cell: URI.

Known gap: unterminated fences

Positron's parser ignores an unclosed fence; the extension's engine parses one to the end of the document. So while a cell is being typed, or after a closing fence is deleted, completion, hover, signature help, and definition go quiet inside that cell until the fence is closed. The extension has gated off and Positron has no cell to serve. I think we should accept this for now rather than fix extension-side; the only extension-side fix is mirroring Positron's fence-closure rule, which recreates the duplication this PR exists to remove. In validation this felt pretty transient and OK in terms of user experience. Closing the fence brings features straight back. The real fix is Positron-side in the next PR over there. Either the parser can own an unterminated final cell, or Positron can expose a cell-ownership query the gate can consult.

Release coordination

We'll need to do an extension release before the next set of changes in Positron, as the extension bump and the default flip will stick together there.

Validating this PR

In a Positron build that carries the virtual notebook support (i.e. the current release or a daily), run this branch's extension dev host (Run Extension from this repo). In the dev host window, set "quarto.embeddedLanguageFeatures.native": true, open a .qmd that has R and Python chunks, a #| option line, and an image link in prose, and start R and Python sessions. Open the Quarto output channel and set its level to Debug. This PR adds [vdoc] Created/Deleted <path> logging there, which is how you can see whether the extension is serving from temp files at all.

With the setting ON:

1. The Quarto channel shows [NativeFeatures] Host serves Quarto cell language features... at startup. If that line is absent, the host has no cell commands and everything below is silently on the old path.
2. Hover, completion, signature help, and go-to-definition work inside R and Python chunks, and no [vdoc] lines appear in the log. Before this PR, each of these created a temp file.
3. Typing #| in a chunk still offers chunk-option completions (those come from the Quarto language server, not the cells).
4. Hovering an ![](...) link in prose still shows the image preview.
5. The Outline shows headings with chunks nested under them and code symbols under the chunks. (A flat "Quarto Code Cells" group that we don't want also appears; removing it is Positron-side, in the next PR.)
6. Format Document formats every R and Python cell; #| lines, fences, and prose are untouched. Note the deliberate scope change described above. Before this PR, only the cursor's language (and for Python, only the cursor's cell) was formatted.
7. Cmd+Enter and F1 work in a chunk on first window load.
8. Toggle the setting off and on WITHOUT reloading the window, and Cmd+Enter keeps working in both states (the statement-range and help-topic registrations follow the setting live).
9. Introduce an error in an R chunk and in a Python chunk, and there is one Problems entry each on the .qmd, with no duplicates.

Controls:

10. Set the setting to false and reload. Now the same interactions produce [vdoc] lines in the log, and everything still works. That is the old path, unchanged.
11. Run the dev host in vanilla VS Code. There is no [NativeFeatures] line, everything is served from vdocs, behavior is unchanged from before this PR.

[posit-snyk-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[juliasilge]

I started running all the Positron E2E tests with these changes, and there is a little bit of cleanup still to do on the Positron side, such as posit-dev/positron#15974.