Skip to content

fix(tools): tighten seven tool definitions - #652

Open
aliasunder wants to merge 8 commits into
mainfrom
fix/tool-definitions-batch-2-fixups
Open

aliasunder wants to merge 8 commits into
mainfrom
fix/tool-definitions-batch-2-fixups

Conversation

@aliasunder

@aliasunder aliasunder commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

Tightens the definitions of seven tools. Four score below 4.8 on Glama's tool-definition grader, and each change answers the reason the grader gave. The other three are at 5.0 and get completeness fixes. Review findings and a reader's pass then corrected statements that contradicted the code and clarified what a cold reader could not follow. The default tool list shrinks by 662 characters.

Tool Grader's reason Change Chars
vault_search_by_property The Parameters block restates schema fields; the numeric-matching paragraph is dense One numeric-matching bullet with examples for the accepted forms. Key and value matching are stated separately. A checkbox matches only as text, so "1.0" and "true" don't match. Numbers compare as 64-bit floats. limit keeps the newest matches and has no upper cap. The array rule is stated once 3,269 → 2,959
vault_find_orphans The two examples repeat the parameter bullets; the daily-notes detail is dense One example. The defaults line opens with "With exclude_folders omitted" and drops a setting that is unset whenever it renders. Keeping a default points to vault_get_daily_note for the daily notes folder. The override line names the schema default instead of an env var the client can't see 2,150 → 2,087
vault_delete_note The error list reads as reference The opener, Behavior and Returns render per server. On an Obsidian Sync server the text no longer claims the "Deleted files" setting applies; it says deletes are permanent and how to recover. Shorter Errors entries, each keeping its remedy. Remedies that need hidden paths or env vars go to the vault owner or server operator 4,211 → 4,032
vault_search_by_tag No guidance past the 20-result cap; Parameters repeats the schema Names frontmatter tags (inline #tags are not indexed) and spells out nested tags. Drops the sentence the schema already states and the undefined "promoted keys". The cap itself is answered by a limit parameter, approved by the maintainer to ship in the same release 1,618 → 1,567
vault_get_backlinks — Errors as bullets quoting "path must end in …". Names notes and canvases as sources. Routes to vault_find_orphans for notes nothing else links to, since self-links count as backlinks but not against orphan status. An empty result means nothing links to the path (a markdown link to a missing note still counts). The empty-result remedy names only the path-finding tools the server serves 1,587 → 1,645
vault_recent_notes — additional_properties? marked optional. States that a small limit can leave out notes without a valid created. created is null when missing or invalid. sort_by described as the timestamp to sort by 1,630 → 1,600
vault_list_notes — Subfolders are listed when there is no glob. The sort is code-unit order. The folder path-error remedy is offered as an alternative 2,279 → 2,192

Default tool list: 126,205 → 125,543 characters. Only these seven tools change in any of the 18 snapshot configurations (six in the read-only ones, which do not serve vault_delete_note). Server instructions and prompts are unchanged.

Code-only cleanups: findOrphans takes a readonly array, so two defensive copies are gone. Comments on the trash bookkeeping and the not-found remedies are clarified.

Tests

  • 8 error-contract tests: vault_list_notes and vault_list_files reject an absolute folder, a folder outside the vault, . and a hidden folder over the real transport. No test covered these listed errors before.
  • tool-definitions.test.ts:
    • vault_delete_note:
      • its opener, Behavior and Returns with and without OBSIDIAN_SYNC, asserted whole;
      • its Errors list in both configurations;
      • its not-found remedy with vault_list_notes disabled;
      • its broken-links entry with vault_get_backlinks disabled;
      • PROTECTED_PATHS and memory-folder protected paths, asserted as whole lines.
    • vault_find_orphans:
      • its defaults and override lines;
      • its vault_get_daily_note hint with that tool disabled and under an override list.
    • vault_get_backlinks: its empty-result remedy with every, one and no path-finding tool served.
    • vault_recent_notes: its sort line, asserted whole.
  • Tool-surface snapshots regenerated.
  • npm test: 4,735 passed. Build, lint, knip and prettier pass.

Review

  • A tool-definition review traced every Errors entry and new claim to the code, found no dropped fact, and raised eight findings. Five were fixed; the other three are listed under "Not in this change" below.
  • Ship-check:
    • PR review found no defects in the change.
    • A cold reader's pass and a code-quality pass found three statements that contradicted the code. All three are fixed: self-links versus orphans, checkbox matching, and "every" with a limit.
    • The rest of their wording suggestions went through review before landing.
    • Test audit added tests for four text branches nothing covered, and rewrote one test that passed on the wrong item. Each new test was mutation-checked.
    • Bug check found no code bug in the change. It found three statements to correct: which component bypasses the trash setting under Sync, the recent-notes limit, and the backlinks empty-result claim.

Not in this change. Each item was reviewed and scheduled by the maintainer:

  • Rare failures that no Errors section lists (database faults, raw filesystem errors).
  • vault_list_files' folder error entry.
  • Text findings on tools this PR doesn't edit, including the ungated references in vault_read_note and vault_move_note. They are grouped into one pass so each tool is re-graded once.
  • Behaviour bugs found during review:
    • tag letter case;
    • a folder-case echo in vault_list_notes;
    • backlinks to missing notes depending on link style;
    • a trash-sweep identity check;
    • created-time ordering across a daylight-saving change.

🤖 Generated with Claude Code

Comment thread src/vault-mcp/mcp-core/tools/search-tools.ts Outdated
Comment thread src/vault-mcp/mcp-core/tools/search-tools.ts Outdated
@umm-actually

umm-actually Bot commented Oct 6, 2026

Copy link
Copy Markdown

Gate the remaining not-found remedies behind tool availability
Low severity · conventions · medium confidence

src/vault-mcp/mcp-core/tools/vault-crud-tools.ts:81 — beyond the diff's line ranges, in code the changes touch or depend on.

Pre-existing: the not-found remedies in vault_read_note, vault_move_note, and vault_update_properties, plus vault_write_note's already-exists remedy, still name their referenced tool unconditionally, while this PR gives vault_delete_note and the edit tools a spelled-out fallback when the named tool is disabled. Under the corresponding DISABLED_TOOLS configs the served text directs the model to a tool the server never registered — the drift the availability-keying rule and this PR's noteNotFoundVerifyRemedy pattern exist to prevent. The new test asserting each not-found entry keeps a remedy without vault_list_notes covers five edit tools but omits vault_read_note, whose entry cites the same error and the same tool.

Failure scenario: DISABLED_TOOLS=vault_list_notes — a client calls vault_read_note with a typo'd path, gets a not-found error, and the tool description's remedy says to verify with vault_list_notes, a tool absent from the served surface, so the remedy dead-ends; likewise vault_write_note's already-exists remedy under DISABLED_TOOLS=vault_patch_note,vault_replace_in_note points only at the disabled vault_replace_in_note.

Suggested fix
In a follow-up (this PR's scope rule freezes other tools' text), route these entries through gated remedy strings like the existing noteNotFoundCheckRemedy/noteNotFoundVerifyRemedy: read_note and move_note take the spelling-and-letter-case fallback, and update_properties and write_note's already-exists remedy fall back to tool-free wording when their named tool is disabled, built like replace_in_note's servedPropertyEditors join.

umm-actually · z-ai/glm-5.3-flash

@umm-actually

umm-actually Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

umm-actually re-reviewed at e08f6f6

2 new finding(s) posted (7 tracked finding(s) across all runs).

Review phase conventions-tests did not complete; its findings are missing from this run. See the check run for details.

The review deadline expired; results from completed phases are shown.

Context notes
  • 18 changed file(s) excluded from review: src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/default.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/disabled-tools.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/file-tools-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/file-tools-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/memory-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/memory-off+file-tools-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/memory-off+file-tools-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/memory-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/obsidian-sync.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+file-tools-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+file-tools-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+memory-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+memory-off+file-tools-off+embedding-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+memory-off+file-tools-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly+memory-off.json (diff_exclude_paths input), src/vault-mcp/mcp-core/__tests__/__snapshots__/tool-surface/readonly.json (diff_exclude_paths input)

umm-actually · deepseek/deepseek-v4.1-flash, z-ai/glm-5.3-flash

aliasunder and others added 4 commits October 6, 2026 13:03
…5.0 tools

- vault_search_by_property: one numeric-matching bullet in place of three
- vault_find_orphans: one example; shorter daily-notes default sentence
- vault_delete_note: shorter Errors entries, each keeping its remedy
- vault_search_by_tag: route past the 20-result cap; drop a schema restatement
- vault_get_backlinks: Errors as bullets quoting the message
- vault_recent_notes: additional_properties marked optional
- vault_list_notes: folder path-error entry cut to its remedy
- Error-contract tests for vault_list_notes and vault_list_files folder errors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- vault_search_by_tag: past-cap guidance moves to When to use and covers
  notes tagged only with the parent; the Parameters bullet names prefix mode
- vault_find_orphans: the defaults sentence opens with "With exclude_folders
  omitted", so "the defaults" has a referent
- vault_search_by_property: the array rule is stated once, the key and value
  schema text drops the discovery routing, additional_properties? optional
- vault_get_backlinks: the empty-result remedy notes vault_list_notes lists
  notes only
- Tests for the two gated text branches no snapshot configuration renders

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- vault_get_backlinks: a self-link counts as a backlink here, but
  vault_find_orphans ignores self-links, so the routing sentence now says
  find_orphans finds notes nothing else links to
- vault_search_by_property: a checkbox is stored as JSON true/false and
  matches only as the text "1"/"0", never numerically; the bullet no longer
  implies a checkbox is a stored number
- vault_search_by_tag: the vault_search_by_property route returns at most
  limit notes, so it no longer promises every note

Ship-Check: code-quality · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- findOrphans takes a readonly excludeFolders list, so vault_find_orphans
  and the vault-orientation prompt pass their resolved lists directly
  instead of copying them
- vault_delete_note handler: the clearStaleTrashEntry comment names the
  trash_entries row, why an unrecorded move clears it, and that a
  "Permanently delete" never calls it
- The not-found remedy comment states what the remedies fall back to
- vault-orientation catch binding: err -> error

Ship-Check: code-quality · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aliasunder
aliasunder force-pushed the fix/tool-definitions-batch-2-fixups branch from 5f191d5 to f045678 Compare October 6, 2026 17:03
aliasunder and others added 3 commits October 6, 2026 13:23
- vault_search_by_property: key and value matching named separately, number
  forms shown as examples, 64-bit float comparison, "true" text-only, limit
  keeps the newest matches and has no upper cap
- vault_find_orphans: the override line names the schema default, the default
  line drops the unset setting, and keeping a default points to
  vault_get_daily_note for the daily notes folder
- vault_search_by_tag: frontmatter tags only, nested tags spelled out,
  "promoted" replaced; the interim past-cap routing is removed ahead of a
  limit parameter
- vault_get_backlinks: notes and canvases named; canvas paths come from
  vault_list_files
- vault_recent_notes: the created sort's consequence stated; invalid created
  values are null; sort_by described as the timestamp
- vault_delete_note: Behavior, opener and Returns render per Obsidian Sync
  setting, so a Sync server no longer claims the Deleted files setting
  applies; remedies that need hidden paths or env vars go to the vault owner
  or server operator
- vault_list_notes: subfolder listing tied to the glob, code-unit sort order,
  and the path-error remedy offered as an alternative

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- vault_delete_note's broken-links entry with vault_get_backlinks
  disabled, and its protected-path list under PROTECTED_PATHS.
- vault_find_orphans leaves the daily-note hint out under an
  ORPHAN_EXCLUDE_FOLDERS list.
- vault_delete_note's Returns line per OBSIDIAN_SYNC.
- The memory-dir protected-path test now asserts the whole entry; its
  "Profile/" substring also matched the when-to-use line.
- findDescriptionLine moves to module scope beside
  extractDescriptionSection and takes the registered calls.
- The obsidian-sync combo comment now says OBSIDIAN_SYNC changes
  vault_delete_note's text, not only its Errors list.

Ship-Check: test-audit · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- vault_delete_note (Sync): the server, not Obsidian Sync, bypasses the
  Deleted files setting
- vault_recent_notes: undated notes can be left out by a small limit, not
  always
- vault_get_backlinks: an empty result means nothing links to the path; a
  markdown link to a missing note still counts as a backlink

Ship-Check: bug-check · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aliasunder
aliasunder marked this pull request as ready for review October 6, 2026 17:51
Comment thread src/vault-mcp/mcp-core/tools/search-tools.ts
Comment thread src/vault-mcp/mcp-core/__tests__/tool-definitions.test.ts
…lt remedy

The remedy now lists whichever of vault_search, vault_list_notes and
vault_list_files the server serves, and falls back to checking the path
when none is. A test covers all three served, one served, and none served.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aliasunder

Copy link
Copy Markdown
Owner Author

Re: umm-actually finding 6021116393 (gate the remaining not-found remedies behind tool availability).

Valid. The four remaining ungated references, in vault_read_note, vault_move_note, vault_update_properties and vault_write_note's already-exists remedy, are text in tools this PR doesn't edit. The maintainer approved fixing them together with every other ungated tool-to-tool reference in one dedicated pass, so each of those tools is re-graded once. The same pass covers vault_read_note's missing case in the not-found remedy test.

In this PR, vault_get_backlinks' remedy now names only served tools (e08f6f6), and vault_delete_note's not-found remedy already falls back when vault_list_notes is disabled.


🔍 ship-check · pr-monitor · claude-opus-5-5

Comment on lines 1115 to 1116
'- "cannot move to trash …',
'- any other "cannot move to trash …"',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert full delete-note error entries, not slices at the first em dash
Low severity · tests · high confidence

Pre-existing: deleteNoteErrorLeads slices each Errors entry at its first ' — ', and for the 'cannot move to trash … — 100 collisions in .trash/' entry that em dash sits inside the quoted message, so the assertion covers only '- “cannot move to trash …' and never that entry's remedy. A dropped or reworded remedy on that entry — the thing the Errors convention requires every bullet to keep — passes the test silently, while a punctuation-only change to the same message would fail it.

Failure scenario: An edit that shortens the collisions entry's remedy (e.g. drops 'ask the vault owner to clear old copies, then retry') leaves the produced line starting with the same '- “cannot move to trash …' prefix, so deleteNoteErrorLeads still equals the expectation and the test stays green with the required remedy gone.

Suggested fix
Assert the whole Errors section between 'Errors:' and '\n\nReturns:' as literal text, the way the new opener/Behavior assertions do, instead of slicing each entry at its first em dash.

umm-actually · deepseek/deepseek-v4.1-flash

@umm-actually

umm-actually Bot commented Oct 6, 2026

Copy link
Copy Markdown

Index the singular tag frontmatter property too
Medium severity · correctness · medium confidence

src/vault-mcp/search/search-index.ts:1240 — beyond the diff's line ranges, in code the changes touch or depend on.

Pre-existing: upsertNote fills the notes.tags column only from the frontmatter tags property, but Obsidian treats the singular tag property as tags too, so notes tagged under tag: never reach vault_search_by_tag or vault_list_tags. That is a strict subset of what Obsidian's tag pane and search recognize, which this repo's conventions explicitly classify as a bug rather than a limitation.

Failure scenario: A note declares tag: [project] in frontmatter with no tags key. Obsidian's Properties UI and tag pane list the project tag, but vault_list_tags omits it and vault_search_by_tag({ tag: 'project' }) returns an empty array, so the note is unreachable through every tag tool.

Suggested fix
In upsertNote, build the stored tags column from both spellings — read frontmatter.tag alongside frontmatter.tags and deduplicate the combined list — and add a fixture note using the singular spelling so the tag tools' Obsidian parity is pinned by a test.

umm-actually · z-ai/glm-5.3-flash

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant