Conversation
…lease chore: backmerge main after release
* feat(core): detect stale index and surface vector index warnings Audit of all RAG library usage confirmed every dependency ( @huggingface/transformers, @lancedb/lancedb, @modelcontextprotocol/sdk, zod, unpdf, mammoth, fast-glob ) is used according to its current API. No library needed replacement or upgrade. Three quality and DX gaps fixed: - store.ts no longer swallows createIndex errors silently. writeRows now returns vectorIndexWarning, propagated into IngestResult and printed by the ingest CLI, so a fallback to flat scan is visible to the operator. - A new index-manifest.json (schemaVersion, embedding provider/model, chunk size/overlap, ragmir version) is written next to the LanceDB table at each ingest. getIndexFreshnessWarning detects a stale or incompatible index cheaply. ask() returns staleWarning, doctor reports indexFreshness with a rebuild next step, and the search/ask/doctor CLI surfaces the warning on stderr without breaking JSON output. - The hybrid lexical scan limit is now configurable (hybridTextScanLimit, default 5000, RAGMIR_HYBRID_TEXT_SCAN_LIMIT env). getLexicalScanWarning flags when the corpus exceeds it so BM25 recall loss is no longer silent. Also includes repository metadata and docs updates: npm package metadata in the root package.json, sponsor badge and issue reporting guidance in the README/CONTRIBUTING, a feature-request issue template, and conventional- commit/branch checkboxes in the PR template. * fix(doctor): drop manual backtick escaping flagged by CodeQL The freshness warning comes from controlled getIndexFreshnessWarning output, not user input, so the manual backtick escaping was unnecessary and incomplete (did not escape backslashes). CodeQL flagged it as an incomplete string-escaping pattern; removing it resolves the alert.
|
🎉 This PR is included in version 2.3.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release
Promote
developtomainto publish a new npm release via the protectedRelease npmworkflow (semantic-release, Conventional Commits).What's included
writeRowsreturnsvectorIndexWarning, surfaced iningestCLI).index-manifest.jsonwritten at each ingest;getIndexFreshnessWarningdetects stale/incompatible indexes cheaply.ask()returnsstaleWarning,doctorreportsindexFreshnesswith a rebuild next step, andsearch/ask/doctorCLI surfaces warnings on stderr.hybridTextScanLimitis now configurable (default 5000,RAGMIR_HYBRID_TEXT_SCAN_LIMIT);getLexicalScanWarningflags BM25 recall loss on large corpora.Post-merge
Merging this PR will trigger the
Release npmworkflow, which runspnpm validatethensemantic-release. The new version is derived from the Conventional Commits history (expect a minor bump from thefeat).Checklist
pnpm validategreen locally before PR.ragmir/state