Repository navigation
feat(index): keep agent-facing json - manifests and tool contracts - #156
Merged
Merged
Conversation
The blanket .json skip from #154 also dropped the one class an agent actually uses: manifests (how the project is wired) and tool contracts (what the server exposes, with annotations and input schemas). Inventory of the corpus first: 375 .json / 24.5 MB, of which docs/benchmark- evidence is 284 files / 22.1 MB of frozen artefacts (command, result, source_sha256), eval/datasets is 56 files with Python source embedded as strings (their .py oracles are indexed as code anyway), and only 18 files are contracts or manifests. 40 files carry code-like strings; the real code is already in the index. FileGuard now indexes .json when it is a known manifest name (package.json, package-lock.json, npm-shrinkwrap.json, tsconfig.json, plugin.json, marketplace.json, *.plugin.json, composer.json, deno.json, renovate.json) or when its head carries contract keys (inputSchema / surfaces / annotations, file up to 512 KB). MSCODEBASE_INDEX_DATA=1 still indexes everything. Measured on the full corpus: 8 of 445 .json pass the allowlist, the index grows from 18 747 to 18 814 chunks (+67, +0.4%) and the database stays at 143 MB. The value is real: the query "destructiveHint" now returns docs/MCP_CONTRACT.json as top-1; before the allowlist the contract was not in the index at all. Guards: tests/test_data_files_not_indexed.py (8 tests). Mutations M63-M66 (allowlist ignored, allowlist too wide, contract keys not recognised, size limit removed) each turn a test red. Docs: CONFIGURATION.md EN/RU/ZH list both exceptions. KNOWN_ISSUES entry extended with the inventory and the +67 chunk cost; the two oldest entries moved to docs/archive/KNOWN_ISSUES_2026_10.md to stay under the 300-line gate.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(index): keep agent-facing json - manifests and tool contracts
The blanket .json skip from #154 also dropped the one class an agent actually
uses: manifests (how the project is wired) and tool contracts (what the server
exposes, with annotations and input schemas).
Inventory of the corpus first: 375 .json / 24.5 MB, of which docs/benchmark-
evidence is 284 files / 22.1 MB of frozen artefacts (command, result,
source_sha256), eval/datasets is 56 files with Python source embedded as
strings (their .py oracles are indexed as code anyway), and only 18 files are
contracts or manifests. 40 files carry code-like strings; the real code is
already in the index.
FileGuard now indexes .json when it is a known manifest name (package.json,
package-lock.json, npm-shrinkwrap.json, tsconfig.json, plugin.json,
marketplace.json, *.plugin.json, composer.json, deno.json, renovate.json) or
when its head carries contract keys (inputSchema / surfaces / annotations,
file up to 512 KB). MSCODEBASE_INDEX_DATA=1 still indexes everything.
Measured on the full corpus: 8 of 445 .json pass the allowlist, the index grows
from 18 747 to 18 814 chunks (+67, +0.4%) and the database stays at 143 MB.
The value is real: the query "destructiveHint" now returns
docs/MCP_CONTRACT.json as top-1; before the allowlist the contract was not in
the index at all.
Guards: tests/test_data_files_not_indexed.py (8 tests). Mutations M63-M66
(allowlist ignored, allowlist too wide, contract keys not recognised, size limit
removed) each turn a test red.
Docs: CONFIGURATION.md EN/RU/ZH list both exceptions.
KNOWN_ISSUES entry extended with the inventory and the +67 chunk cost; the two
oldest entries moved to docs/archive/KNOWN_ISSUES_2026_10.md to stay under the
300-line gate.