Skip to content

About

Local-first token reduction engine and MCP server: gives AI coding agents dependency skeletons (types, signatures, exports) instead of full files, so they read far fewer tokens.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

token-shrink

Cut the tokens your AI coding agent reads, without hiding how your code fits together.

When an agent such as Claude Code, Cursor or Cline works on a file, it also needs the files that file imports. token-shrink sends those imported files as skeletons: every type, interface and function signature stays, and the function bodies are left out. The agent still knows how to call everything correctly, and you pay for far fewer tokens.

⚡ token-shrink: 29,597 → 12,274 tokens (−17,323 · 58.5% saved)

On its own source code, token-shrink removes about 70% of the tokens. It works with 14 languages, runs entirely on your machine, and needs no API key.


Contents


Quick start (Claude Code)

You need Node.js 18 or newer (node --version to check). There's nothing else to install, because npx downloads token-shrink when it first runs.

1. Add the server

Recommended: once, for all your projects. Run this from any folder:

claude mcp add token-shrink --scope user -- npx -y -p @ajdev0/token-shrink token-shrink-mcp

Leave out --root here. token-shrink finds the project you're working in by itself, from the nearest .git folder or package.json.

Or: for one project only. Run this from inside that project's folder:

cd /path/to/your-project
claude mcp add token-shrink -- npx -y -p @ajdev0/token-shrink token-shrink-mcp --root .

Run that command in the project folder, not your home folder. Without --scope user, Claude Code saves the server only for the folder you ran the command in. If you run it in your home folder, it won't appear when you open Claude in your project. See Troubleshooting.

2. Restart Claude Code and check

Open Claude Code in your project and type /mcp. You should see token-shrink listed as connected. From a terminal, claude mcp list shows the same thing:

token-shrink: npx -y -p @ajdev0/token-shrink token-shrink-mcp - ✔ Connected

3. Just work

You don't need to change how you prompt. The first time the server starts in a project, it writes a small rule file (.claude/rules/token-shrink.md) that tells Claude to use token-shrink before working on a file. The first run also downloads the parsers for your languages, so it needs internet access once.


Cursor, Cline and other agents

Any agent that supports MCP servers can use token-shrink. Use this server definition:

{
  "mcpServers": {
    "token-shrink": {
      "command": "npx",
      "args": ["-y", "-p", "@ajdev0/token-shrink", "token-shrink-mcp", "--root", "/absolute/path/to/your-project"]
    }
  }
}
Agent Where the definition goes Rule file token-shrink writes
Cursor .cursor/mcp.json in your project .cursor/rules/token-shrink.mdc
Cline MCP Servers → Configure MCP Servers .clinerules/token-shrink.md
Other MCP clients wherever that client keeps MCP servers none; ask the agent to call get_compressed_code_context

By default, rule files are written for Claude Code, Cursor and Cline. Pick only the ones you use with --rule-target (see Server options).


See how much you save

Every request is measured against what the same files would cost without token-shrink. The totals are saved in your project, in .token-shrink/stats.json.

In your terminal

Run this inside your project folder:

npx -y -p @ajdev0/token-shrink token-shrink stats

Example output:

⚡ last call (get_compressed_code_context): 29,597 → 12,274 tokens (−17,323 · 58.5% saved)
⚡ session · 6 calls: 84,920 → 32,610 tokens (−52,310 · 61.6% saved)
⚡ all time · 41 calls: 512,400 → 188,950 tokens (−323,450 · 63.1% saved)

To check a different folder, add --root /path/to/project. If you see No token-shrink stats yet, the agent hasn't asked for context in that project yet.

In Claude Code's status line (yellow, always visible)

1. Download the status line script:

curl -fsSL -o ~/.claude/token-shrink-statusline.mjs \
  https://raw.githubusercontent.com/ajdev0/token-shrink/main/hooks/claude/statusline.mjs

2. Add this to ~/.claude/settings.json (create the file if it doesn't exist):

{
  "statusLine": {
    "type": "command",
    "command": "node ~/.claude/token-shrink-statusline.mjs"
  }
}

Restart Claude Code. Below the prompt you'll see something like this, in yellow:

⚡ token-shrink −17,323 tokens (58.5%) · session −52,310 (61.6%)

The line stays empty until token-shrink has handled a request in the current project. If you already have a statusLine setting, this replaces it.

Good to know

  • Add .token-shrink/ to your project's .gitignore. The stats are personal and change constantly.
  • To turn stats off, add --no-stats to the server command, or set TOKEN_SHRINK_STATS=0.
  • Colors follow your terminal. NO_COLOR=1 turns them off, and FORCE_COLOR=1 forces them on.

Troubleshooting

"MCP server token-shrink already exists in local config", but it isn't in /mcp

You added it earlier from a different folder, often your home folder. A server added without --scope user only shows up in the folder it was added from. To fix it, remove that copy and add it for all projects:

cd ~                                            # the folder you originally added it from
claude mcp remove token-shrink --scope local
claude mcp add token-shrink --scope user -- npx -y -p @ajdev0/token-shrink token-shrink-mcp

Then restart Claude Code in your project.

If you use --scope user, don't add --root /some/project. Every project would then get that one project's code.

It shows as failed in /mcp

  • Check your Node version with node --version. You need 18 or newer.
  • Run the server by hand to see its error: npx -y -p @ajdev0/token-shrink token-shrink-mcp --root ., then press Ctrl+C to stop it.
  • The first run downloads parsers from GitHub. Behind a firewall or offline, files are still indexed, but they aren't shrunk until the download succeeds.

Claude doesn't seem to use it

  • Check that .claude/rules/token-shrink.md exists in your project. If you started the server with --no-create-rule, ask Claude directly: "use get_compressed_code_context for src/app.ts".
  • The rule only applies to code files (.ts, .py, .go and so on), not to Markdown or JSON.

I'm getting an older version

npx can reuse a version it downloaded earlier. To always get the newest, use @ajdev0/token-shrink@latest in the server command. Check what's published with npm view @ajdev0/token-shrink version.


What your agent gets

token-shrink adds four tools. You don't call them yourself. The rule file tells your agent when to use each one.

Tool What it does When it's used
get_compressed_code_context Your file in full, plus skeletons of the files it imports Before working on any code file
git_diff_context The changed files, their imports, and the files that call them Reviews, multi-file edits, uncommitted changes
expand_symbol The complete code of one function, class or method When a skeleton isn't enough
search_symbol_signatures Where something is defined, as file:line — signature lines When a name isn't in the context yet

What a skeleton looks like

src/user.ts as written:

/**
 * Builds the label shown next to a user's avatar. Falls back to
 * the part of the email before the @ when no name is set.
 * @param u the user to label
 */
export function displayName(u: User): string {
  // prefer the real name when there is one
  if (u.name) {
    return u.name.trim();
  }
  return u.email.split('@')[0];
}

What the agent receives:

/** Builds the label shown next to a user's avatar. @param u the user to label */
export function displayName(u: User): string /* ... */

What gets trimmed:

  • Function and method bodies, in all languages.
  • Long one-line arrow functions (over 40 characters of expression).
  • Large constant objects and arrays (over 200 characters at the top of a file). Type annotations and as const values stay.
  • Comments. Doc comments shrink to their first sentence plus @param and @returns, and other comments are removed.
  • Unused exports, if you opt in with ringOneMode: "used". A dependency then keeps only what your file imports from it.
  • The active file, when it's too big for the budget. Pass focusLines and only the function you're editing stays whole.

The agent always gets the file it's actually editing in full, unless that file alone is bigger than the token budget.


Project settings

Optionally, add a .tokenshrinkrc.json file to your project root:

{
  "ignorePatterns": ["**/generated/**"],
  "keepUnpruned": ["src/types/global.d.ts"],
  "preserveAnnotations": ["@api"],
  "ringOneMode": "used"
}
Setting What it does
ignorePatterns Folders and files to leave out entirely (glob patterns).
keepUnpruned Files always sent in full, such as shared type definitions.
preserveAnnotations Mark a function with a comment such as // @api and its body is kept.
ringOneMode "used" keeps only the exports your file imports (TypeScript and JavaScript). The default, "full", keeps the whole skeleton.
autoWorkflow false makes the rule file lighter, so the agent decides on its own when to use the extra tools.

The MCP server picks up changes to this file while it runs. If the JSON is invalid, a warning is logged and the defaults are used.


Server options

Add these after token-shrink-mcp in the server command.

Option What it does
--root <folder> The project to work on. Optional; without it the project is detected automatically. Same as the ROOT environment variable.
--rule-target=claude Which rule files to write: claude, cursor, cline or all (default). Comma-separate several: --rule-target=claude,cursor.
--no-create-rule Don't write any rule files. Same as TOKEN_SHRINK_CREATE_RULE=0.
--no-stats Don't save savings to .token-shrink/stats.json. Same as TOKEN_SHRINK_STATS=0.

Example, for Claude Code only and without stats:

claude mcp add token-shrink --scope user -- npx -y -p @ajdev0/token-shrink token-shrink-mcp --rule-target=claude --no-stats

Supported languages

Language File extensions
TypeScript .ts .mts .cts
JavaScript .js .mjs .cjs
React (TSX / JSX) .tsx .jsx
Python .py .pyi
Go .go
Rust .rs
Java .java
Kotlin .kt .kts
Swift .swift
Dart / Flutter .dart
C .c .h
C++ .cc .cpp .cxx .hpp .hh .hxx
PHP .php

Unused-exports filtering (ringOneMode: "used") works for TypeScript and JavaScript; other languages get the full skeleton. In React files, 'use client' / 'use server' directives are kept.


Updating

  • With npx (the setup above), you get new releases automatically. If an update doesn't arrive, use @ajdev0/token-shrink@latest in the server command and restart your agent.
  • Rule files (.claude/rules/token-shrink.md and the others) are upgraded automatically when a release changes them. A rule file you wrote yourself is never touched. Edits to a generated file are kept until a release changes the rule again, so put custom instructions in a separate rule file.

Advanced

HTTP server

For tools that aren't MCP clients, run a local HTTP server:

npx -y -p @ajdev0/token-shrink token-shrink --root . --port 3000
curl -s -X POST http://localhost:3000/v1/context \
  -H 'Content-Type: application/json' \
  -d '{"activeFilePath": "./src/page.ts", "maxTokens": 8000}'
Route Method Body Returns
/health GET — status, version, root, indexed file count
/v1/context POST { activeFilePath | activeFiles, maxTokens?, maxSkeletons?, focusLines?, ringOneMode?, includeStats? } markdown, dependencies, unresolved, tokenStats

tokenStats includes originalTokens, totalTokens, savedTokens and reductionPct. The server prints the yellow savings line after each request. Other options: --host (default 0.0.0.0), --max-tokens (a default budget) and --silent. The HTTP server reads .tokenshrinkrc.json once at startup.

Library

npm install @ajdev0/token-shrink
import { prune, createWatcher, assembleContext } from '@ajdev0/token-shrink';

// One file → skeleton
const { code } = await prune('src/user.ts', source);

// A whole project, kept up to date as files change
const watcher = createWatcher({ root: process.cwd() });
await watcher.indexAll();
const { markdown, tokenStats } = await assembleContext(['src/profile.ts'], watcher.cache.entries, {
  maxTokens: 8000,
  ringOneMode: 'used',
});

Works with both import and require. Other exports include analyze, assembleMany, formatSavings, readStats and gitChangedFiles.

Tool reference

get_compressed_code_context

Argument Type Description
activeFilePath string The file being worked on. Use this or activeFiles.
activeFiles string[] Several files to send in full; their imports are combined.
maxTokens number Budget for the whole reply. Your files come first, then imports, most-used first. Anything that doesn't fit is listed in a note.
focusLines number[] or { [path]: number[] } Lines being edited. If your file alone exceeds maxTokens, only the definitions around these lines keep their bodies.
ringOneMode "used" | "full" Overrides the project setting for this call.
maxSkeletons number Maximum number of imported files (default 50, max 200).
includeStats boolean Adds token counts at the end of the reply.

git_diff_context: scope (worktree default, staged, or branch for base...head), base, head, includeUntracked, maxFiles (default 30), maxImporters (default 20), maxSkeletons, maxTokens, ringOneMode. With a budget, the reply is packed as changed files, then their imports, then the files calling them. If the changed files alone are over budget, they're trimmed to the changed definitions.

expand_symbol: filePath, symbolName, maxMatches. Returns the complete definition, including all overloads.

search_symbol_signatures: query, kind, maxResults. Matches are ranked exact, then prefix, then substring.


How it works

  your file (Ring 0)  ──imports──►  dependencies (Ring 1)  ◄──imports──  callers (Ring 2, git_diff_context)
        full text                       skeletons                            skeletons
  1. Parse. Each file is parsed with tree-sitter, so cuts follow the real syntax instead of guesses. Parsers (.wasm files) are downloaded once, pinned to exact versions, and cached.
  2. Prune. Bodies, long arrows, large literals and comments are replaced with placeholders such as /* ... */ (pass in Python). The result is still valid code.
  3. Watch. A file watcher re-prunes a file only when its contents change, so skeletons are ready before the agent asks.
  4. Assemble. Your file, its imports and, for reviews, its callers are packed into one Markdown reply, within the token budget if one is set.

Development

git clone https://github.com/ajdev0/token-shrink.git
cd token-shrink
npm install
npm run build        # compile to dist/
npm run typecheck
npm test             # offline suite
NETWORK=1 npx vitest run tests/grammars.test.ts   # downloads and checks every language parser

Releases are published to npm by .github/workflows/publish.yml when a version tag is pushed (npm version minor && git push --follow-tags).

License

MIT

About

Local-first token reduction engine and MCP server: gives AI coding agents dependency skeletons (types, signatures, exports) instead of full files, so they read far fewer tokens.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages