Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 36 additions & 20 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,48 @@
# RTK Token Optimization for Command Code

RTK (Rust Token Killer) is installed on this system. It reduces LLM token
consumption by 60-90% by filtering and compressing shell command output.
RTK (Rust Token Killer) compresses **shell command output** before it enters the
context window. Used well it cuts tokens on noisy commands *and* sharpens context
(less noise → better reasoning). Used badly — compressing output you actually
needed — it hides detail and forces re-runs that cost *more* than they save.
These rules keep it net-positive.

## Prerequisite
## Setup

RTK must be installed and in PATH. Verify with: `rtk --version`
RTK is optional. If `rtk --version` fails, run commands normally — never block
work to install it. Best setup: install RTK's auto-rewrite hook once with
`rtk init -g` (then restart Command Code) so Bash commands are rewritten to `rtk`
automatically and you never prefix by hand. Without a hook, prefix manually per
the tiers in @references/commands.md.

If not installed, direct the user to https://github.com/rtk-ai/rtk#installation.
## The one rule: compress noise, preserve signal

## Core Rule
Use `rtk <cmd>` when output is **large, repetitive, low-stakes** — skim, not
study: `rtk ls`, `rtk tree`, `rtk git status`, `rtk git log`, `rtk docker ps`,
`rtk pip list`, and big test/build runs (`rtk cargo test`, or `rtk err <cmd>` —
RTK keeps the failures and drops the green).

Always prefix supported shell commands with `rtk`. Instead of running
`git status`, run `rtk git status`. Instead of `ls -la`, run `rtk ls`.
Run the command **raw (no `rtk`)** when you need every character:

@references/commands.md
@references/analytics.md
- a **diff/patch you'll apply**, or a hunk you'll edit — exact bytes and line numbers matter
- **structured output you'll parse** — JSON, `--format=…`, anything piped into another tool
- **small output** (≲30 lines) — nothing to save, and real risk of dropping the one line you need
- **secrets / exact config** — never reason about a lossy view of these
- a file you intend to **edit** — use the native Read tool (lossless; it bypasses RTK anyway)

When unsure, start raw. Lean context comes from cutting *noise*, not *signal*.

## Ultra-Compact Mode
## Never default to lossy modes

Add `-u` or `--ultra-compact` for maximum savings:
```bash
rtk git status -u
```
Plain `rtk <cmd>` is near-lossless (keeps errors, diffs, stack traces, exit
codes). `-u` / `--ultra-compact`, `rtk read … -l aggressive`, and `rtk smart`
(2-line summary) are **lossy** — opt-in only for skimming something huge and
unimportant, never your default.

## Exclusions
## If a compressed view isn't enough

- Agent built-in file tools (non-Bash) do not pass through RTK
- Use shell equivalents: `rtk read <file>`, `rtk grep <pattern>`, `rtk find <pattern>`
- Piped commands (`|`) and heredocs (`<<`) bypass RTK rewrite
- Commands already prefixed with `rtk` are left as-is
RTK retains the full unfiltered output, so you don't have to guess and you
shouldn't thrash: re-run that one command raw (or `rtk proxy <cmd>`) to see
everything, then move on. One deliberate re-run is fine; blind repeated re-runs
erase the savings.

@references/commands.md
74 changes: 59 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,11 @@ learns your coding style.
## How It Works

Command Code CLI reads `~/.commandcode/AGENTS.md` and injects it into the system
prompt for every session. This repository provides that AGENTS.md file, teaching
Command Code to prefix shell commands with `rtk` for automatic output compression.
prompt for every session. This repository provides that AGENTS.md file (plus a
loadable skill), teaching Command Code to route **noisy** shell output through
`rtk` while keeping **full fidelity** for output it needs exactly. The fastest
setup is RTK's auto-rewrite hook (`rtk init -g`), which rewrites commands for you
so nothing has to be prefixed by hand.

```
Without RTK: With RTK:
Expand All @@ -20,6 +23,37 @@ cargo test (25,000 tokens) rtk cargo test (2,500 tokens)
ls -la (800 tokens) rtk ls (150 tokens)
```

## When RTK helps — and when to keep full fidelity

RTK only pays off when it removes **noise**. This integration is deliberately
selective:

**Compress (🟢/🟡)** — large, repetitive, low-stakes output you skim: listings,
`git status` / `log`, dependency installs, container/cluster status, and big
test/build runs (plain `rtk` keeps the failures and drops the green).

**Keep full fidelity (🔴)** — run the bare command for diffs/patches you'll apply,
JSON or `--format` output you'll parse, secrets, small outputs, and files you'll
edit (use the agent's native file tools). Compressing these is exactly what makes
a tool like this *cost* tokens — the model loses the detail and re-runs the
command.

Two design choices keep it net-positive:

- **Lossless by default.** Plain `rtk <cmd>` preserves errors, stack traces, diff
hunks, and exit codes; the lossy modes (`-u` / `--ultra-compact`, `-l
aggressive`, `rtk smart`) are opt-in for skimming only — never the default. On
output it can't parse, RTK falls back to the full raw text.
- **Measure net, not gross.** `rtk gain` reports gross savings; the goal is *net*
— savings minus any re-runs and minus the standing cost of these instructions.
`rtk gain --failures` and `rtk discover` show where RTK fits and where it
doesn't (see [references/analytics.md](references/analytics.md)).

This matters for quality too: every frontier model degrades as irrelevant context
grows ("context rot" / "lost in the middle"), so cutting genuine noise can *help*
reasoning — while over-compressing real signal hurts it. The rules above aim for
the first and avoid the second.

## Installation

### Prerequisites
Expand All @@ -36,6 +70,11 @@ ls -la (800 tokens) rtk ls (150 tokens)
> enforcement, install the memory (Method 2); the skill (Method 1) is the
> quickest install and is enough when you mainly want it during shell-heavy work.

> **Even better — the auto-rewrite hook.** RTK can install a `PreToolUse` hook
> (`rtk init -g`) that rewrites Bash commands to `rtk` automatically, so the agent
> never prefixes anything by hand. Pair it with the memory or skill below, which
> carry the *when-to-compress* rules. See `rtk init --help`.

### Method 1: Skill (quick install, on-demand)

```bash
Expand Down Expand Up @@ -81,22 +120,24 @@ cp references/analytics.md ~/.commandcode/references/analytics.md

## What It Teaches the Agent

Once installed, Command Code will automatically use RTK-optimized commands:
Once installed, Command Code routes **noisy, low-stakes** commands through RTK and
leaves precise output alone:

| Category | Without RTK | With RTK | Savings |
| Category | Command | Through RTK? | Est. savings |
|---|---|---|---|
| Git | `git status` | `rtk git status` | ~80% |
| Files | `cat file.rs` | `rtk read file.rs` | ~70% |
| Search | `grep -r "pattern" .` | `rtk grep "pattern" .` | ~80% |
| Tests | `cargo test` | `rtk cargo test` | ~90% |
| Build | `cargo build` | `rtk cargo build` | ~80% |
| Docker | `docker ps` | `rtk docker ps` | ~80% |
| GitHub | `gh pr list` | `rtk gh pr list` | ~80% |
| Status / listings | `rtk git status`, `rtk ls`, `rtk tree` | 🟢 yes | ~80% |
| Logs / containers | `rtk docker ps`, `rtk log app.log` | 🟢 yes | ~80% |
| Dependencies | `rtk pip list`, `rtk pnpm list` | 🟢 yes | ~70% |
| Tests / build | `rtk cargo test`, `rtk err <cmd>` | 🟡 plain mode (keeps failures) | ~90% |
| Diffs you'll apply | `git diff`, `git show` | 🔴 run raw | — |
| JSON / parsed output | `… --format json` | 🔴 run raw | |
| Files you'll edit | native Read tool | 🔴 not RTK | — |

_Savings are illustrative estimates; actual numbers vary by command and output
size. Run `rtk gain` to measure your own._
_Savings are illustrative; actual numbers vary by command and output size. Run
`rtk gain` to measure your own — and `rtk gain --failures` to spot poor fits._

See [references/commands.md](references/commands.md) for the full list.
See [references/commands.md](references/commands.md) for the full tiered list and
[references/analytics.md](references/analytics.md) for measuring net savings.

## Verify

Expand All @@ -106,7 +147,10 @@ rtk gain --graph # Visual savings chart
```

After running a few commands through Command Code, `rtk gain` will show the
accumulated savings.
accumulated savings. Track **net** savings, not just the headline number:
`rtk gain --failures` lists commands RTK had to pass through raw (poor fits), and
`rtk discover` finds new high-value targets. See
[references/analytics.md](references/analytics.md).

## Files

Expand Down
130 changes: 78 additions & 52 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
---
name: rtk-token-optimizer
description: >-
Compress shell command output to cut LLM token usage 60-90% by prefixing
commands with `rtk`. Use whenever running shell commands in Command Code —
git, cargo, npm/pnpm, pytest/jest, docker, kubectl, gh, grep, find, ls, cat,
build and lint tools — so large outputs are filtered before entering context.
Cut LLM token usage on noisy shell output by routing large, low-stakes
commands (listings, status, logs, dependency installs, test/build runs)
through RTK — while keeping full fidelity for diffs, structured/JSON output,
errors you're debugging, and anything you'll parse or edit. Use when running
shell commands in Command Code that produce big, repetitive output.
license: Apache-2.0
compatibility: >-
Requires the RTK binary (>= 0.42.0) installed and on PATH. Verify with
`rtk --version`. Install from https://github.com/rtk-ai/rtk
Requires the RTK binary (>= 0.42.0) on PATH. Verify with `rtk --version`.
Install from https://github.com/rtk-ai/rtk. RTK is optional — run commands
normally if it isn't installed.
metadata:
author: Coding-Dev-Tools
version: "0.1.0"
version: "0.2.0"
homepage: https://github.com/rtk-ai/rtk
allowed-tools: Bash(rtk:*) Bash(git:*) Bash(cargo:*) Bash(ls:*) Bash(cat:*) Bash(grep:*) Bash(find:*) Bash(diff:*) Bash(docker:*) Bash(kubectl:*) Bash(gh:*) Bash(glab:*) Bash(pnpm:*) Bash(npm:*) Bash(pip:*) Bash(bundle:*) Bash(ruff:*) Bash(tsc:*) Bash(eslint:*) Bash(pytest:*) Bash(go:*) Bash(jest:*) Bash(vitest:*) Bash(dotnet:*) Bash(aws:*) Bash(psql:*) Bash(prisma:*) Bash(wget:*)
---
Expand All @@ -20,67 +22,91 @@ allowed-tools: Bash(rtk:*) Bash(git:*) Bash(cargo:*) Bash(ls:*) Bash(cat:*) Bash

## What it is

[RTK](https://github.com/rtk-ai/rtk) (Rust Token Killer) is a CLI proxy that
filters and compresses shell command output before it reaches the context
window — removing noise, grouping similar items, truncating redundancy, and
deduplicating repeated lines. Typical savings are 60-90% on common dev
commands. RTK ships as a single self-contained Rust binary that adds minimal
per-command overhead.
[RTK](https://github.com/rtk-ai/rtk) (Rust Token Killer) is a single Rust binary
that filters **shell command output** before it reaches the context window —
dropping noise (progress bars, passing tests, decorative formatting, repeated log
lines) while keeping signal (errors, stack traces, diff hunks, exit codes). When
it can't parse a command's output it falls back to the full raw text, so it never
silently eats data. Typical savings are 60–90% on noisy commands, at <10 ms
overhead.

## How it works
Done right this is a **double win**: fewer tokens *and* a leaner context, which
measurably improves model reasoning — every frontier model degrades as irrelevant
context grows ("context rot" / "lost in the middle"). Done wrong —
over-compressing output you actually needed — it hides detail and triggers
re-runs that cost more than they saved. The rest of this skill is how to stay on
the winning side.

```
You run: git status
RTK rewrites: rtk git status
RTK filters: smart filtering · grouping · truncation · deduplication
Result: ~200 tokens enter context instead of ~2,000
## Install — two ways

### 1. Auto-rewrite hook (recommended)

Let a `PreToolUse` hook rewrite Bash commands to their `rtk` equivalents
automatically, so you never prefix by hand and nothing gets forgotten. RTK ships
this for natively-supported agents:

```bash
rtk init -g # installs the PreToolUse rewrite hook, then restart the agent
```

## Core rule
RTK natively targets Claude Code, Copilot, Cursor, Gemini, Cline, and more.
Command Code also supports `PreToolUse` hooks — register one that pipes the Bash
command through RTK's rewrite (see the Command Code hooks docs and `rtk init
--help`). With a hook installed you **run normal commands**, and only need the
fidelity rules below for the cases where you want to *bypass* compression.

Prefix supported shell commands with `rtk`. Run `rtk git status` instead of
`git status`, `rtk cargo test` instead of `cargo test`, `rtk read file.rs`
instead of `cat file.rs`.
### 2. Manual prefixing (no hook)

Most-used commands:
Without a hook, prefix commands yourself per the tiers in
[references/commands.md](references/commands.md). Reliable, but you have to
remember it — and `|` pipes and `<<` heredocs bypass the rewrite.

| Instead of | Use |
|---|---|
| `git status` / `git diff` | `rtk git status` / `rtk git diff` |
| `ls -la` | `rtk ls` |
| `cat file` | `rtk read file` |
| `grep -r "x" .` | `rtk grep "x" .` |
| `cargo test` / `pytest` / `npm test` | `rtk cargo test` / `rtk pytest` / `rtk jest` |
| `docker ps` | `rtk docker ps` |
## Decision rule: compress noise, preserve signal

Full command list: [references/commands.md](references/commands.md).
Savings analytics: [references/analytics.md](references/analytics.md).
- 🟢 **Compress freely** — large, noisy, low-stakes output you skim:
`rtk ls`, `rtk git status`, `rtk git log`, `rtk docker ps`, `rtk pip list`.
- 🟡 **Default mode only** — big runs where you need the failures: `rtk cargo
test`, `rtk err <cmd>`. Plain `rtk` keeps errors/diffs — don't add `-u`.
- 🔴 **Keep full fidelity (run raw)** — diffs/patches you'll apply, JSON or
`--format` output you'll parse, secrets, small outputs, and files you'll edit
(use the native Read tool).

## Prerequisite
Full tiered table: [references/commands.md](references/commands.md).

RTK must be on PATH. If `rtk --version` fails, install it:
## Fidelity ladder

- **macOS:** `brew install rtk`
- **Linux/macOS:** `curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh`
- **Windows:** download `rtk-x86_64-pc-windows-msvc.zip` from the [releases page](https://github.com/rtk-ai/rtk/releases) and put `rtk.exe` on PATH.
Use the *least* compression that still answers the question:

## Ultra-compact mode
```
raw / native Read → rtk <cmd> (near-lossless, default) → -u / -l aggressive / rtk smart (lossy, skim-only)
```

Add `-u` / `--ultra-compact` for maximum savings: `rtk git status -u`.
Start as far left as the task needs. Escalate compression only for big, boring
output; escalate *fidelity* (drop back to raw or `rtk proxy <cmd>`) the moment a
compressed view is missing something — once, deliberately, not by re-running
blindly.

## When NOT to use RTK

- RTK filters **command output**, not conversation messages.
- Piped commands (`|`) and heredocs (`<<`) bypass the rewrite.
- No auto-rewrite hook in Command Code — prefix each command with `rtk` explicitly.
- Diffs/patches you'll apply, JSON/`--format` you'll parse, secrets, small
outputs — run raw.
- Files you'll edit — use the native Read tool (lossless; bypasses RTK anyway).
- It filters command output, not conversation messages; `|` and `<<` bypass the
hook.

## Verify savings
## Measure net savings

```bash
rtk gain # session summary
rtk gain --graph # 30-day savings chart
```
`rtk gain` shows gross savings; `rtk gain --failures` shows what passed through
raw; `rtk discover` finds good new targets. Optimize **net** tokens (savings
minus re-runs), not the headline number. Full reference:
[references/analytics.md](references/analytics.md).

## Prerequisite

RTK on PATH (`rtk --version`). If missing:

- **macOS:** `brew install rtk`
- **Linux/macOS:** `curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh`
- **Windows:** download `rtk-x86_64-pc-windows-msvc.zip` from the [releases page](https://github.com/rtk-ai/rtk/releases) and put `rtk.exe` on PATH.

Savings vary by command and output size; `rtk gain` reports your actual
numbers. See [references/analytics.md](references/analytics.md) for the full
analytics reference.
RTK is optional; never block work to install it.
44 changes: 25 additions & 19 deletions references/analytics.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
# RTK Analytics
# RTK Analytics — measure *net* savings, not just gross

Quantify token savings with `rtk gain`, and find more opportunities with
`rtk discover`. This is the canonical analytics reference (command rewrites
live in [commands.md](commands.md)).
`rtk gain` reports **gross** tokens saved. The number that actually matters is
**net**: gross savings minus (a) tokens spent re-running commands when a
compressed view hid something, and (b) the standing cost of these instructions in
context. Optimize for net.

| Command | Description |
| Command | Use it to |
|---|---|
| `rtk gain` | Session summary: tokens saved, efficiency meter |
| `rtk gain --graph` | ASCII bar chart of daily savings (last 30 days) |
| `rtk gain --history` | Recent command history with per-command savings |
| `rtk gain --daily` | Day-by-day breakdown |
| `rtk gain --weekly` | Weekly breakdown |
| `rtk gain --monthly` | Monthly breakdown |
| `rtk gain --all --format json` | JSON export for dashboards |
| `rtk gain --quota` | Monthly quota savings estimate |
| `rtk gain --failures` | Commands that bypassed RTK (parse-failure log) |
| `rtk gain --reset --yes` | Reset all saved-token counters to zero |
| `rtk discover` | Find commands that could benefit from RTK |
| `rtk discover --all --since 7` | All projects, last 7 days |
| `rtk gain` | session summary: tokens saved, efficiency |
| `rtk gain --graph` | 30-day savings trend |
| `rtk gain --history` | per-command savings — see where RTK actually pays off |
| `rtk gain --failures` | commands RTK **couldn't parse** and passed through raw |
| `rtk discover` | find *good* new opportunities (don't blanket-apply) |
| `rtk session` | RTK adoption across recent sessions |
| `rtk gain --all --format json` | export for dashboards (run raw if you'll parse it) |

Run `rtk gain` periodically to quantify actual savings; numbers vary by
command and output size.
## Reading the signal

- **High `--history` savings on noisy commands** → working as intended; keep going.
- **Entries in `--failures`** → RTK fell back to raw output for those, so they
aren't saving — and may be a poor fit (structured or edge-case commands). Run
them raw and stop wrapping them.
- **You re-ran a command raw right after its `rtk` version** → that pair was a net
*loss*. Note the command type and stop compressing it.
- **`rtk discover`** surfaces high-volume, noisy commands worth wrapping — a far
better guide than wrapping everything by reflex.

Savings vary by command and output size; let `rtk gain` show your real numbers
rather than assuming the headline 60–90%.
Loading
Loading