Code graph: how my_architect is aware of the architecture without taking the index at its word

The agent with amnesia

An LLM agent has no memory between sessions. Every new conversation about a repository starts from a blank slate: the agent greps again, reads the same files again, builds the same map in its head again — and throws it away at the end of the session. We measured this on our own repository: gathering facts about a well-known codebase cost roughly 77 thousand tokens and 20 tool calls — and that's every session, on repeat.

A separate pain is questions like "what breaks if I change this function's signature?" Answering requires knowing every consumer of the symbol. Grep finds them, but at the price of a full pass over the repository, and — crucially — that knowledge has nowhere to survive past the session.

Around this pain a market of "code-as-graph" tools grew up, with slogans on the order of "90% token savings". We didn't need slogans, we needed an answer to two questions: which classes of error this removes, and where the savings are real versus where they're hype.

Not building our own: Graphify

A tool for persistent memory of code already exists — Graphify (open source, MIT). It parses code via tree-sitter fully locally, with no API calls or telemetry, stores a graph of symbols and relations in graphify-out/graph.json, and drops an interactive HTML map and a ready text report about the repository next to it. Index freshness is kept up by git hooks; for teams there's a merge driver so the graph can be committed without conflicts.

We deliberately didn't build our own graph engine. Instead we taught our decomposition-discipline skill recursive-context to recognize an index lying in the repository and use it correctly. The whole feature is a markdown reference of rules plus a few pointer lines; the plugin didn't gain a single hard dependency. No graph — everything works as before; no graph and a heavy task — the agent offers, once, to have the owner install and build it (and does so itself after a "yes"; silent installs are forbidden).

How cheap this is in practice: on our repository (around 339–351 files) building the graph took seconds and produced 2655 nodes, 4948 edges and 153 automatically found clusters — about 6 MB on disk, zero API cost. The report immediately surfaces "god nodes" (for us that's HierarchyNode with 84 edges), import cycles and "unexpected links" we'd forgotten about ourselves. One query after that returns every consumer of a function, with paths, in a second — what used to require a full pass. An important detail for all the downstream economics: the graph itself spends no tokens at all — both building and querying are local utilities, not model calls.

The rule this was all for

An index is a cache, and a cache can lie: it lags behind the code between rebuilds. So at the center of the feature aren't commands but three rules.

First: the graph is navigation and candidates, NOT a source of facts. The graph's answer says WHERE to look; something becomes a fact only once it's confirmed by a live file. The fact contract {claim, proof-path, confidence} hasn't changed one iota.

Second: a freshness check is mandatory. The first thing the agent does is compare the index build time with the last commit time. Stale — say so out loud, offer the owner a rebuild (and if a git hook isn't installed — offer the hook too, so staleness doesn't recur every session), and mark candidates as "from a stale index".

Third: verify sparingly, by region. A candidate arrives with a path:line coordinate, and verification reads ±30 lines around it, not the whole file. Without that rule, verification would eat everything the graph saved on search.

What the tests caught

A skill is text that's supposed to change the model's behavior. You can't compile text, so the checking is layered: triggers (does the skill load itself by name and description), behavior (is the discipline followed in a plan), an adversarial review of the text itself (the reviewer doesn't read snippets politely — it executes them), and live runs against a real graph. The iron rule: first pin down "how the agent errs without the rules", and only then write the edits.

The most useful findings came from execution, not proofreading. The "no graph → offer to build" branch turned out to be unreachable: every pointer into the reference started with the words "if the graph EXISTS". The freshness-check snippet used macOS stat syntax, and on GNU the same flag means something else — the gate would forever answer STALE, and on any Linux machine graph-first would silently never turn on. We were advising hiding graphify-out/ in .gitignore — while the upstream README recommends teams commit the graph; our advice silently threw away the team scenario. And the executor of the behavioral test didn't believe the "graph is fresh" condition, checked mtime for real, discovered its environment's graph was already stale, and handled the discrepancy by the recipe — the "don't trust, verify" discipline fired against its own assignment.

The savings are real, but selective

Qualitative checks don't answer the question "how much does it cost". Only a controlled A/B with telemetry does. We took a large repository unfamiliar to both groups (NestJS, 2125 files — an order of magnitude bigger than ours) and made two identical clones: one with the graph (12309 nodes, 738 clusters) and one without. The prompts are identical down to the path, with not a word about the graph — the with-graph arm has to find it itself.

The result is sobering. On impact questions ("who consumes the symbol, what breaks") — parity on tokens: grep on the literal symbol name is cheap even at 2125 files; the graph's win there is in completeness and speed, not tokens. On a full subsystem audit — also parity: both need orchestration. But on the class "understand a subsystem end-to-end" the graph came out roughly 3.5× cheaper — a measured −71%. Not because queries are cheaper than grep, but because the graph narrowed the corpus up front, and one agent managed where the graph-less arm was forced to silently fan out five sub-agents.

And here's the methodological surprise. By the top-line numbers the graph-less arm on "understanding" looked cheaper — until we cracked open its transcripts and found 205.8k tokens of nested agents. An honest measurement has to count the whole tree of agents; we suspect a fair share of pretty benchmarks don't. There's no universal "90%" multiplier — there's a class of task. The full write-up, with the token-savings measurements and the index-check discipline is in a separate article, along with the limitations (n=1 per cell, one repository, one model tier).

The bottom line

The feature's formula is simple: an existing tool, plus the discipline of using it, plus layered tests that trust no one — not the author of the text, not a pretty snippet, not the condition of their own scenario, not the top line of someone else's benchmark. We gave the agent persistent memory of the architecture — and at the same time forbade it from taking that memory at its word. The index says where to look; only the code becomes a fact.