HomeAboutExperienceProjectsContactBlog
RésuméDex0/17
← Projects

Tokenuinely

A semantic, graph-aware index of your repo, served to AI coding agents over MCP.

QUERY.RSSAMESAMESAMEWHYEFFECTSCALLSHEADER1024-D

Click a function in the file to edit it, then run the index.

The file is an example. Which symbols get re-sent follows the repo's indexer, which compares SHA-256 hashes of the file and of each symbol's body.

A file hash decides whether a file is touched at all; a per-symbol body hash decides whether a symbol needs a new header and embedding. Editing one function in a large file re-does one chunk. A file that partly fails is stored with an empty-hash sentinel so the next run retries only it.

Instead of an agent firing a dozen grep, glob and read calls, it asks one plain-English question and gets back the exact functions with source. Each symbol gets a short LLM-written header describing what it's for and what it touches; search fuses meaning, keywords and exact names over a local SQLite index.

The problem

Coding agents spend most of their tokens finding code. Vector-only search misses exact names; AST-only search misses intent. Tokenuinely does both and keeps the agent-facing surface tiny, because every tool description is paid for in context on every turn.

It ships three MCP tools (query, inspect a symbol, repo overview), a Claude Code hook that quietly adds the top three semantic matches to any Grep or Glob the agent runs, and a CLI for everything else: dependency tracing, dead code, a 3D graph viewer, and a Cypher-style query language over the call graph.

How it's put together

CLAUDECODESTDIOGREPTOP 3MCP SERVER3 TOOLSHOOKPRETOOLUSEINDEX.DBSQLITEQUESTIONHOSTED APISCLIINDEXHAIKUHEADERSVOYAGEEMBEDDINGS
Claude Code reaches the index two ways: the MCP server over stdio, and a PreToolUse hook that adds the top three matches to any Grep or Glob. Both query the same local SQLite file. The network comes in twice: at index time the CLI sends each changed symbol to Claude Haiku for a header and the header to Voyage for an embedding, and at query time one Voyage call embeds the question, skipped on a cache hit.

Trade-offs

Embed the header, not the code. Search matches what a function is for and what it touches, not which tokens it happens to use. The downside: Indexing needs an LLM call per changed symbol, on your own key.

Wipe and rebuild on schema changes. The index is a derived cache, so there is zero migration code to maintain. The downside: A schema bump re-pays the header and embedding cost.

Brute-force cosine over SQLite blobs. No native extension to install; plenty fast for repo-sized indexes. The downside: O(n) per query. sqlite-vec is on the roadmap for large repos.

Three tools and no more. Tool descriptions cost context on every turn, so niche features live in the CLI. The downside: Agents can't reach the graph queries without a shell.

Details worth a look

A hand-written tokenizer, parser and translator turn a Cypher subset (MATCH, -[:CALLS]->, WHERE, RETURN, LIMIT) into parameterized SQL. Properties and edge kinds are whitelisted, so identifiers never reach SQL unchecked.

The 3D graph viewer is a tiny hand-rolled HTTP server bound to loopback only.

An eval harness reports top-1/3/5 hit rate, MRR and latency over 22 labelled queries against its own source. Storage and search are local; headers and embeddings come from hosted APIs.

What I'd do next

Swap brute-force cosine for sqlite-vec so very large repos stay fast.

Add a reranking pass over the fused top 50 before cutting to k.

Wire the file watcher in, and make the keyword index update incrementally instead of rebuilding.

Next: WhispA private dictation keyboard for iPhone. Speech becomes text on the Neural Engine and never leaves the phone.

Year
2026
Platform
CLI + MCP server
Status
Published on crates.io (v0.4.0)
Code
Sourcecrates.io
Built with
Rust, tokio, tree-sitter, SQLite + FTS5, MCP, Claude Haiku, Voyage embeddings
  • 5,658 lines of Rust
  • 3 MCP tools, on purpose
  • 7 tree-sitter grammars
  • 33 tests, no API keys needed