Simon Willison has released llm-echo 0.5, a new version of the debug plugin for his LLM command-line tool. The plugin is unusual: it offers a “model” that doesn’t involve an LLM at all. It just repeats back whatever prompts it receives. According to Simon Willison, the main change in this release is that the simulated token usage counts now include tool calls, system prompts and reasoning tokens.
It’s a small release, but it tells you something about where LLM tooling is going. Willison says he’s working toward a new LLM release, and testing pull requests for it led him back to this plugin.
What llm-echo 0.5 Actually Does
- It echoes prompts instead of generating text. You send a prompt and get back a structured view of what the model received. That includes the prompt text, the system prompt, any attachments, whether streaming was on, and any previous conversation turns. Nothing is generated and no API is called.
- Token counts now cover more of the request. Earlier versions gave a simpler picture. Version 0.5 updates the simulated usage numbers to account for tool calls, system prompts and reasoning tokens, which are the parts of a modern LLM request that tend to break accounting code.
- It can simulate a model that “thinks.” With the thinking option turned on, the echo model adds a placeholder reasoning line (“First I consider the prompt, then I decide what to say.”) before the echoed payload. That lets developers check how their code handles reasoning output without paying a real reasoning model to produce it.
- It runs with a single command. Willison’s example uses uvx to pull in the plugin and run it in one go:
uvx --with llm-echo llm -m echo 'hi there' -o thinking 1 --usage. The output is the echoed JSON plus a usage line: “Token usage: 2 input, 24 output.”
Why a Fake Model Matters
It sounds like a toy, but it isn’t. Testing software that talks to LLMs is awkward. Real models cost money, need network access and API keys, and give different answers each time. None of that works well in a test suite.
An echo model fixes all three problems. It’s free, it runs offline and it’s deterministic, so the same input gives the same output every time. When you’re building plugins, logging, cost tracking or tool-calling features on top of LLM, you can check that the plumbing works before any real model gets involved.
The token counting update is the bigger deal. Usage accounting used to be simple: input tokens in, output tokens out. Now a single request can carry a system prompt, tool definitions, tool call results and hidden reasoning tokens, and each one affects cost and context limits. If your test model ignores those, your tests won’t catch bugs in the code that handles them.
Who It’s For
This plugin is aimed squarely at developers:
- LLM plugin authors who need a stable model to test against
- Contributors to LLM itself, which is exactly why Willison came back to it
- Anyone building cost or usage dashboards on top of LLM who wants to check that the numbers flow through correctly
- Teams writing CI tests for LLM-powered scripts, where calling a paid API on every run isn’t practical
Availability and Caveats
llm-echo is a free plugin for the open source LLM tool. You can install it like any other LLM plugin or run it on the fly with uvx, as in Willison’s example.
One caveat is built into the design: the token counts are simulated. They show what your code does with usage data, not what OpenAI, Anthropic or Google would actually bill you. Treat the numbers as test fixtures, not estimates.
You’ll also notice the reasoning output is a fixed placeholder line. That’s fine for testing how your code handles a reasoning block, but it tells you nothing about how a real model reasons.
What Comes Next
The more interesting news is what this points to. Willison says he’s preparing a new LLM release, and the focus of this update (tool calls, reasoning tokens, system prompts) suggests better handling of exactly those features. Tool use and reasoning models have become standard, and tools that track usage need to keep up.
For developers who rely on LLM, it’s worth keeping an eye on that upcoming release. Simon Willison’s original post has the full details and example output.