Datasette Can Now Run on Your Own Local LLM

Datasette’s LLM plugin now works with any model that speaks the OpenAI API format. That includes models running on your own laptop. Simon Willison released datasette-llm 0.1a11, which adds support for OpenAI-compatible endpoints. The plugin sits in the middle: it handles model configuration so other Datasette plugins can call an LLM without each one setting it up separately.

The change is small, but it means Datasette’s AI features no longer depend on a hosted API. If you’re running LM Studio, Ollama or llama.cpp, you can point Datasette at your own machine and keep everything local.

What this release changes, in 5 points

  1. Support for OpenAI-compatible endpoints The OpenAI API format has become what Willison calls a “pseudo-standard web protocol.” Most local model runners and plenty of hosted providers copy its request format. Datasette now supports it, so you can connect to “an LLM hosted anywhere” that uses that format. No custom integration is needed for each provider.
  2. One shared setup for other plugins datasette-llm doesn’t do much by itself. Its job is to handle model configuration so other plugins can use it. You set a default model once, and any plugin built on datasette-llm picks it up. That keeps model settings in one place instead of spread across separate plugins.
  3. Setup takes one command Willison shared a working example. You download a sample database (Datasette’s own content.db). Then you launch Datasette with uvx, a tool that runs Python packages without a permanent install, and add the datasette-agent plugin. The model goes in one config setting with three values:
  • model: a local label, such as “lmstudio-qwen”
  • model_name: the model ID your runner exposes, here “qwen/qwen3.8-27b”
  • api_base: the local endpoint, here LM Studio’s default at http://localhost:1234/v1

The result, in Willison’s words, is “a local server running a Datasette Agent instance powered by your local model.”

  1. Datasette Agent with local models This is where it gets practical. Datasette Agent lets an LLM work with the data in your Datasette instance. Until now, that meant sending your data to a hosted API. With a local model, you can let an AI agent query your databases while the data stays on your computer. That matters a lot for sensitive data, offline work, or anyone who doesn’t want to pay per token for experiments.
  2. Good timing for local AI Willison also noted that llama.cpp just hit v1.0.0. That project is the engine behind much of the local model ecosystem, and a 1.0 release suggests it’s ready for more serious use. Pair a stable runtime with capable open models like the 27-billion-parameter Qwen in the example, and running data tools on your own machine looks a lot more realistic than it did a year ago.

Who can use it

Datasette and its plugins are open source and free. With a local model, your only cost is hardware. A 27B model needs a fairly powerful machine, though, so smaller models are the easier starting point on modest laptops. Since the plugin works with the standard OpenAI format, you can also point it at hosted providers that offer compatible endpoints.

Caveats to know

  • It’s still alpha. The version number, 0.1a11, says it plainly. Expect configuration details to change between releases.
  • The example is for local testing only. The –root flag signs you in as an admin user, and –secret 1 sets a throwaway signing secret. That’s fine on your laptop. Don’t do it on a public server.
  • Results depend on the model. Agent tasks lean hard on tool use and reasoning. A local model may handle them less reliably than top hosted models, so test before you rely on it.

Why it matters

AI tools keep moving toward model-agnostic designs, and this release is a clean example of that. Datasette doesn’t bet on one provider. It uses the protocol most of the industry already speaks, and lets you choose whether your data goes to the cloud or stays home. For journalists, researchers and anyone working with private datasets, a local, free, agent-capable data explorer is worth paying attention to.

Willison’s original post has the full command and configuration details.

Scroll to Top