Keep Big Data Out of Your LLM’s Context Window

A team building RidgeText, an SMS-based mapping tool, hit a wall that any developer wiring data into an LLM will recognize. As detailed in a Hacker News writeup that pulled a score of 164, they wanted the model to generate a fire perimeter map with a trail route layered on top, rendered as one image and texted back. The lesson they walked away with is blunt: you cannot pass GeoJSON through an LLM tool call. It’s too much data to be useful. Here’s how they fixed it, and how you can apply the same pattern to any tool that fetches heavy data.

What you’ll learn

A design pattern called “layer-first” that keeps large payloads out of the model’s context window. You’ll see why the naive approach breaks, and the exact tool structure that replaces it. Basic familiarity with LLM tool calls (functions the model can invoke) is all you need.

Step 1: Understand why the naive approach fails

The obvious build is two tools. One fetches fire data and returns it to the LLM. A second accepts that data and renders a map.

• LLM calls get_wildfire_data() and receives 2,000 fire polygons as GeoJSON
• LLM calls render_map(geojson: …) and passes that GeoJSON along

Why it breaks: a modest wildfire dataset is 50 to 500KB of raw GeoJSON. A token is roughly 4 bytes, so 500KB is about 125,000 tokens. That’s larger than many context windows, and expensive even when it fits. The model becomes a pipe for data it cannot reason about. It can’t simplify the GeoJSON, can’t validate it, and pays the full context cost every time.

Step 2: Store the data server-side, return only an acknowledgment

Instead of handing data back to the LLM, each fetching tool stores its result on the server and returns a tiny receipt:

• retrieve_wildfire_layer(location: “Cascades”) gives back { status: “queued”, layerId: “wildfires-0”, featureCount: 847 }
• retrieve_trail_layer(trailName: “PCT Section J”) gives back { status: “queued”, layerId: “trail-1”, featureCount: 1 }
• generate_map() gives back { mapUrl: “https://storage…/map-abc123.jpg” }

Why it matters: the model can’t truncate, hallucinate a summary, or fail silently on data it never touches.

Step 3: Queue layers in an ordered stack

Each retrieve_ call appends to an ordered array held in the request context:

interface MapLayer {
  type: 'wildfire-perimeters' | 'fire-hotspots' | 'trail' | 'heatmap' | ...;
  data: GeoJSON.FeatureCollection;
  style: LayerStyle;
}
const layerQueue: MapLayer[] = [];

Layers render in insertion order, so earlier layers sit below later ones. The LLM controls stacking implicitly through the sequence it calls tools. Call retrieve_satellite_base before retrieve_trail, and the trail draws on top.

Best practice from the team: they used an in-process Map keyed by session ID with a 30-minute TTL, so layers queued but never rendered get evicted instead of piling up. The mechanism isn’t the point. Redis, a database, or a request-scoped object all work. What matters is that the data lives somewhere other than the context window.

Step 4: Make the render step deterministic

generate_map receives no GeoJSON, only optional parameters like zoom or map style. It drains the queue and composites the image:

async function generateMap(options: MapOptions): Promise<string> {
  const layers = drainLayerQueue();
  const base = await fetchMapboxBase(options);
  const composed = await compositeLayers(base, layers);
  return await uploadToStorage(composed);
}

The result is a single image URL the model can attach to its reply.

Why this mirrors Mapbox

Mapbox’s model is simple: sources provide data, layers define rendering, and layers compose in declaration order. The server-side queue is the same abstraction without a browser. Mapbox never sees the GeoJSON, it only supplies the background tile. Storing layer descriptors in Mapbox’s format is a forward-looking bet: if static tiles stop being enough, they can swap in a headless Mapbox GL renderer that consumes the same queue with zero changes to the tools or the LLM.

Next steps

Audit your own tools. Any function that returns more than a few hundred tokens of structured data to the model is a candidate for this pattern. Move the payload server-side, hand back an ID and a count, and let a deterministic final step do the heavy lifting. Adding a new data source then becomes one new retrieve_ tool, and your render pipeline never changes. Full code walkthrough is at the original Hacker News source.