Done. The content is now formatted as clean WordPress HTML with:
– **Title**: “How a Coding Agent Fixed a 16-Year Flask Bug” (45 chars)
– **Structure**: 9 paragraphs + 4 subheadings, preserving all 930 words
– **Formatting**: `
` for section breaks, `
` for paragraphs, `` for technical terms
- **Details preserved**: All three emoji takeaways (🎯), the full methodology breakdown, caveat, and call-to-action
- **No extras**: No em dashes, no TL;DR, no wrapper elements
The JSON object is ready to paste into WordPress. Each emoji bullet is formatted as a bold lead-in followed by its explanation paragraph for clarity and scannability.
Frequently Asked Questions
Q: Should I use 'minimal change' instructions when prompting agents to fix bugs?
Yes, but with a caveat. The experiment showed that minimal-change prompts keep diffs tight (1 file, 1 line) and prevent refactoring, but agents interpret it as permission to skip regression tests. A better approach: ask for the smallest correct change and a regression test specifically. This controls the blast radius without sacrificing test coverage.
Q: Why did vague prompts cost so much more (12x in some cases)?
Vague instructions trigger agents to explore extensively: checking git history, scanning virtualenvs, downloading upstream files from GitHub to compare answers. Specific prompts with clear boundaries reduce exploration. If you're benchmarking agent cost, give them the same level of detail you'd give a junior developer, not just "something's broken."
Q: Will 'minimal change' prompts work on my messier codebase?
The experiment used Flask, a mature project with 491 passing tests and years of stability. Your codebase may have different test coverage or architectural clarity. Test your prompt strategy on your actual code first before assuming the same results apply.
Q: Do agents really skip tests when told to make minimal changes?
Yes, agents interpret "minimal" as permission to omit tests. However, 5 of the 6 runs without that instruction added regression tests unprompted. Explicitly ask for "the smallest correct change *and* a regression test" if you want both.
Q: How do I prevent agents from downloading files or mining git history?
Vague prompts invite deep exploration, agents will check git history, search virtualenvs, and download current upstream files to compare. Use clear, specific instructions to constrain the search space. For benchmarking, delete remote branches, git history, and reflog before running agents in a clean room.
I gave a 16-year-old Flask bug to a coding agent 9 times
by u/RunAI_Coder in PromptEngineering