Brian Roberg's Blog

The Power of LLM Context

Today I learned how much leverage you can gain by using a tool that automates feeding context to an LLM.

Over the past couple days I've been spending some time in Claude Code, a beta tool from Anthropic that semi-autonomously feeds files from a directory tree on your computer to Claude models. I've been working on re-implementing a Python application that gathers survey data covering hundreds of data points into a coherent report. In February I committed to delivering a working system by the end of March. Having realized I'd painted myself into a corner with a prior implementation attempt, I decided to start over using Claude Code to see whether it helped me make progress faster. It has.

The power of this approach—and I imagine other LLM coding systems work similarly—is the way it automatically sends relevant data from the project to the model. Previously I'd interacted with models mostly by manually copy-pasting files into a chat interface. This works, but it requires the following on my part:

Claude Code, by contrast, not only has read access to everything in the directory tree where you run it, but will automatically pull files into its context as needed. It can also write directly to files in the directory tree, almost taking me out of the loop altogether. (Before it writes to a file, it shows you a diff of the changes it's proposing and asks for permission to proceeed.) This ability to act semi-autonomously is what lets Anthropic bill Claude Code as "agentic."

Buzzwords aside, there's a big difference in friction between these two modes of working. When the model can access data on its own, it can move much more quickly and with better awareness of the code in the project. Applying edits is far smoother than copy-and-paste from a chatbot, or even than in-editor tools like Copilot. The proof is in the pudding, of course, and this afternoon my new implementation surpassed the capability of my previous attempt in (at most) half the coding time. (And this iteration still has good momentum going forward, whereas the previous attempt had stalled because of mistakes I'd made in its design.)

I should point out that lower friction doesn't remove the need for good prompts; in fact you might say it makes prompting more important. With reduced friction, you can end up way off track all the faster. I've been following a strategy (via) that begins with an iterative building-out of project requirements that itself is LLM-assisted, and it seems to be working quite well. (I am fascinated by how important good prompting is, and what you can discover about models when you prompt creatively. I hope to write soon about another project I've been working on exploring this.) Similarly, the person operating a tool like Claude Code still needs to understand the code so as to keep the model heading in the right direction.

But the real flipside of reduced friction is increased token use. (Commercial LLMs like Claude and GPT-4o are billed in units called "tokens," the basic units of meaning for LLMs.) Copying-and-pasting into a chatbot tends to be quite parsimonius token-wise because you can be very selective about what you send the model. A context-filling tool like Claude Code is another thing entirely, and simple numbers bear this out. In the time since Claude became my go-to commercial model in January, the most I spent on Claude API calls in one day (before today) was just over $5. My total for today stands at $36.80. This seems to be the fundamental tradeoff with tools like this: you're paying a cost for sending much more data to the model, but the result is so much better that it's worth it. I'm confident that's the case with what I was able to produce today, as the financial outlay is trivial compared to the potential value of this report for my organization. I'm sure there are lots of people who have (or can learn) enough coding skill to make a tradeoff like this quite attractive to them.