Crafting Software with LLMs

Recently I’ve been working on cavalier_contours, using LLMs as I refactor, optimize, and improve the project. Working this way has made me reflect on what motivates me and where I find meaning in building software.

I started cavalier_contours before LLMs were practical tools for working with code; returning to it now with an LLM has been a bit jarring. I’ve been trying to preserve the style and organization I originally breathed into the project, even as I overhaul and refactor large parts of the code to make the improvements I have in mind.

I keep trying to move quickly without giving up that style and organization. Having an LLM work on my project is like bringing in another engineer to work on code I built and own. Even if the engineer is brilliant and makes great decisions¹, the code starts to lose its original feel. I find myself fighting to retain a personal touch, even in choices that have no technical impact².

And then there are all the decisions my past self made that I now dislike. So it’s not just a matter of always preferring “my style”; I’m also trying to refine and improve what “my style” is. This conflict churns on as the LLM introduces its own style far faster than any human could. With an LLM, I can change things so easily that I keep second-guessing myself; in the past, changing something took enough work that I would settle for good enough.

I am having to reflect on what the goals of the project are and what keeps me coming back to it. Part of what drew me to the project was crafting and building the code itself: taking on the challenge of implementing its raw textual representation felt like an accomplishment, and it was something I found satisfying about the project. I built these types and functions byte by byte, one key press (maybe autocomplete) at a time! I found meaning in taking the time to make these types and functions great, knowing they would be a lot of work to change later.

Now an LLM is generating the types and structures to implement a goal I describe. It chooses how to break things down in code, names things, and arranges the control flow. I am still reviewing and reshaping what it generates, often substantially, but I am no longer the one discovering and piecing together the types and functions directly. This gap has stripped away much of the meaning and connection I used to find in shaping the code myself.

I think the shift from hand tools to power tools is a useful analogy. Building a high-quality wooden cabinet with power tools still demands skill, but using them changes the skills required and the experience of building things. Power tools also create an expectation that you can go faster and do more. Power tools make you beholden to how they work: they force you to solve new problems unique to using them, but you still draw on much of the same knowledge about building and working with wood as you do with hand tools. Hand tools also preserve subtleties that power tools lose: quirks or imperfections from the builder, or a perfected quality that power tools cannot quite reproduce.

You can always fall back to hand tools, but doing so is often slower and less effective. To stay competitive, you have to master the skills needed to get the most from power tools. Power tools also change how we value things. A perfectly rounded and sanded wooden component for a cabinet has the same practical utility regardless of how you make it, but power tools make it so much faster and easier to produce that they diminish both its perceived value and its market value.

Linus Torvalds recently described AI as a tool in software development, much like a compiler: compilers generate assembly, and assembly itself abstracts away the need to know hardware-specific offsets and machine code.³ The analogy isn’t accurate in every respect: compilers produce deterministic (or at least predictably bounded according to a specification) output, while an LLM’s output follows no comparable specification. But when it comes to tool use, the analogy works well: humans use tools to amplify their capabilities, and LLMs do the same for building software.

Experience and knowledge of computers, software, and engineering matter more than ever because an LLM can amplify what you bring. Give a master woodworker a set of power tools, and they can do everything they did before and more. But coming back to this project that I started before LLMs, I realize LLMs have killed some of the joy in making decisions about the code itself.

No matter how deeply involved I am in the engineering (researching, learning, designing, solving problems, reviewing, and refining what the LLM generates), I still have a deeper feeling that the code is not my code. Even if I rename and reshape every function, every type, and every variable exactly the way I think is best, I’m still refining what the LLM has already generated. I suppose it’s similar to how I don’t see the assembly compiled from my Rust code as my code.

Fixating on tiny stylistic details takes time and doesn’t interest me much, and I can never reclaim that deeper artisanal connection I once had to the code, so I am letting that go. The code’s style must be good in the sense of practical impact and utility: I have to be able to review the code without pain and be willing to take ownership of it. It’s OK that my connection to the text itself is not the same as it once was.

But LLMs let me focus more on other aspects that have always given the work meaning to me: pursuing knowledge through research and experiments and measuring how well a solution holds up against the problem I’m trying to solve. LLMs let me do more:

• Research and think about the problem and application space
• Architect and design systems with correctness, testability, and optimization in mind
• Find and fix more bugs, faster
• Build out deeper and richer tests

The project and pursuit are still mine, and I can pursue more of my ideas than ever before, but the experience of building software has changed, and it’s never going back to the way it was.⁴