Since I started coding with AI full time in February (“AI engineering” if you will), I have felt a constant race between AI’s progress in my project and my own understanding of it. I wrote about this (and using code tours to help) in March, about a month into that project.
I’ve found that I can notice when I’m taking cognitive shortcuts. When I ask the AI where something is in my codebase instead of working to remember it or search through the code, which would strengthen my memory. When I ask AI to make a small change and I don’t care to know where it made it. When I click through a code tour passively, rushing to get through it without analyzing the decisions that it’s documenting.
Since then, I have made a habit of doing all of those things. I have actually stopped using code tours, because, for the most part, I don’t look at code anymore. There is always another feature that has to be delivered next week. It’s hard to slow down, and I let my understanding lag far behind.
It is not just that I don’t read the code, but I stopped being able to fully understand the architecture of the app. I started forgetting why features were implemented they were, and sometimes whether we even had specific functionality. I stopped having a sense of whether my code was good, by any metric other than “it works”.
Over the past few weeks, I have started an intentional effort to catch up. I now budget a couple hours every day towards cleanup and comprehension.
- I built an
/explainskill that prompts AI to produce explanations that fit how I process information: step by step, one self-contained statement at time. I’ve used this to understand existing decisions, new changes, and infrastructure concepts, and I will often just respond/explainto a rambling agent. - I armed a gaggle of subagents with a combination of code review skills to review every file in the project, write a detailed report, and chart a remediation path. The goal was to make sure the codebase was in a good place before I focused on learning it. Along the way, I learned about some of the problems we were facing and what we were going to change to fix it, giving me a head start on understanding the architecture.
- I started a humans-only wiki for the project. It is fully git-committed alongside the code, but I edit in Obsidian and use
[[internal links]]. This allows me to work on understanding the project piece by piece. So far, I’ve been writing up our AWS infrastructure. I have carved out a space for myself. - I started a comprehensive E2E testing system. This serves not just as verification but as a catalog of functionality.
I’m directing my comprehension efforts top-down. My first goal is to lay out all of our features, then understand how they work at a high level, learning prerequisite concepts (e.g. aspects of AWS resources that our project is built on) along the way. If I’m curious and have more time—or the feature is particularly important—I can dive into the code. I did this for our audit logging system, but I probably won’t do this for our data export system, for example.
I see it as a series of levels (maybe a plant with roots is the right analogy). At the top level is understanding what the feature (or change) is. The next step is to understand how it works at the highest level. The lowest level is the code itself, but for a complex feature there are many layers of understanding the highest level of “how it works” and the code. It’s also helpful to understand why we have this feature and why we’ve built it this way.
The priority is breadth of understanding before depth. As time, importance, or interest allows, reading the code can enrich that understanding.
My goal is to understand how it works so I can reason about the system and make informed engineering and product decisions. My goal isn’t to be able to work in the codebase myself. That’s a concession, and I’m a bit sad to make it, but if agents writing the code nearly 100% of the time and there’s too much of it for me to read, it’s the right prioritization.
I want to focus my next few posts on the efforts I’m making to catch up with my codebase. The wiki especially has made me feel much more confident about overcoming comprehension debt, and I can see my progress through my notes.
I’ve also been gnawing on the endgame of AI coding. Will it get so good that even understanding how it works isn’t relevant? I’ve been thinking about the idea of “understanding to participate” from Geoffrey Lit. We may get to the point where you can build perfectly production-ready products by talking to AI. It will become a choice to be involved in the engineering, a choice to understand.