Kliv
Blog

Scaling yourself when AI writes the code

Anders Olsson

Founder, Kliv

Almost all of our code at Kliv is written by AI these days. I suspect the same is true in most engineering organizations by now, with more or less manual review and oversight on top.

For us, writing the code by hand isn't really an option anymore, given what we're trying to build. Even reviewing more than a fraction of it by hand is a challenge.

I'm planning to write a series of notes about how we deal with that. This first one is about scaling yourself. How do you produce software of the highest possible quality when you suddenly have what is essentially an army of engineers?

It turns out that the bottleneck is mostly you, the human in control. Frontier models are very smart, and they write better code than most engineers. What they still lack is some of the ability to look at the bigger picture, keep track of what's important, and make sure things move towards the goals of the organization. So how do you manage an army of agents, and how is that different from managing a team of humans?

From one session to a dozen

When you install Claude Code, or whatever your favorite agentic harness is, what you get is effectively a management problem instead of an engineering one. You have an extremely capable engineer that you essentially pulled in from the street. They have no institutional knowledge, but an endless ability to follow rules, instructions and engineering processes.

So as the manager, how do you build a good team out of dozens or hundreds of these agents, all working on the same project? You have to give them the knowledge they need. You have to set the engineering standards, too. Maybe you want the NASA Power of 10 rules, or formal validation of the logic, or over 90% coverage in a mutation test suite. You have to decide how code reviews work, and how many rounds you want before code can be committed. I'll cover how we do all of that in separate posts.

This post is about the coordination side. I think most people who use Claude Code or Codex start with a single window. Then they slowly start opening more sessions, to work on several things at the same time. But how far can you take that? How many things can you work on at once and still be productive? And when do you manage a project across several sessions, and when do you let a single agent run it with subagents?

Within days you have dozens of initiatives running, and it's on you to keep track of them. Projects stall out. Specs get written but only partially implemented. Agents ping you for feedback.

Projects stalling out was a material problem for us for the longest time. We built an ambitious commerce engine that took several months of engineering effort to bring to the rigor we were aiming for, and getting it to the finish line was a struggle. After reviewing it and trying to restart progress, Claude wrote a file named WHAT_IS_ACTUALLY_LEFT.md. The file opens with "This document should have existed on day one." I felt that was a good summary of our lack of a forcing function at the time. Here's a paragraph Claude wrote in it that I think still holds value today:

Each workstream closed with an honest D5 gap list. Nobody read the five lists side by side, so a gap recorded in one ticket and a gap recorded in another never became a single backlog — and "we wrote it down" stood in for "we did it".

After this, we decided to make changes so that we drive projects all the way through, past the initial phases to everything that's needed in terms of observability, robustness, release runbooks and so on. Our definition of done helps with this, but we also needed a forcing function to make things happen.

At some point you decide you need a war room to manage all of this, an agent that brokers the requests from all the other agents and manages the integration phases. That helps. But today's frontier models are still not good at keeping track of long-horizon tasks that span months.

So you need some technology to help manage the agents, a forcing function that makes projects and tasks move forward relentlessly without stopping. Many companies have sprung up around this. We built our own.

Lanes

We call it lanes. It's our in-house command line tool. It has a SQLite database that tracks all initiatives, a CLI for the model to use, and a built-in web server that gives me a web interface. It also hooks back into Claude Code, so it can push a notification when something happens, for example when an initiative stalls out. It's essentially a set of swim lanes for work streams. Everything in it is tied to something similar to OKRs, and to our definition of done. Every initiative has a set of key results, and underneath those are the work units that work towards meeting them.

The lanes web page for a demo project. Three agent steps are running. One question waits for the owner: whether a card form should validate while typing or only on submit, with three options and a recommended answer. Below it, the initiative Checkout works first time shows its three key results, one of them met.
The lanes web page.

The rule that does the forcing is simple. Every work unit has either a running step or a named reason it's waiting. If it has neither for about 15 minutes, it gets flagged and the war room pushes it forward.

A terminal session with the lanes command line tool. lanes stalled lists one stalled lane. Trying to mark it as waiting with a vague reason is refused with an error that asks for a blocker, a date and a reason. The lane is then deferred until a date, with a reason and an external blocker, and lanes stalled reports zero stalled lanes.
A wait without a typed reason is refused.

The war room also steps back and looks at the big picture. Individual work streams tend to end up in myopic thinking, and someone has to catch that.

Specs define what done means

Before we start on anything substantial, we workshop the requirements. I describe what's needed, and Claude writes a first draft of the spec. Then Claude and Codex go back and forth, critiquing and improving it. The spec is split into phases and includes a definition of done. That's the building block for a new entry in the lanes interface. When the work is done, a model from a different model family double-checks it. This is iterative, and we keep looking for issues until the code meets our rigorous standards.

Staying in the feedback loop

From there, lanes tracks and drives the work. That's how we keep attention on long-horizon tasks that can take weeks or months to implement. It calls back into Claude Code whenever something needs attention. When there's a question for the human, which is me, it gets registered. I can read it in the web interface, and I get a push notification on my phone.

Some parts of the work are hard gated for my review, such as DDL changes that differ from the spec, and UI drafts. Most of the rest is cross-reviewed by the models automatically. I do a final review only when it's needed, based on the big picture and the final results.

The DDL gate has worked well for us. The data model is a good proxy for the architectural direction, so it tells me quickly if something is going down an incorrect path. With the UI, it's simply that the models, in the current iterations at least, tend to build a UI that is too verbose, or not what a human would want to see.

These days I mostly spend my days talking to the models through voice transcription, following the dashboard, tracking the conversations, and course correcting as issues pop up.

Things are moving so fast right now that updating your mental model is almost a full-time job. Lanes works well for us today, as we aim to build software at the 99th percentile of the industry. It wouldn't surprise me if it's obsolete in six months, though.

I dictated this post. Claude critiqued and edited it, and did the research into our notes and history.

Anders Olsson

anders@kliv.dev