Engineering to the 99th percentile
Anders Olsson
Founder, Kliv
I think most of us have always had high ambitions for the code we write. We've all wanted our work to be a lighthouse example of the most correct and well-architected engineering possible. But practical matters come in between. We didn't have time to write all the tests we wanted. There was no time to write a second implementation just to test whether the first one was correct. The code reviewers got tired. The deadline decided that the code was done and had to ship. There were always some industries that built to higher standards, but most of the market didn't like the cost-benefit trade-off and built their software as best they could. Maybe there were some unit tests, which in the best case covered some of the code paths. Maybe there was no time to build tests at all. Today's AI slop compares favorably to some of the code I've seen in codebases at large firms.
That has all changed. In the last year, everything about how we practice engineering was shaken up, and it became functionally unreasonable to write code by hand anymore. We now have a new world in front of us. It is a world filled with AI slop. It is also, at the same time, a world where AI models have proved the generalized Riemann hypothesis in Lean, a task that humans have failed at for 150 years. We can choose which world we want to live in, and at Kliv we're picking the beyond-human-ability world.
Today, everyone can engineer to the 99th percentile of the industry, and hopefully that shifts the entire distribution towards higher quality. We decided that all the engineering work we do will be done to that standard. We're aiming for vastly better code than we could ever build ourselves. Frontier models still have weak spots, but they also have traits that only exist in the very best human engineers and coders. They can choose the best implementations, and when instructed, they follow the highest possible rigor without failing.
When engineering moved to AI first at Kliv, we sat down and laid out what we thought was the right bar to hold ourselves to. This is what we came up with, after a fair bit of research and down-selection of the criteria we felt would be most appropriate for our team.
- Research and a reviewed spec before any code is written
- A subset of the NASA Power of 10
- All sources of non-determinism injected, including time, randomness and UUID generation
- A second, deliberately naive implementation as a reference oracle
- Models in TLA+
- Crash campaigns
- Mutation testing and near-total coverage
- Architecture tests and SQL statement budgets
- Review by a second model family
- A step back that asks whether anyone would be surprised by this
These standards are described in two files that the agent reads, RIGOR.md and INVARIANTS.md. The rigor file contains a slightly expanded description of the development process above. The invariants file contains a set of conditions that always have to hold true. For example, multi-tenant separation must be absolute. One customer can under no circumstances see another customer's data. We have a few dozen of those invariants, which we've collected in response to what we've seen in the generated code in the past. These invariants are programmatically checked as part of our test suite to ensure that they hold.
Our second file system
Kliv provides a virtual file system for the source code that models write on behalf of our users. When users create apps that upload files, there's a file system that stores those files too. We looked very hard at what would be the appropriate file system to use, and we ultimately settled on building our own.
This is in fact our second file system. The first one developed organically as we stored data in earlier versions of Kliv. We realized that we needed to formalize it into something that could be held to the highest standards and meet the most stringent quality criteria. This post recounts some of the processes and approaches we used when building the second version.
Coding is a minority of the work
We apply the rule that the coding is ideally a minority of the work involved. Much of the highest-value work comes in the planning phases, and we put a lot of effort into that.
Before we embark on a new project or a feature implementation, we run multiple deep research jobs iteratively. We want to determine if our ideas are valid, if the industry has settled on a different approach, who's doing what, which algorithms are most applicable, if there's any research that applies, and so on. The research results are synthesized by an agent and discussed with the team. We then decide what path we want to go down, based on various criteria we've established, such as whether it's something we'll want to build, or whether we should go down a different path altogether or change our approach.
We then build a spec and workshop it with multiple models from different model families, to make sure that we get different perspectives. This is iterative. We punch holes in the spec and address the issues that come up. We also weigh that against our risk models, to find the threshold between addressing ever more exotic issues and documenting them and keeping the spec simple.
Engineering practices
In terms of engineering practices, we apply a subset of the NASA Power of 10. Specifically, that means bounded loops, short functions, dense assertions, every return value checked, and all warnings treated as errors. The Power of 10 has additional rules that we deem less applicable to our business.
We also inject all sources of non-determinism, following the lead of FoundationDB, TigerBeetle and Antithesis. We inject time, randomness, sleeps, any UUIDs that are generated, and any calls to outside systems like model providers, payments and email. Our test suite has ratcheted all usage of clocks and random numbers, which means that no code can use those constructs directly. Everything has to go through the injected sources. When we simulate, the scheduler is replaced with a deterministic scheduler that decides the order in which the threads run and is able to inject faults into key components like the database and the network stack. Failing runs can later be replayed from the same seed and tested deterministically.
We also build a second, naive implementation of the core logic, with entirely separate code that is targeted towards correctness rather than efficiency. This code is used as our pure reference oracle, and our tests run against both, expecting identical outcomes.
We then create models in TLA+. We model the protocols and check them exhaustively. Every property we claim also has a corresponding broken property that we specifically check against, to make sure that a model that proves nothing can't pass.
To stress test the system, we run crash campaigns that run a large number of processes concurrently, killing them at inconvenient mid-write locations, and making sure that after replay, everything comes back correctly again.
Mutation testing is done using a tool that changes the code in a myriad of ways, and we make sure that for each change there is a failing test. In our file system, 97% of over 2,000 changes were caught. Each surviving mutant has to be explained and documented, showing why it doesn't affect the correctness of the program. In terms of coverage, the project must show 100% line coverage and at least 98% branch coverage to pass.
Everything is also guarded by ArchUnit, a Java test framework that inspects the code and checks that it conforms to certain architectural patterns. This is what enforces the determinism and bans direct use of SDK functions for getting random numbers and dates. We guard against regressions in performance too. For example, each test can establish a SQL statement budget, and the test fails if the pinned number of SQL statements goes up, ensuring that the complexity of operations can't increase without review.
For code reviews, we demand that all code is reviewed by a second model family, and we rotate the model. So when Claude is writing the code, Gemini or GPT must review it, and reviews continue until the reviewers are satisfied. By doing so, we buy down the risk of reviewing with a model that has the same blind spots as the model that wrote the code.
The human perspective still adds something
We've found that even with these rules applied, there is still a need for humans to be involved in the development, for much the same reason that we demand code reviews from a different model family. A human provides a very different perspective and will uncover blind spots that wouldn't otherwise be uncovered.
For example, in the initial version, the maximum number of files a virtual file system could store was 50,000. This was a constraint the model imposed to follow the Power of 10 rules, where everything needed a bound. It passed through code review after code review, but no one asked themselves, is this reasonable? The fact that we can create a hyper-competent virtual file system and then apply a hard cap of 50,000 files is baffling. The rule now reads "bound the loop, but never the customer's data".
This remains a blind spot for models. They tend to create a myopic view of the code after a while.
Another example is how our ArchUnit tests were set up to guard the system, so that file system functions are always accessed through the file system interface, and nothing bypasses it by talking directly to the database or some other component. The models kept finding ever more sophisticated ways of breaking out of this, such as reflection, serialization tricks and computed table names. A human had to step in and say, wait, we are not a malicious actor against ourselves. We don't need to guard against us trying to hack ourselves. The ArchUnit tests are there to guard against mistakes, not as a security boundary.
To address this, we explicitly added a step where the model stops looking at things through a microscope and asks, is there anything here that anyone taking a step back would be surprised by? That kind of review is important. One fresh agent looked at the code with that goal in mind and found the file limit in seven minutes, the same limit that dozens of ordinary reviews had passed.
What it has given us
So far, we've had great success with this approach to engineering. We've found that entire classes of bugs that used to exist have essentially disappeared. And we can be comfortable bringing code in-house that we wouldn't have wanted to be responsible for in the past.
Building our own file system, which needs to be completely correct, is an example of something where we might have picked a third-party solution before. Now we can engineer to a much higher degree of correctness in-house, and we have the ability to build custom work that integrates much more tightly with our product than we could in the past. By reducing the number of third-party components, we're able to drive down cost and create a faster and more cohesive system.
I dictated this post. Claude critiqued and edited it, and did the research into our notes and history.
Anders Olssonanders@kliv.dev