While I sympathize with the sentiment of reducing layers, I struggle with the reduction of reviews being about quality assurance (only). Nor that reviews are being done over many layers in large orgs, while everything just does quality assurance.
Reviews are as much about reducing the bus factor and making sure other people know the code base and that we have a shared understanding of what it is supposed to be doing, as it is about style, quality and so forth. But in what larger org is another layer reviewing that code, really? I have seen no engineering manager or CTO review the code after my peer had - and why would they?
Sure, the coordination problem is real. And the author correctly identifies it is about reducing risk. I have to talk to lawyers about what we are building because there is legal risk involved - and I am not an expert in that. This is an engineering profession and just as in any other engineering you have to coordinate with outside expertise and requirements.
That's also why the car assembly line analogy falls short to me and the Toyota model isn't applicable: it's not an assembly process, it's an engineering process. You are telling me the car engineers at Toyota, when designing a new car don't have to coordinate with legal, marketing, etc? Of course they do. And there designs go through an iterative process with reviews and input from various sources. Of course it does.
That Toyota assembly line thing is so omnipresent in the software development process discussion and I don't understand why. The part of our process that does this - "constructing the final product item and ship it to the customer" - we have full on automatized away with automatic testing, CI/CD and removing any physical part in the process (remember we used to ship software on CD-ROMs!?!).
If we want to learn from Toyota for our process we would have to study his they engineer new cars. But I think that isn't that interesting and doesn't have these nice short-hand learnings "you just have to truly apply".
Don't get me wrong, I am for fewer communication cycles and more trust in quality engineering. But we also have to acknowledge that there is a lot of expertise in other domains that is relevant for the success or risk that we software engineers just don't have and this require input on.
But in what larger org is another layer reviewing that code, really? I have seen no engineering manager or CTO review the code after my peer had - and why would they?
I think you may be misunderstanding what the article is describing. They’re not reviewing the code as you go up/out. Different, often higher level, artifacts are being reviewed rather than the code (eg design documents, business cases, proposals for new products etc), just like the QA layer isn’t reviewing the code either.
If I had to name something that improved the quality of code being shipped, then it's code reviews. I get the point that reviews slow you down, but don't see it as a problem and instead embrace the pause or do something else instead. Not everything has to be rushed, and you don't need to be a 100% effective all day long, except you want to burn out quickly.
I feel like that's often a statement on the quality of the code review more than anything else. Being able to describe what you're doing and answer questions about the direction/code really helps everyone in the process.
Here is the fun stuff that I think is being missed in this post.
If you do all the things to reduce the number of review layers, the (alleged) speedup from AI help... does not register anymore!
Because the speedup from reducing the layers is dominating that reduction. Which means you do not need the AI at all (It would even introduce more layers, because the human writing the code by himself is already a layer that you have to emulate now).
The struggle is real! I think the author's description of the "AI Developer’s Descent Into Madness" is spot on here. But as they also say, there are a lot of people on this Karpathy path, at the moment, so it feels more like a natural progression than madness.
Some high profile mistakes will probably help alert the industry that this is the wrong path to be on. But I worry that instead we will get used to the feeling of not actually understanding the complexity of software and how to reduce it. Arguably this is a path that the software industry has been on for some time.
I am personally hopeful when I hear people talking about using genAI coding tools to increase engineering rigor , rather than simply focusing on productivity.
Nuclear? Yes. But someone like me who isn't using AI can no longer afford to tie sandbags around my ankles and hope for the best. When I keep iterating on code, I keep seeing things I wrote and fixing and improving them even without a formal review step.
Similarly testing every change in isolation is probably absurdly slow. The better way to do it is to test 10-20 changes together.
But part of why I think it works well for me is that I'm writing all the code
gnunicorn | 6 hours ago
While I sympathize with the sentiment of reducing layers, I struggle with the reduction of reviews being about quality assurance (only). Nor that reviews are being done over many layers in large orgs, while everything just does quality assurance.
Reviews are as much about reducing the bus factor and making sure other people know the code base and that we have a shared understanding of what it is supposed to be doing, as it is about style, quality and so forth. But in what larger org is another layer reviewing that code, really? I have seen no engineering manager or CTO review the code after my peer had - and why would they?
Sure, the coordination problem is real. And the author correctly identifies it is about reducing risk. I have to talk to lawyers about what we are building because there is legal risk involved - and I am not an expert in that. This is an engineering profession and just as in any other engineering you have to coordinate with outside expertise and requirements.
That's also why the car assembly line analogy falls short to me and the Toyota model isn't applicable: it's not an assembly process, it's an engineering process. You are telling me the car engineers at Toyota, when designing a new car don't have to coordinate with legal, marketing, etc? Of course they do. And there designs go through an iterative process with reviews and input from various sources. Of course it does.
That Toyota assembly line thing is so omnipresent in the software development process discussion and I don't understand why. The part of our process that does this - "constructing the final product item and ship it to the customer" - we have full on automatized away with automatic testing, CI/CD and removing any physical part in the process (remember we used to ship software on CD-ROMs!?!).
If we want to learn from Toyota for our process we would have to study his they engineer new cars. But I think that isn't that interesting and doesn't have these nice short-hand learnings "you just have to truly apply".
Don't get me wrong, I am for fewer communication cycles and more trust in quality engineering. But we also have to acknowledge that there is a lot of expertise in other domains that is relevant for the success or risk that we software engineers just don't have and this require input on.
lcapaldo | an hour ago
I think you may be misunderstanding what the article is describing. They’re not reviewing the code as you go up/out. Different, often higher level, artifacts are being reviewed rather than the code (eg design documents, business cases, proposals for new products etc), just like the QA layer isn’t reviewing the code either.
klingtnet | 4 hours ago
If I had to name something that improved the quality of code being shipped, then it's code reviews. I get the point that reviews slow you down, but don't see it as a problem and instead embrace the pause or do something else instead. Not everything has to be rushed, and you don't need to be a 100% effective all day long, except you want to burn out quickly.
timthelion | 4 hours ago
I have the opposite experience. You might catch the typos in review, but to get to a great architecture you have to change fast and be very dynamic.
spicycode | 2 hours ago
I feel like that's often a statement on the quality of the code review more than anything else. Being able to describe what you're doing and answer questions about the direction/code really helps everyone in the process.
alper | 6 hours ago
Reviewing the code? At this point we can barely review what the code is doing, let alone how it works.
Diana | 4 hours ago
Here is the fun stuff that I think is being missed in this post.
If you do all the things to reduce the number of review layers, the (alleged) speedup from AI help... does not register anymore!
Because the speedup from reducing the layers is dominating that reduction. Which means you do not need the AI at all (It would even introduce more layers, because the human writing the code by himself is already a layer that you have to emulate now).
munksgaard | 8 hours ago
I struggle with this in our team right now. Some part of me is tempted to forego reviews entirely, but that also feels like a very nuclear option.
edsu | 7 hours ago
The struggle is real! I think the author's description of the "AI Developer’s Descent Into Madness" is spot on here. But as they also say, there are a lot of people on this Karpathy path, at the moment, so it feels more like a natural progression than madness.
Some high profile mistakes will probably help alert the industry that this is the wrong path to be on. But I worry that instead we will get used to the feeling of not actually understanding the complexity of software and how to reduce it. Arguably this is a path that the software industry has been on for some time.
I am personally hopeful when I hear people talking about using genAI coding tools to increase engineering rigor , rather than simply focusing on productivity.
conartist6 | 2 hours ago
Nuclear? Yes. But someone like me who isn't using AI can no longer afford to tie sandbags around my ankles and hope for the best. When I keep iterating on code, I keep seeing things I wrote and fixing and improving them even without a formal review step.
Similarly testing every change in isolation is probably absurdly slow. The better way to do it is to test 10-20 changes together.
But part of why I think it works well for me is that I'm writing all the code
peter-leonov | 6 hours ago
Pretty much summarizes the economy of trust point of view, thanks for sharing 🍕