I’ve used it most extensively doing Ruby on Rails greenfield apps, and also some JS front ends, some Python mid sized apps, and some Rust and Nix utilities. You’re absolutely right about it struggling with code base scale, I had to rework the design process around this. Essentially, design documentation telling the story, workflow documentation describing in detail every possible functionality, and an iteration schedule. So the why, what, and how formalized and in detail, in that order. It can generate the bulk of those documents given high level explanations, but require humans to edit them before making them the ‘golden’ references. Test driven development is beyond critical, telling it everywhere to use it extensively with writing failing tests first seems to work best.
So to actually have it do a thing I load those documents into context, give it a set unit of work from the iteration schedule, and work on something else.
It does go down some seriously wrong paths sometimes, like writing hacky work arounds if it incorrectly diagnosing some obscure problem. I’ve had a few near misses where it tried to sneak in stuff that would bury future work in technical debt. Most problematic is it’s just subtle enough that a junior dev might miss it; they’d probably get sent down a rabbit hole with several layers of spaghetti obscuring the problem.
That sounds like you’re still doing a lot of work. Is that net new work you wouldn’t have done before (like would you have needed to write those docs before as well)? Writing code never feels like the complicated or time expensive part to me. Figuring out what I want to do is, and I need to do that with either approach, and then thinking through how I’d like to organize things is another time sink, and perhaps that can be replaced/augmented by ai, but organizing things well requires long term thinking and is very hard to explain
I’ve used it most extensively doing Ruby on Rails greenfield apps, and also some JS front ends, some Python mid sized apps, and some Rust and Nix utilities. You’re absolutely right about it struggling with code base scale, I had to rework the design process around this. Essentially, design documentation telling the story, workflow documentation describing in detail every possible functionality, and an iteration schedule. So the why, what, and how formalized and in detail, in that order. It can generate the bulk of those documents given high level explanations, but require humans to edit them before making them the ‘golden’ references. Test driven development is beyond critical, telling it everywhere to use it extensively with writing failing tests first seems to work best.
So to actually have it do a thing I load those documents into context, give it a set unit of work from the iteration schedule, and work on something else.
It does go down some seriously wrong paths sometimes, like writing hacky work arounds if it incorrectly diagnosing some obscure problem. I’ve had a few near misses where it tried to sneak in stuff that would bury future work in technical debt. Most problematic is it’s just subtle enough that a junior dev might miss it; they’d probably get sent down a rabbit hole with several layers of spaghetti obscuring the problem.
That sounds like you’re still doing a lot of work. Is that net new work you wouldn’t have done before (like would you have needed to write those docs before as well)? Writing code never feels like the complicated or time expensive part to me. Figuring out what I want to do is, and I need to do that with either approach, and then thinking through how I’d like to organize things is another time sink, and perhaps that can be replaced/augmented by ai, but organizing things well requires long term thinking and is very hard to explain