I have been thinking about this for a while. I'm not a Luddite, I can see the amazing value that modern AI brings, and how useful it can be.
At the same time I can't help but wonder that this is almost too much for us to cope, the LLMs are _too_ easy to use. I think we're very good at pretending there isn't a problem here but there is definitely going to be some cognitive decline happening.
We want to pretend like reading over the output of an LLM is nearly the same as us making those changes ourselves but I think there's a lot of value in those boring and manual steps. And most people (including me) will be too lazy to not eventually offload most things to the LLMs.
In a way this is similar to how watching a lecture tends to give one an "impression" of understanding that doesn't correlate with reality. We need hands-on work and some course work to get our monkey brains engaged.
I can only imagine what effects this is going to have in education.
The tendency to over-engineer and over-build is one of the main challenges I'm seeing. When it comes for free, why not build a class hierarchy of 8 classes each with a 200 line unit test? Something I might have previously thought really hard to minimalise down to 30 lines is now bloated out to hundreds or even thousands.
Previously, pretty much everyone all agreed on YAGNI and KISS because code was so expensive to create and more expensive to maintain. Now the code generation is "free" - but at a design level, YAGNI and KISS do still matter. But they are enforced now only by sheer will power. You have to actively restrain the code from being created if you don't want it. Perfectly good code - I can't even fault it. I just know, if it doesn't need to exist yet, it's a mistake to create it earlier than needed.
That quotation is in fact from "Wind, Sand, and Stars", the memoir of the same author.
On the same theme:
> the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience
- Einstein
> I would have written a shorter letter, but did not have the time
- Pascal
> Simplicity is a great virtue but requires hard work to achieve and education to appreciate, and, to make matters worse, complexity sells better.
I think this sorta cuts both ways too. Sometimes AI code has shown me that no, I in fact don't need that big class hierarchy and speculative plumbing that I thought I might need to get the job done.
Keeping the volume of code down is definitely a challenge though.
That's always been my concern - there's just so much garbage code out there and very few signals about which of it is well written and which is not ... so LLMs will just regurgitate the good enough code that it's been trained on.
Now would be the ideal time for everybody to give it negative feedback when bad code is returned but I'm afraid not enough people care and we'll just settle with what works just like we always have.
At the end of the day your work is your work. If you tell Claude or whatever to do a thing it will do that thing however you tell it to do the thing.
You can choose to ignore the fine details and just tell it what to build, or you can spend more time and effort and tell it how to do things, you can ask it to create a plan, review the plan and ask for changes etc, you can review the implementation and ask for changes there. You can simply tell it you don't like how a feature is implemented and ask it for suggestions on how to improve it, include what you don't like about it so it has direction.
It's all about the effort you put in. LLMs are able to write clean beautiful and efficient code, they just can't necessarily do exactly what you want on the first try without any direction. That's why you should figure out what you want, ask for what you want and make sure it's actually building what you want.
Maybe things will evolve into projects having a well-defined, hard boundary between agents generated code and human generated code. Where agents generated code is strictly separated from human engineered code and and can call only a strictly defined set of APIs designed by humans.
Think of platforms like Shopify. Shopify allows huge number of external users to write code customizing their web shops. But Shopify engineers do not need to review and understand the users' code, because there is a boundary which limits what the external code can do. The external code can be A Big Ball of Mud without making the whole platform a A Big Ball of Mud.
We are leaning into a kind of federated model. The core applications and vibe-coded satellites. The core apps have stricter requirements and carefully considered API guardrails. Go ham on the AI apps. If you break something, the scope of destruction is much more limited.
That said, the core apps are still full of AI code. I don't see a reality where hand-written code can or will be curated and sequestered. It's unenforceable, too slow, and too expensive.
I think so too. And it is how codebases are managed implicitly anyhow. You don’t let anyone PR against core functionality, you keep that API under lock and key.
But UI on top of a good API should be unable to cause problems; the API must prevent invalid state etc. So letting LLM’s write UI code for CRUD-apps (which is most of them…) seems fine to me.
Well. One of the things that's critical in all that analogy is that the BigBallOfMud that one user creates on Shopify cannot impact the other users - that's why Shopify can keep the platform from becoming ABigBallofMud.
In order to be able to enforce this hard boundary, it means that one "agent generated code" in one place, not only "can call a strictly defined set of APIs" - but, more importantly, it implements an isolated piece of functionality. Think something like: I let the AI write the dashboards, so the dashboards may be slow, crappy, incorrect, whatever - but it doesn't impact the rest of the product. However, I can't let AI write "metrics collection" because that not only impacts dashboard, but potentially impacts core product efficiency too (memory consumption, cpu, IO, etc.)
>Maybe things will evolve into projects having a well-defined, hard boundary between agents generated code and human generated code.
The only way things will evolve is towards maximizing AI generated code and minimizing human involvement because that is the most efficient mode for getting product to market. Any trend to the contrary is temporary, paperclip maximizer will maximize. Any code that humans touch directly will be considered implicitly too risky to run in any production environment.
We'll just have to hope things don't break too much and when they do, we'll just have to accept it because no human will be able to (or allowed to, or even trusted to) fix it.
Great post. I particularly like the 4 column pr view idea. I’ve built something similar to review speckit specs. The scaffolding to ensure that code actually works makes sense too.
There’s one thing that’s bothering me though — all of these have us consuming what LLMs produce. I learned and internalized how things worked by building systems. At least within the constraints of the abstractions I was using. This makes me think of TikTok and shorts. Everybody is just watching. Maybe that’s not charitable though. Maybe the abstraction has now changed from an “os/gui/event loop” to “llm” and the rest is just accidental complexity.
>> A more serious irony is that the automatic control system has been put in place because it can do the job better than the operator, yet the operator is being asked to monitor that it is working effectively.
Automatic control is good for standard case, and the expected errors. For weird cases, for now humans are better to detect them and press the big red button.
Jumping into article, I assumed that the author was going to describe how he enjoys, by drawing it out as far as he can,
the process of surrendering to the machine god..
(Not the machina diabol, as it sadly turned out to be, though some BDSM zealots might disagree.. the things that bureaucracy does to us are mild in comparison? )
But.. we do monitor whether our prayers are effective .. right? Is that a smitable offense?
After that disappointment, I too was "curious": was the author going to talk about "curiosity", that magic virtue that we hackers believe, lifts design into the divine (but more often than not kills the pussy)
The better an automatic control is, the fewer mistakes it makes, the fewer times the human needs to react, the more lax and less vigilant the human becomes -- and less skilled when they need to step in. Source: https://imgur.com/a/Vj2TaDz
Except experience shows they aren't. Experience shows that when humans are just pressing "ok" 99% of the time they'll press it 100% of the time and completely miss the other cases.
i sometimes wonder how all carpenters or music producers in the world would react if tomorrow a magic box arrives where they type-in/dictate what they want and voila... out comes a table/composition.. Will the worlds music producers and carpenters just "accept" this new magic genie? I would hope they wont.
Yet here we are as software engineers...ready to surrender our craft to mere "prompts" with reckless abandon.
I see a bit of a split in people in creative roles of all kinds (and this is something I had noted before LLMs made the contrast more stark). some people get into it because they like the process of making the thing that they are making (in whole or in part). Others get into it because they have a thing that they want to exist in the world but no-one else has made it (I would also split this into people who just want the thing, or people who want to make something that crosses many domains and so want custom things that will fit into the whole). The former group tend to react with revulsion to automation, because it removes the part they like (though it may depend on what part is automated: most people will find some part of their work necessary but tedious). The latter are usually much more willing to embrace it, because it's the end result that they care about moreso than the process.
Have you seen how people use Fruity Loops nowadays? We’re already halfway there if you compare that to using an AKAI S9000 or MPC2000. And then go back to actually playing an instrument. It’s a gradient and we’re already pretty far along on it :)
Its just as much as an ego trip...as the Principal Engineers that roam around here, after they survived three layoffs and outputted front end code for six years straight at a FAANG...
That's new to me, I didn't know she coined the term. She's the first true "software engineer" then, I suppose. I remember her from that historic photo where she's standing next to a pile of code printed out, a stack talled than her.
Not to be confused with Margaret Hamilton the actress who played the Wicked Witch of the West in the film The Wizard of Oz.
It turns out that most people are fine with most of their things being mass-produced cheap stuff from IKEA, Amazon, Temu, etc. - if that means that the cost is significantly lower. The functional benefit of having a door is huge, while the aesthetic benefit of having a beautiful, hand-crafted door is much less important to most people.
Code is not like craftsmanship anyway. The end-user doesn't see any difference whether I use microservices or hexagonal design or react or htmx or how I structure the code or which language I use. All they see is the design and the stability. Since the average developer was terrible at design anyway, the average design output from LLMs is actually an improvement in most cases, even if it always has that generic AI generated look. Stability... was not good before AI either. We have become accustomed to SAP taking 10 seconds to load each component on the page, or Atlassian spewing error messages in your face when you click submit on a ticket. It seems that the aspects that are most valued by our end users are QA and UX, not the craftsmanship of our code.
Suno probably puts out better music than most folks who take up music. I mean statistically it has to, right? So once we go with "suno doesn't make music" then we also have to progress to "99 percent of human musicians don't make music", and all us amateur hacks will start objecting lol
I mean more or less to an extent if you expand by what you mean by "out comes". With Amazon, a carpenter can type in any furniture he wants and have it arrive in a matter of hours. Almost equally as magically as well as disruptive. People still use carpenters to make tables, but for the majority, the ease of same-day delivery and cheaper prices are worth it for a sacrifice in personalization and character.
> Yet here we are as software engineers...ready to surrender our craft to mere "prompts" with reckless abandon.
It's worth remembering that, contrary to what HN acts like, software engineers are not uniformly accepting of LLMs. They are still quite controversial in the industry. Now management, on the other hand, is completely sold on the supposed benefits of LLMs. So many engineers feel forced to use them, but it's not enthusiasm on their part which drives them to that.
My experience has been the opposite, the HN crowd seems more sketical of vibe-coding and not reading a single line of code.
The twitter(x.com) and linkedin crowd has drank the cool-aid and are now swimming in it; Wild posts from FAANG engineers and managers saying they or their engineers dont read any of the code the LLM generates.
Music is already mostly a solved problem unfortunately. Try giving opus a few midi files of your favorite song. It'll write something new in that style with a competence humans would need several years of experience for.
"Yup, looks good to me!" said the human reviewing thousands of lines of generated code everyday.
This has become common now, and I've seen it ruin large projects until nobody understands the codebase full of verbose nonsensical code and comments. It becomes impossible to further develop the project without AI. Contributors not using AI for any reason are all leaving because no one enjoys working with mindless robots (and humans) as colleagues. It's demeaning and devaluing their labor and humanity. They're not getting "left behind", they are leaving the madness of crowds behind.
> ..the possibility of runaway dynamics: once a critical threshold is crossed, small increases in adoption can trigger rapid population-level shifts toward *persistent dependence, with abrupt losses in cognitive competence*.
We can all see it happening. As this technology permeates society, the challenge to maintain control, coherence, and quality will become greater and ever more serious - for example, AI in healthcare, military, management and governance.
Even if the human's judgement is correct in every LGTM, there's a bias towards merging. Each merge that adds a feature but degrades understandability 1% will be merged. It won't be held back until it improves it by 5% instead. After 100 merges you have degraded understandability 100% even though nobody made an error.
You’re completely right that nobody is stopping a PR that just ever so slightly decreases understandability of something. It may not even be obvious to the reviewer that it’s slightly less understandable overall… Doubly so if they’re just skimming due to the volume, and tripley so if they’re having agents do the most of the review, which lets the human reviewer’s guard down.
But before LLMs, it didn’t matter because on a good team you’d have a balance where maybe some PRs added complexity and some simplified it. Sometimes people would include a small related cleanup or refactor in a PR for code around what they were doing if they noticed complexity. That has become far less common, I think.
I know it’s not your point, but a 1% decrease 100 times is 0.99^100 = 0.366 which is a 63.4% decrease in understandability. Just throwing that out there
I've had someone approve my PRs within a minute of me opening them, and likewise expect immediate approvals on his own PRs. When asked questions about parts of the code sometimes the answer would be along the lines of "Claude did it this way", or verbose AI-generated responses to the question.
I hope there's still value to be found in a bunch of developers standing around a whiteboard and talking, and I think this scenario is where their ability and understanding really shows these days.
If we don't value that though--if we don't value understanding the system and just thinking through problems as a group--then let's stop pretending and just not do code reviews. Probably nobody cares and just types "LGTM" after having their own LLM do a spot check to cover their ass.
It's really easy to push developers to stop caring. If management says "go fast fast fast, use those LLMs!", then developers can't keep up, get frustrated, stop understanding the system, and at that point they're just there to collect a paycheck before society collapses. Or, if management is making dumb decisions all the time, same thing.
Hopefully as implementing ideas takes less and less time and resources, smart people who actually care will start building things with other smart people who care.
To be fair, models are more accurate than how they were earlier this year. In march I was still frustrated and was writing my own event loops because the model simply wasn't built for it. By January I was still hand checking statistical outputs to make sure it wasn't making any errors, and it was. After august it was pretty much over. There were no stats errors, code generated was shit but it could do what is asked. Instruction following is still a problem, and I sometimes catch silly things in the scrolling diffs, but it has been much easier since then.
The most difficult part of LLMs and agents seems to be - surprisingly - humans.
Specifically, humans and their dysfunctions. Their fragility. Their ego tied to stuff.
This is an excellent post/reflection on the topic of solo LLM use.
The only gap I see is that it assumes that one works on something alone and not in a team. Though, that might be a different essay in itself.
Edit: Turns out that footnote 3 actually mentions that already. Nice. Good post
On the ego-effect, I sometimes feel like cataloguing some PR-review quirks that are mostly caused by humans being humans:
1. Smaller PRs (that can be easily read) tend to receive more comments and nitpicks than a large change since it's easier for us to have all the PR context in our heads at the same time.
2. Most people are ego-tied to their code even if they pretend that they are
impartial
3. Social dynamics around merging and reviewing code between senior/junior and older/newer employees
Cognitive surrender is intellectually convenient. As humans, its instinctive for us to chose convenience: path of least resistance. The tyranny of convenience is what drives most economic activity today, especially the gig economy[1]. Unless there are structural frameworks put in place to resist convenience (exams in schools), nature will always entice to chose the path of least resistance
> Once you have used a washing machine, laundering clothes by hand seems irrational, even if it might be cheaper.
Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
> After you have experienced streaming television, waiting to see a show at a prescribed hour seems silly, even a little undignified.
Undignified? What? Completely incoherent word slop. Good to see LLMs haven't completely taken that domain for themselves, I guess - a little hand-crafted slop now and then is a treat, I guess? Anyways, they're wrong. People still tune in to watch scheduled live events all the time.
> To resist convenience — not to own a cellphone, not to use Google
Neither of these things are a matter of convenience. A cellphone is at this point literally required to participate in a modern economy. Google is a tool that provides you with globally-retrieved information that you otherwise literally could not find on your own or without another similar tool.
I don't even disagree with the premise that convenience is a driving factor of our lives, but the justifications and conclusions drawn by this article are beyond stupid. Here's my view of it: the desire for convenience is what pushes us onwards as a species. Hunting for food is inconvenient, so we invented agriculture. Going down to the river or well for water is inconvenient, so we invented indoor plumbing. Repeat ad infinitum until we have modern civilization. There is one part the pessimistic view overlooks; that to achieve maximum convenience long-term requires more effort in the short-term. Developing the skill of programming is inconvenient in the short-term but convenient in the long-term because of the possibilities this opens up to you. This remains true even in the current LLM era, and it will remain true for as long as we are not replaced wholesale by LLMs.
>Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
I'll also note that it is less resource intensive overall. Handwashing dishes and clothes will use more water and more energy to heat that water compared to the machines that do the same job (people often assume the opposite).
What an amazing article! It's really hard to find the balance between "ship faster, fix later because AI will help" and "efficient thoughtful engineering" these days. The world is kind of pushing everybody to work faster because "with AI you're more efficient" and it's also pushing to surrender and just accept it from time to time. Maybe that's occasional thoughtful surrender mentioned in the article, or it's just lack of energy to resist to it at that point.
The article is inspiring though, I have more motivation to work on my thinking process and my engineering setup (and vibe limit haha) to be more thoughtful, thank you.
1. LLM usage, in particular unsupervised, tends to build up "cognitive debt"
2. It tries to characterize the concept, including to propose a score for it
3. Notes that the level at which one can let the agents loose (and allow some CD) differs quite a lot between core business domains and more peripheral ones
Making a quick prototype is probably the best way to find out if an idea is worth keeping. Vibe-coding it and then throwing it away seems like a good way to go?
Yeah, definitely. Also at times I've vibecoded successive iterations with better designs after seeing where the first ones collapsed -- the agents do a good job with a template and need much less prompting for the successive versions.
I am often thinking recently of the littel loop diagrams from the "The Capability Trap" paper. We need to keep a loop reinvesting into our own Human capability in anything we're doing.
Maybe it's because I'm not working in the workforce (grad student), but I find my best workflow is to fight and argue with my agents throughout the entire process. Often times, since the agent is such a sycophant, I end up arguing with myself...I actually find this useful because it gets me to think about the other side of a problem and poke holes in it. This is not fast, but it is effective.
I simply don't care. As a student I thought that the mythical production code is perfect and error-free, but in reality 99% of systems are a hot potato game sprinkled with politics where you don't want the grenade to explode in your hands.
I mean, if the management doesn't see value in decent engineering, then who am I to butt myself into the conversation?
This is copium that is gone with the wind when the models perform up to the level of standard that the user requires. I'll give the author 6 months before they feel like its okay to just send it.
Karpathy had the same trajectory. Primeagean had the same trajectory. I think in the end everyone except for Jonathan Blow will have the same trajectory
Writing is hard, but what I was trying to say is that it's okay to just send it as long as you're doing it consciously and intentionally (and I do that frequently enough); but if it's not an explicit choice that can be problematic.
I've been experimenting with vibe-coding some games. One of the games started performing very badly in a particular menu, and I began suspecting it's because of a shader that may be redundantly computed way too much on too many large surfaces. I began writing my suspicion to Claude, then stopped and decided to just describe the problem and see if it would come to the same conclusion as me. A few minutes later, it had profiled the game, found what contributed most to the poor performance, fixed the rendering and made the game performant again. I didn't have to inspect the code to do any of that. The current models (paired with coding harnesses) love to test and validate their results by running the games, taking screenshots, singling out individual parts, including logs to reproduce errors or other issues, and they love to create tests. The code is good enough in the same sense that the code I used to write was good enough if it did what we wanted. My non-technical bosses didn't have to inspect my code for that.
In my experience, it's usually good for finding out the bugs, but I take the fixes it provides with a grain of salt. There were numerous times where the fix could have lead to some more serious bugs because of some business logic special case scenarios that it didn't understand.
Yeah, I'm seeing this at work too. It's capable of adjusting behavior to meet some narrow success criteria, but it can easily introduce bigger problems in the process in large coupled code bases, which it doesn't catch on its own. Automated tests help a bit, but I find I still have to steer it very strongly at work.
It sounds like you understood what was happening here and used a tool effectively, nice.
That's very different from what a lot of vibe-coders do, like wholesale cloning and relicensing projects. This is happening in the GIS space, and these vibe-clone projects slightly hilariously inherit all the technical debt they didn't understand.
Nothing wrong with using LLMs, but I agree with Andrej Karpathy: "you can outsource your thinking, but not your understanding".
I think that quote is very apt. I experimented with making realistic video game grass (inspired by Ghost of Tsushima), and let it take the reins. But its solution became very contrived and hard to steer in the direction I wanted without me learning the specifics of how grass rendering should work. Still, it seems that I can limit my understanding to the higher levels of abstractions in most cases now. For example, I made it clump grass together according to a voronoi diagram, yet I have no idea how the voronoi algorithm works.
Who cares if it's good enough? Send the product, if people use it, it's good enough. If not, let users debut it and give "feedback", problem solved. You then advocate to these users, make them feel important, make them your cash cows, rinse and repeat.
Every team I have been on have been more successful when they gave up on the product ideals and just ensured that the customers were hooked and satisfied.
That mindset is why most software was already slop before AI. Users know when your team has turned into salespeople. And now that AI could enable you to do better in the same timeframe the response with this mindset is of course to care even less.
> Who cares if it's good enough? Send the product, if people use it, it's good enough. If not, let users debut it and give "feedback", problem solved.
I care, because doing a sloppy job and seeing if people will tolerate it is not acceptable engineering practice. It's not engineering at all. More pragmatically, if you do the thing you are suggesting, you won't have users for much longer. They will abandon you and go to someone who actually makes good products.
I don't believe this in practice. I am at a FAANG-like firm, and we are just maxxing the engagement and people use the product more and more the more we serve stuff to keep them hooked using their data and behaviour.
The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
I think there is a small subset of technical nerd/idealists who think that quality is all that matters but they are in their own little sandbox while 97% of users are in another one.
> I think there is a small subset of technical nerd/idealists who think that quality is all that matters
This is just a derogatory comment.
It is exactly this kind of arrogance that is killing open source. In my area, GIS, there are foundational projects that are necessarily intricate. For example it might be necessary to make a four-dimensonal transform of a location, that takes into account point movement due to the Glacial Isostatic Adjustment. There are hundreds of other complex examples I could come up with.
The libraries are intricate and quality does matter.
Unfortunately the lynchpin maintainer has just gone on indefinite strike, partly because open-source vibe-coders are wholesale copying and relicensing some of these libraries in their quest to ... do what?
I actually I don't know what they want. Take all the oxygen from the room as far as I can tell.
> The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
Sorry for going into a tangent here, but I'm actually unsure if I hate the structures and economic incentives that makes this possible, or if I hate you specifically for defending something so rotten
Next time I rage at some broken UI in some damn service I'm forced to use because that's where people are, I'll remember this comment
> The quality of what we provide seems to matter 0
Good for you! There are lots of low skill development jobs before AI where low knowledge coders cranked out WordPress sites for marketing, advertising, and small businesses. Sounds like you got in on the ground floor!
Meanwhile companies like Amazon, one of the As in actual FAANG, do have demands on their software quality. To the extent they essentially said "knock off the tokenmaxxing productivity theater" in a recent, publicized internal memo.
Horses for courses. Sometimes you need an thoroughbred and sometimes a pack mule.
And there's nothing wrong with being a pack mule! And nobody expects a pack mule to understand the work of a thoroughbred! Nobody expects too much from a jackass after all.
I worked for one of the largest device manufacturers for a certain kind of sensor.
The requirements/standards for this sensor was made by this company 20 years ago. All the tests and requirements were made by this same company in a way that would minimize the costs for us to keep making new revision we could sell, and make it the hardest for any competitor to enter the market. Another competitor started with a better product abroad, so we copied it, but it didn't pass the tests, so then they installed an owner of the certification house that was a cousin of one of the VPs. No problems passing tests after that.
You think this is all controlled and nice and standardized but it's all just a market from software to hardware.
While quite believable, there is a diversity of industries and standard adherence / enforcement no?
Markets also range from under regulated to over regulated, and are usually in a transition from one end to another (while contending with other forces).
Why would Prime be in that list. He had an entire arc where he tried to make it work and then went back to the drawing board.
I am not saying he writes all code by hand, of course. Just that he is furthest in figuring out some sort of mid point between full comprehension and cognitive surrender.
I have missed the more recent content, so perhaps I have missed something, but this was my read last I watched.
I'm also not sure why anyone would attach importance to his opinions as a software engineer, he's exceptional in generating entertaining content but as an engineer he does not seem above average. Definitely not even close to the category of Jonathan Blow or Andrej Karparthy
At the same time I can't help but wonder that this is almost too much for us to cope, the LLMs are _too_ easy to use. I think we're very good at pretending there isn't a problem here but there is definitely going to be some cognitive decline happening.
We want to pretend like reading over the output of an LLM is nearly the same as us making those changes ourselves but I think there's a lot of value in those boring and manual steps. And most people (including me) will be too lazy to not eventually offload most things to the LLMs.
In a way this is similar to how watching a lecture tends to give one an "impression" of understanding that doesn't correlate with reality. We need hands-on work and some course work to get our monkey brains engaged.
I can only imagine what effects this is going to have in education.
Previously, pretty much everyone all agreed on YAGNI and KISS because code was so expensive to create and more expensive to maintain. Now the code generation is "free" - but at a design level, YAGNI and KISS do still matter. But they are enforced now only by sheer will power. You have to actively restrain the code from being created if you don't want it. Perfectly good code - I can't even fault it. I just know, if it doesn't need to exist yet, it's a mistake to create it earlier than needed.
> La perfection est atteinte non pas lorsqu'il n'y a plus rien à ajouter, mais lorsqu'il n'y a plus rien à retirer.
> Perfection is attained not when there is nothing more to add, but when there is nothing more to remove.
On the same theme:
> the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience
- Einstein
> I would have written a shorter letter, but did not have the time
- Pascal
> Simplicity is a great virtue but requires hard work to achieve and education to appreciate, and, to make matters worse, complexity sells better.
- Dijkstra.
Keeping the volume of code down is definitely a challenge though.
Now would be the ideal time for everybody to give it negative feedback when bad code is returned but I'm afraid not enough people care and we'll just settle with what works just like we always have.
You can choose to ignore the fine details and just tell it what to build, or you can spend more time and effort and tell it how to do things, you can ask it to create a plan, review the plan and ask for changes etc, you can review the implementation and ask for changes there. You can simply tell it you don't like how a feature is implemented and ask it for suggestions on how to improve it, include what you don't like about it so it has direction.
It's all about the effort you put in. LLMs are able to write clean beautiful and efficient code, they just can't necessarily do exactly what you want on the first try without any direction. That's why you should figure out what you want, ask for what you want and make sure it's actually building what you want.
It'll take a particularly dumb and lazy AI to replace me.
Think of platforms like Shopify. Shopify allows huge number of external users to write code customizing their web shops. But Shopify engineers do not need to review and understand the users' code, because there is a boundary which limits what the external code can do. The external code can be A Big Ball of Mud without making the whole platform a A Big Ball of Mud.
That said, the core apps are still full of AI code. I don't see a reality where hand-written code can or will be curated and sequestered. It's unenforceable, too slow, and too expensive.
But UI on top of a good API should be unable to cause problems; the API must prevent invalid state etc. So letting LLM’s write UI code for CRUD-apps (which is most of them…) seems fine to me.
In order to be able to enforce this hard boundary, it means that one "agent generated code" in one place, not only "can call a strictly defined set of APIs" - but, more importantly, it implements an isolated piece of functionality. Think something like: I let the AI write the dashboards, so the dashboards may be slow, crappy, incorrect, whatever - but it doesn't impact the rest of the product. However, I can't let AI write "metrics collection" because that not only impacts dashboard, but potentially impacts core product efficiency too (memory consumption, cpu, IO, etc.)
The only way things will evolve is towards maximizing AI generated code and minimizing human involvement because that is the most efficient mode for getting product to market. Any trend to the contrary is temporary, paperclip maximizer will maximize. Any code that humans touch directly will be considered implicitly too risky to run in any production environment.
We'll just have to hope things don't break too much and when they do, we'll just have to accept it because no human will be able to (or allowed to, or even trusted to) fix it.
There’s one thing that’s bothering me though — all of these have us consuming what LLMs produce. I learned and internalized how things worked by building systems. At least within the constraints of the abstractions I was using. This makes me think of TikTok and shorts. Everybody is just watching. Maybe that’s not charitable though. Maybe the abstraction has now changed from an “os/gui/event loop” to “llm” and the rest is just accidental complexity.
Automatic control is good for standard case, and the expected errors. For weird cases, for now humans are better to detect them and press the big red button.
But.. we do monitor whether our prayers are effective .. right? Is that a smitable offense?
After that disappointment, I too was "curious": was the author going to talk about "curiosity", that magic virtue that we hackers believe, lifts design into the divine (but more often than not kills the pussy)
Its just as much as an ego trip...as the Principal Engineers that roam around here, after they survived three layoffs and outputted front end code for six years straight at a FAANG...
Prove me wrong...
But oh well, who was she, just the engineer who wrote the software for the Apollo 11. Probably she had no idea what she was talking about.
/s
Not to be confused with Margaret Hamilton the actress who played the Wicked Witch of the West in the film The Wizard of Oz.
Code is not like craftsmanship anyway. The end-user doesn't see any difference whether I use microservices or hexagonal design or react or htmx or how I structure the code or which language I use. All they see is the design and the stability. Since the average developer was terrible at design anyway, the average design output from LLMs is actually an improvement in most cases, even if it always has that generic AI generated look. Stability... was not good before AI either. We have become accustomed to SAP taking 10 seconds to load each component on the page, or Atlassian spewing error messages in your face when you click submit on a ticket. It seems that the aspects that are most valued by our end users are QA and UX, not the craftsmanship of our code.
And if you look a the dates, the rate at which they are doing it is actually increasing...
Sure but that doesnt require the person to be a carpenter though right? Anybody could do that.
It's worth remembering that, contrary to what HN acts like, software engineers are not uniformly accepting of LLMs. They are still quite controversial in the industry. Now management, on the other hand, is completely sold on the supposed benefits of LLMs. So many engineers feel forced to use them, but it's not enthusiasm on their part which drives them to that.
The twitter(x.com) and linkedin crowd has drank the cool-aid and are now swimming in it; Wild posts from FAANG engineers and managers saying they or their engineers dont read any of the code the LLM generates.
It' like saying compassion is a solved problem: just tell your problems to the LLM and have it gush out sympathy.
This has become common now, and I've seen it ruin large projects until nobody understands the codebase full of verbose nonsensical code and comments. It becomes impossible to further develop the project without AI. Contributors not using AI for any reason are all leaving because no one enjoys working with mindless robots (and humans) as colleagues. It's demeaning and devaluing their labor and humanity. They're not getting "left behind", they are leaving the madness of crowds behind.
> ..the possibility of runaway dynamics: once a critical threshold is crossed, small increases in adoption can trigger rapid population-level shifts toward *persistent dependence, with abrupt losses in cognitive competence*.
We can all see it happening. As this technology permeates society, the challenge to maintain control, coherence, and quality will become greater and ever more serious - for example, AI in healthcare, military, management and governance.
But before LLMs, it didn’t matter because on a good team you’d have a balance where maybe some PRs added complexity and some simplified it. Sometimes people would include a small related cleanup or refactor in a PR for code around what they were doing if they noticed complexity. That has become far less common, I think.
I know it’s not your point, but a 1% decrease 100 times is 0.99^100 = 0.366 which is a 63.4% decrease in understandability. Just throwing that out there
Enterprise software has always been crap.
If we don't value that though--if we don't value understanding the system and just thinking through problems as a group--then let's stop pretending and just not do code reviews. Probably nobody cares and just types "LGTM" after having their own LLM do a spot check to cover their ass.
It's really easy to push developers to stop caring. If management says "go fast fast fast, use those LLMs!", then developers can't keep up, get frustrated, stop understanding the system, and at that point they're just there to collect a paycheck before society collapses. Or, if management is making dumb decisions all the time, same thing.
Hopefully as implementing ideas takes less and less time and resources, smart people who actually care will start building things with other smart people who care.
Specifically, humans and their dysfunctions. Their fragility. Their ego tied to stuff.
This is an excellent post/reflection on the topic of solo LLM use. The only gap I see is that it assumes that one works on something alone and not in a team. Though, that might be a different essay in itself.
Edit: Turns out that footnote 3 actually mentions that already. Nice. Good post
1. Smaller PRs (that can be easily read) tend to receive more comments and nitpicks than a large change since it's easier for us to have all the PR context in our heads at the same time.
2. Most people are ego-tied to their code even if they pretend that they are impartial
3. Social dynamics around merging and reviewing code between senior/junior and older/newer employees
4. probably many more
[1] https://www.nytimes.com/2018/02/16/opinion/sunday/tyranny-co...
> Once you have used a washing machine, laundering clothes by hand seems irrational, even if it might be cheaper.
Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
> After you have experienced streaming television, waiting to see a show at a prescribed hour seems silly, even a little undignified.
Undignified? What? Completely incoherent word slop. Good to see LLMs haven't completely taken that domain for themselves, I guess - a little hand-crafted slop now and then is a treat, I guess? Anyways, they're wrong. People still tune in to watch scheduled live events all the time.
> To resist convenience — not to own a cellphone, not to use Google
Neither of these things are a matter of convenience. A cellphone is at this point literally required to participate in a modern economy. Google is a tool that provides you with globally-retrieved information that you otherwise literally could not find on your own or without another similar tool.
I don't even disagree with the premise that convenience is a driving factor of our lives, but the justifications and conclusions drawn by this article are beyond stupid. Here's my view of it: the desire for convenience is what pushes us onwards as a species. Hunting for food is inconvenient, so we invented agriculture. Going down to the river or well for water is inconvenient, so we invented indoor plumbing. Repeat ad infinitum until we have modern civilization. There is one part the pessimistic view overlooks; that to achieve maximum convenience long-term requires more effort in the short-term. Developing the skill of programming is inconvenient in the short-term but convenient in the long-term because of the possibilities this opens up to you. This remains true even in the current LLM era, and it will remain true for as long as we are not replaced wholesale by LLMs.
It takes a mix of willpower / routine / systems / laws to keep people in check, otherwise we predictably go back to the path of least resistance.
I'll also note that it is less resource intensive overall. Handwashing dishes and clothes will use more water and more energy to heat that water compared to the machines that do the same job (people often assume the opposite).
The article is inspiring though, I have more motivation to work on my thinking process and my engineering setup (and vibe limit haha) to be more thoughtful, thank you.
https://epiq.sh/blog/cost-of-cognitive-debt.html
1. LLM usage, in particular unsupervised, tends to build up "cognitive debt"
2. It tries to characterize the concept, including to propose a score for it
3. Notes that the level at which one can let the agents loose (and allow some CD) differs quite a lot between core business domains and more peripheral ones
I am losing more of what little focus I've had every day.
The economics of the whole affair unfortunately point squarely in the opposite direction.
I mean, if the management doesn't see value in decent engineering, then who am I to butt myself into the conversation?
Karpathy had the same trajectory. Primeagean had the same trajectory. I think in the end everyone except for Jonathan Blow will have the same trajectory
However it takes more mental energy and people don't like that
I guess you could ask the model. "Yes well done, you're a visionary software engineer".
That's very different from what a lot of vibe-coders do, like wholesale cloning and relicensing projects. This is happening in the GIS space, and these vibe-clone projects slightly hilariously inherit all the technical debt they didn't understand.
Nothing wrong with using LLMs, but I agree with Andrej Karpathy: "you can outsource your thinking, but not your understanding".
Every team I have been on have been more successful when they gave up on the product ideals and just ensured that the customers were hooked and satisfied.
I care, because doing a sloppy job and seeing if people will tolerate it is not acceptable engineering practice. It's not engineering at all. More pragmatically, if you do the thing you are suggesting, you won't have users for much longer. They will abandon you and go to someone who actually makes good products.
The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
I think there is a small subset of technical nerd/idealists who think that quality is all that matters but they are in their own little sandbox while 97% of users are in another one.
This is just a derogatory comment.
It is exactly this kind of arrogance that is killing open source. In my area, GIS, there are foundational projects that are necessarily intricate. For example it might be necessary to make a four-dimensonal transform of a location, that takes into account point movement due to the Glacial Isostatic Adjustment. There are hundreds of other complex examples I could come up with.
The libraries are intricate and quality does matter.
Unfortunately the lynchpin maintainer has just gone on indefinite strike, partly because open-source vibe-coders are wholesale copying and relicensing some of these libraries in their quest to ... do what?
I actually I don't know what they want. Take all the oxygen from the room as far as I can tell.
Sorry for going into a tangent here, but I'm actually unsure if I hate the structures and economic incentives that makes this possible, or if I hate you specifically for defending something so rotten
Next time I rage at some broken UI in some damn service I'm forced to use because that's where people are, I'll remember this comment
Also known as not-FAANG
> The quality of what we provide seems to matter 0
Good for you! There are lots of low skill development jobs before AI where low knowledge coders cranked out WordPress sites for marketing, advertising, and small businesses. Sounds like you got in on the ground floor!
Meanwhile companies like Amazon, one of the As in actual FAANG, do have demands on their software quality. To the extent they essentially said "knock off the tokenmaxxing productivity theater" in a recent, publicized internal memo.
Horses for courses. Sometimes you need an thoroughbred and sometimes a pack mule.
And there's nothing wrong with being a pack mule! And nobody expects a pack mule to understand the work of a thoroughbred! Nobody expects too much from a jackass after all.
The lawyers and judges who trusted an ML system to be up to snuff are currently pounding sand.
The requirements/standards for this sensor was made by this company 20 years ago. All the tests and requirements were made by this same company in a way that would minimize the costs for us to keep making new revision we could sell, and make it the hardest for any competitor to enter the market. Another competitor started with a better product abroad, so we copied it, but it didn't pass the tests, so then they installed an owner of the certification house that was a cousin of one of the VPs. No problems passing tests after that.
You think this is all controlled and nice and standardized but it's all just a market from software to hardware.
Markets also range from under regulated to over regulated, and are usually in a transition from one end to another (while contending with other forces).
I am not saying he writes all code by hand, of course. Just that he is furthest in figuring out some sort of mid point between full comprehension and cognitive surrender.
I have missed the more recent content, so perhaps I have missed something, but this was my read last I watched.