Better to revert than lose a community that is invested in your ecosystem that starts looking elsewhere. I congratulate the team in acknowledging and doing the right thing. As a developer myself, I know it is a painful thing to do but important to listen to the community.
I think maintainers here listened to their gut feeling and trusted their experience.
On top of that I think this is actually great FLOSS news, knowing how many commercial products have sacrificed proper professional programming on the altar of shareholder expectations.
As an engineer, this revert post is such a positive signal of appropriate stewardship. Devs know how painful it is to build and refine something only for it to be flushed.
Congrats on the team for putting stability ahead of features for such an important codebase.
Agreed. People using Postgres aren’t there for the next feature release. We want to see development continue but not at the expense of stability. This is a very mature team and I’m glad they shared this.
> I have been on the receiving end of some of that
Multiple features?
> Maybe we just didn’t do a good enough job. Maybe the code was just not good enough. We’ll learn and improve. It’s probably some of that, and I’m mentioning it here to not give the impression that it was only the other factors that I’m going to mention.
> It could also be a coincidence. Maybe we had been on such a good roll that we attempted to tackle several features that turned out to be too hard, or at least too hard to finish in time.
All of the above seem clearly the case--not that it could be realized at the time of cutoff and not getting it cleaned up in the window.
> we, meaning all the developers affected, have had enough time to fully reflect on all this.
Title should be: The reverts will continue until quality improves. Humour is good for the clicks but also a means of deflection. I sort of feel bad saying without putting in effort contributing myself but having it merged would have been worse.
I’ve experienced this myself. I’ve had to downgrade my opinion of my own code. LLMs are great at pointing out the edge cases that you might have taken years to find. Honestly feels quite dispiriting but we move on.
Previously my main concern was with LLM's writing worse code. Now my concern is the ability to write better code than I was ever able to write before -- considering every possible potential bug and race condition and error state and what have you -- but at what cost? How much more time do you have to spend researching, designing, making decisions around every possible edge case and state?
Before, you couldn't even attempt to balance quality with speed, because you couldn't see the quality in advance. You had to wait for bug reports to come in. Now, you're confronting quality up-front in a much deeper way. Which allows for much more reliable software, but then you have to decide how much effort is worth it and how much isn't?
This keeps bringing me back to John Backus’s Turing award lecture/paper, “Can Programming be Liberated from the Von Neumann Style?”
In it he pointed out a huge deficiency of most programming languages that still haunts us: they’re wickedly difficult to reason about. So proving important properties about programs is all but infeasible.
So we try to handle it empirically with unit tests, but that has its limitations.
LLMs can kinda sorta do it analytically, but that isn’t completely satisfying, either. The results are probabilistic, and they are pretty bad at inferring intent which can be a source of both type I and type II errors. But still, it would be silly to throw out a layer of Swiss cheese for having holes.
But I sure do wish we could have a plausible version of Backus’s original vision of a language that lets us formally declare (a useful subset of) the specification and then statically verify that the program conforms to it.
The blog post mentions rising standards of quality enabled by AI-assisted code review, and that reverted code predates the era of AI. Even very competently managed codebases are not above mistakes, especially when held to as high of a standard as this.
I’ve noticed the GitHub copilot reviews have gone from mostly nitpicks to very complicated issues that are hard to even understand or validate. Some of these are incorrect but some of them are actual findings of issues that simply would have gone unaddressed previously.
So a lot of this increase in bug reports lately is simply that we are more able to find issues which would previously have been shipped and probably wouldn’t have even been noticed by users for a long time. The minimum bar for software quality is going up, there's no excuse to ship software which contains bugs that could be detected by an LLM.
GP is referring to the submission, in case that wasn't obvious. It's opining as to possible reasons, given the author is one of the people who had a feature reverted.
> Did someone not understand what went in?
Unlikely, from the context.
> Might someone have offloaded their thinking to some external tool?
No, according to the author, since the features predate most current LLM assisted coding.
> Did people stop caring?
I'm not sure what this means. Presumably things were reverted because people cared enough to make sure it was good quality before being released.
> Did someone not plan or design a feature in full?
An admitted possibility
> Were the timelines unrealistic?
Depending on what you mean, that may be correct according to the article, but I suspect not in the way you mean.
> Maybe with so many reverts timelines should be revised?
It sounds like there was a very large increase in load in some of the people that would commonly review during this time period, and that may or may not continue to happen to a greater or lesser degree in the future, it depends on how LLMs advance in finding bugs and if there are other large waved of bugs reported.
Take any codebase you built in the pre-AI era and run it through AI review. I guarantee, no matter how smart you are or how diligently you write tests, you will be absolutely gobsmacked by the findings. Turns out we haven't been writing perfect software all this time and we have some catching up to do.
Seems like this is more about the introduction of LLM code reviews that has found edge cases that would have otherwise been missed and fixed as they appeared over years in the future. The issue is that the volume of issues has been beyond what people could do in time.
Maybe for many projects it's time to do a feature freeze. LLM provide a great opportunity to improve testing coverage, run automated code reviews and so on, so basically it's time to improve quality, while developers can adapt to this new LLM world and figure out the proper ways to integrate LLM to the development cycle.
Also adversaries are using abliterated LLMs right now to search for vulnerabilities, so it's important to stay ahead.
It's a play on "the beatings will continue until morale improves". You obviously can't just beat morale into people, and, knowing that, the audience should understand improvements to a bad situation will not be forthcoming. It's often used for dramatic effect as dark/silly humor when something bad happens.
A bunch of features that made it to beta turned out not to be in a state that could be made satisfactory for release time and so was reverted.
That seems to me to be the right thing to do. Unfortunate, but the right thing to do.
“[something] will continue until morale improves” is usually a sacastic dig at tone deaf and incompetent leadership (e.g, “firing will continue until morale improves”)
Not reverting would have consequences. Reverting hurts morale. It's a bad situation either way. Consider the hypothetical statement "the reverts will continue until code quality improves" while also keeping in mind that the application of popular satirical phrases can be a bit loose at times.
Yes, not only is there always improvements to existing functionality possible due to changes in improvements in general software engineering and mathematical methods beyond it or supporting new hardware features.
Generally, software engineering like other industries is evolving and there is need for different facilities. Consider graph and vector databases featuring, as well as json columns as examples of things you that are relatively recently in common use.
Yes, postgres has all sorts of features that hadn't been planned on or even conceived of (at least AFAIK) when the project started. It goes far beyond the bog standard SQL queries you'll find in introductory material.
On top of that I think this is actually great FLOSS news, knowing how many commercial products have sacrificed proper professional programming on the altar of shareholder expectations.
Congrats on the team for putting stability ahead of features for such an important codebase.
Multiple features?
> Maybe we just didn’t do a good enough job. Maybe the code was just not good enough. We’ll learn and improve. It’s probably some of that, and I’m mentioning it here to not give the impression that it was only the other factors that I’m going to mention.
> It could also be a coincidence. Maybe we had been on such a good roll that we attempted to tackle several features that turned out to be too hard, or at least too hard to finish in time.
All of the above seem clearly the case--not that it could be realized at the time of cutoff and not getting it cleaned up in the window.
> we, meaning all the developers affected, have had enough time to fully reflect on all this.
Title should be: The reverts will continue until quality improves. Humour is good for the clicks but also a means of deflection. I sort of feel bad saying without putting in effort contributing myself but having it merged would have been worse.
Previously my main concern was with LLM's writing worse code. Now my concern is the ability to write better code than I was ever able to write before -- considering every possible potential bug and race condition and error state and what have you -- but at what cost? How much more time do you have to spend researching, designing, making decisions around every possible edge case and state?
Before, you couldn't even attempt to balance quality with speed, because you couldn't see the quality in advance. You had to wait for bug reports to come in. Now, you're confronting quality up-front in a much deeper way. Which allows for much more reliable software, but then you have to decide how much effort is worth it and how much isn't?
In it he pointed out a huge deficiency of most programming languages that still haunts us: they’re wickedly difficult to reason about. So proving important properties about programs is all but infeasible.
So we try to handle it empirically with unit tests, but that has its limitations.
LLMs can kinda sorta do it analytically, but that isn’t completely satisfying, either. The results are probabilistic, and they are pretty bad at inferring intent which can be a source of both type I and type II errors. But still, it would be silly to throw out a layer of Swiss cheese for having holes.
But I sure do wish we could have a plausible version of Backus’s original vision of a language that lets us formally declare (a useful subset of) the specification and then statically verify that the program conforms to it.
a particular statement to make during the slop apocalypse
Not quite as bad as those recipe sites that expect you to read the rando's life story before the recipe, though.
So a lot of this increase in bug reports lately is simply that we are more able to find issues which would previously have been shipped and probably wouldn’t have even been noticed by users for a long time. The minimum bar for software quality is going up, there's no excuse to ship software which contains bugs that could be detected by an LLM.
> Did someone not understand what went in?
Unlikely, from the context.
> Might someone have offloaded their thinking to some external tool?
No, according to the author, since the features predate most current LLM assisted coding.
> Did people stop caring?
I'm not sure what this means. Presumably things were reverted because people cared enough to make sure it was good quality before being released.
> Did someone not plan or design a feature in full?
An admitted possibility
> Were the timelines unrealistic?
Depending on what you mean, that may be correct according to the article, but I suspect not in the way you mean.
> Maybe with so many reverts timelines should be revised?
It sounds like there was a very large increase in load in some of the people that would commonly review during this time period, and that may or may not continue to happen to a greater or lesser degree in the future, it depends on how LLMs advance in finding bugs and if there are other large waved of bugs reported.
This is quite a nice perspective to end up on, and a little humor too.
Some of the beauty of linux that wouldn't happen in corp.
Also adversaries are using abliterated LLMs right now to search for vulnerabilities, so it's important to stay ahead.
Or if not the title, the general context is this: https://news.ycombinator.com/item?id=50119991
A bunch of features that made it to beta turned out not to be in a state that could be made satisfactory for release time and so was reverted.
That seems to me to be the right thing to do. Unfortunate, but the right thing to do.
“[something] will continue until morale improves” is usually a sacastic dig at tone deaf and incompetent leadership (e.g, “firing will continue until morale improves”)
I don’t think that is fair to say here.
Since people refuse to read things on HN, I might as well bubble up this particular bullet point.
Generally, software engineering like other industries is evolving and there is need for different facilities. Consider graph and vector databases featuring, as well as json columns as examples of things you that are relatively recently in common use.
sounds like a revolutionary rather than incremental change, does that get into postgres?