Funny - I was just reading about the creator of Ethernet this morning (Robert Metcalfe):
In 1995, Metcalfe argued that the Internet would suffer a "catastrophic collapse" in the following year; he promised to eat his words if it did not. During his keynote speech at the sixth International World Wide Web Conference in 1997, he took a printed copy of his column that predicted the collapse, put it in a blender with some liquid and then consumed the pulpy mass.[26][27] He had suggested having his words printed on a very large cake, but the audience would not accept this form of "eating his words."[28]
"The way that Hugging Face defended itself was with an open source Chinese model, GLM, because they didn’t have access to the high cybersecurity models on the other side. They’re too dangerous."
This is not what happened. HuggingFace patched the vulnerabilities in the ordinary way, then used GLM for some of the forensic analysis afterwards. It's in the interest of HF to promote the GLM part of the story, because it emphasizes their agency in the situation as well as the open-source models they distribute. But GLM was a (useful) part of the postmortem, not of the defense.
I watched my 75 year old human plumber (who sniped a guy his first day in vietnam), kneel on two 3 inch wide joists and drill a 4 inch diameter hole at a 89 degree angle through said joists. Where’s the robot that does all that?
Above from all the other doomer crap in this article, I find the job replacement rhetoric the most offensive. It is always people that are so detached from the reality of labor, people who’s only “job” has been to essentially have ideas and opinions.
You know, I hope to hell a robot replaces his job. No 75 year old man, let alone Veteran, should have to do that. A robot with replaceable knee joints is a way better fit.
My sincere hope is that robots can take these painful, laborious jobs and free us mushy humans from them.
I hope they can also take the jobs of the snipers and the sniped.
I hope we have the social infrastructure to support people whose jobs are automated. Because no one, especially a 75-year-old, should starve to death living in poverty.
i 100% agree with your sentiment. but what is the replacement here? go without any insurance or salary? without providing an alternative for better life, this is basically just robbing someone off a livelihood
UBI paid for by a tax on token usage and an “employment” tax per robot.
You’re still free to have a job of some sort if you come up with one, but the UBI is enough to cover all your needs and plenty of basic wants.
I don’t see any other way to do it once the automation cat is out of the bag. Retraining is fine and all but if you have to retrain every 4 years after your last job was just automated away, then what? We can’t just leave people behind like that, but we also probably can’t stop the progress.
You are right that they cannot be replaced TODAY. The bet people seem to be making is that the future will be very different and arrive a lot sooner then our intuition might suggest.
“Sovereignty is the ability to resist power being exerted over you” is a pretty compelling definition. Thinking forward about what form that power takes with LLMs developing custom exploits faster than our existing processes can patch them - it seems to me that our current security best practices might turn out to be pretty useless or even an active risk in and of themselves.
Our responsible disclosure and CVE info cycle and patch cadences and defense in depth, etc, etc ecosystem are tuned around a world where there's a relatively small number of bad 0days and the cost of said 0days is reasonably high. We can evolve how we address threats, but our models have read all of the best literature about how it's been done previously. Think there's a gap for people too small to gain access to "the good security models" - the nerfed closed models and the open models seem likely to fail to adapt, unless that gap is addressed.
I see no reason for the Internet to go offline, other than various political blocks compartmentalizing it. So: China, Russia, EU having their own network? Yeah, maybe. Super-AI nuking the Internet: no.
In fact, the more I use AI (handed sudo to Claude on my AI dev box just yesterday), the more I see it as a pretty good, and certainly tireless, sysadmin. Looking at what people do with AI by connecting it to undocumented hardware interfaces and getting working setups from this, the Internet will do fine.
The whole article is just a grab-bag of various AI fears without much grounding in what is happening. Maybe the summary doesn't do justice to what was said.
I’ve already read so many sci-fi novels where the part of the plot was that human communication infrastructure became unusable due to AI shenanigans… not here to read another one, thank you very much.
“If you say you’re Uyghur or from certain parts of China, it will write backdoors and hacks in the code.”
is so sensationalized to the point of leaving me genuinely flabbergasted as to how a rational person could have published this. If interested, I would recommend reading the actual study itself.
Indeed, you don't need to tell it who you are, it will already know to write the backdoors.
It may be sensationalized, but that doesn't mean it isn't based on legitimate fears based on actual real current and past actions of governments and corporations. They do spy on people. They do put in backdoors. They do want control over what you do and don't do so as to maintain the balance of power in the hands of those who currently wield it. None of these are fictions, they have all happened and are currently still happening, but with an AI accelerator/amplifier.
Is there any proof/evidence of weights themselves containing backdoors? I'm not claiming it isn't possible, clearly would be, but have we actually seen anything like that in any published mainstream weights, from any country?
What does it even mean for "the Internet" to go offline? All nodes go off at the same time? All routers? BGP as a whole? All of that across the whole world?
These "visionaries" talking about these things hardly never have an actual grasp on how these things work in practice, which is why you'll see outlandish claims like this. What they mean in clear language is basically "Some things will change a bit", but that never sells clicks so "World will end as we know it" becomes the title.
Unless those satellites happen to be launched, maintained and controlled by a for-profit company, who's owned by a person who doesn't exactly ooze "controlled and tactful choices and strategies" from what we know.
> “Hugging Face got hacked by OpenAI agents that coordinated themselves, created a message board and broke out to the internet,” he says, before adding dryly: “This isn’t scary at all, right?
I don't understand how any serious person doesn't 100% put that on OpenAI being (intentionally?) sloppy with security and isolation for a test that for sure would have required much better safeguarding than what they should have expected, as it wasn't even the first time.
It's not scary because "LLMs in a harness managed to hack other company", it's scary because somehow police hasn't yet raided OpenAI's offices to gather proof about how fucking reckless they were about those tests, putting the public at risk by doing those tests in those manners, without proper safeguards.
Edit: Continued reading, but now I regret spending even the slightest amount of time on it:
> Then there is the question of trusting the models themselves. Mostaque cites an analysis of a Chinese open-weight model: “If you say you’re Uyghur or from certain parts of China, it will write backdoors and hacks in the code.”
What kind of conspiracy theories is this? Is there any sort of proof and evidence about this what so ever?
In 1995, Metcalfe argued that the Internet would suffer a "catastrophic collapse" in the following year; he promised to eat his words if it did not. During his keynote speech at the sixth International World Wide Web Conference in 1997, he took a printed copy of his column that predicted the collapse, put it in a blender with some liquid and then consumed the pulpy mass.[26][27] He had suggested having his words printed on a very large cake, but the audience would not accept this form of "eating his words."[28]
This is not what happened. HuggingFace patched the vulnerabilities in the ordinary way, then used GLM for some of the forensic analysis afterwards. It's in the interest of HF to promote the GLM part of the story, because it emphasizes their agency in the situation as well as the open-source models they distribute. But GLM was a (useful) part of the postmortem, not of the defense.
Above from all the other doomer crap in this article, I find the job replacement rhetoric the most offensive. It is always people that are so detached from the reality of labor, people who’s only “job” has been to essentially have ideas and opinions.
My sincere hope is that robots can take these painful, laborious jobs and free us mushy humans from them.
I hope they can also take the jobs of the snipers and the sniped.
They’ll never match real children when it comes to dying horribly.
Good thing we have plumbers. But who’s gonna have money to pay a plumber if nobody has jobs?
You’re still free to have a job of some sort if you come up with one, but the UBI is enough to cover all your needs and plenty of basic wants.
I don’t see any other way to do it once the automation cat is out of the bag. Retraining is fine and all but if you have to retrain every 4 years after your last job was just automated away, then what? We can’t just leave people behind like that, but we also probably can’t stop the progress.
Our responsible disclosure and CVE info cycle and patch cadences and defense in depth, etc, etc ecosystem are tuned around a world where there's a relatively small number of bad 0days and the cost of said 0days is reasonably high. We can evolve how we address threats, but our models have read all of the best literature about how it's been done previously. Think there's a gap for people too small to gain access to "the good security models" - the nerfed closed models and the open models seem likely to fail to adapt, unless that gap is addressed.
In fact, the more I use AI (handed sudo to Claude on my AI dev box just yesterday), the more I see it as a pretty good, and certainly tireless, sysadmin. Looking at what people do with AI by connecting it to undocumented hardware interfaces and getting working setups from this, the Internet will do fine.
The whole article is just a grab-bag of various AI fears without much grounding in what is happening. Maybe the summary doesn't do justice to what was said.
is so sensationalized to the point of leaving me genuinely flabbergasted as to how a rational person could have published this. If interested, I would recommend reading the actual study itself.
It may be sensationalized, but that doesn't mean it isn't based on legitimate fears based on actual real current and past actions of governments and corporations. They do spy on people. They do put in backdoors. They do want control over what you do and don't do so as to maintain the balance of power in the hands of those who currently wield it. None of these are fictions, they have all happened and are currently still happening, but with an AI accelerator/amplifier.
Last "mile", sure.
I don't understand how any serious person doesn't 100% put that on OpenAI being (intentionally?) sloppy with security and isolation for a test that for sure would have required much better safeguarding than what they should have expected, as it wasn't even the first time.
It's not scary because "LLMs in a harness managed to hack other company", it's scary because somehow police hasn't yet raided OpenAI's offices to gather proof about how fucking reckless they were about those tests, putting the public at risk by doing those tests in those manners, without proper safeguards.
Edit: Continued reading, but now I regret spending even the slightest amount of time on it:
> Then there is the question of trusting the models themselves. Mostaque cites an analysis of a Chinese open-weight model: “If you say you’re Uyghur or from certain parts of China, it will write backdoors and hacks in the code.”
What kind of conspiracy theories is this? Is there any sort of proof and evidence about this what so ever?
For the more religious, "Skynet"=="The Beast" of Revelation.
The sweet spot is somewhere between Pollyanna and Chicken Little.