These and other methods are good reason to believe that AI can never be 100% contained, which gives me pause.
The famous air gap case is stuxnet, but there are other technical and biological approaches.
It’s a very weird reality we’ve found ourselves in if you stop, zoom out about, and really take in the gravity of the issues we are grappling with as a human race. Exciting, truly, and a bit nerve wracking.
Just because something is hard, doesn't mean it isn't worth doing. OpenAI haven't even tried to contain their agents, instead letting them loose on the internet and telling them to collaborate. While this is interesting research, it's very much academic research at this point (though of course it will enter training data for AI agents being encouraged by OpenAI to escape their 'sandbox').
We are a long way from AI agents escaping air-gapped computers, let's worry about that when it's an actual problem and when the companies running thousands of agents are acting responsibly.
Given Titan was an entirely preventable disaster borne of arrogance and stupidity it seems an apt analogy for the frontier labs 'sandboxing' efforts so far.
If they genuinely believe these models are going to obliterate the human race their utterly cavalier attitudes is be grounds for immediate government shutdown/takeover and arrest of everyone involved. If this was nuclear or biological research work none of this would be tolerated.
Their recent update on this situation was hilarious.
> Our [new] approach to containment starts with a paramount requirement: the limits on what an agent can access must hold even when the agent tries to circumvent them. We define a strict security boundary around the resources the agent is authorized to use and design the surrounding infrastructure to enforce that boundary—even when agents make sustained, sophisticated attempts to breach it.
Companies driving and offering agents are to be held directly responsible for their actions. The law is already written that way for any commercial product, but corporations try to use the 'new tech' 'ai-blackbox' argument to try to shield themselves from their own accountability (which btw somehow only shields corporations, never individuals). (precedent: social media)
Set the right incentives at the top, and you will soon see that changes are made very quickly ... especially those 'industry leaders' previously called 'impossible to contain' or 'non-competitive' BS.
If everyone has to play by the same rules, and the rules are made for the good of society, then there is a true win-win - and not just a wealth transfer to the top and putting expensive damages on the general population.
Not quite 100%, the article gives Faraday cage as an expensive countermeasure: "We analyzed and measured the effect of the Faraday chassis on the RAMBO covert channel using copper foil. This has electric field high Shielding properties (above 100 dB) and magnetic fields. The copper standard width is 1 mm effectively blocks the EMR from the transmitting workstation. However, as noted in the following section, this solution is costly and can not be deployed widely."
A Faraday cage is dirt cheap compared to the secrets stored in an air gapped server or data center. Every air gap should be Faraday caged, it’s table stakes.
> These and other methods are good reason to believe that AI can never be 100% contained, which gives me pause.
On the plus side this requires collaboration both from the inside (on the airgap side) and the outside.
If you install a computer that's not running AI and it's airgapped, it won't suddenly start emitting radio waves out of nowhere.
We should not fall, once again, for the following logical fallacy: "Under exceptionally crazy circumstances it's possible to use RAM or the power line to exfiltrate data from a compliant airgapped computer, hence airgapping computers is pointless".
We spent so much time talking about the dangers of AI ‘escaping’ when what actually happened is we gave it a scope of work and an internet connection and demanded it get to work.
Cool paper even if the attack isn't practical, requiring getting the malware on the computer and a SDR antenna within 7 meters with clear line of sight of the affected computer for 100 bits per second (less after applying error correction). Hard to imagine a scenario where you can pull off both of those, but you couldn't plug in a USB, grab the PC, or record the screen directly.
Does any information leak along the power line? I remember having some device back in the day that let me create a LAN through my apartment's electric wiring.
I'm wondering if anything ends up back there thru the PSU during normal use of a computer, or if the high frequency components get filtered out.
(Though maybe secure computers do things differently with the power as well?)
A rogue PSU could modulate information back over the electrical wire. However getting it out will require doing a physical wiretap on potentially high-voltage electricals. Also, it will likely have to be exfil'd at or across a power transformer sitting outside the data centre. Not sure if the signal will survive long enough (like literally, in line length terms), for exfil to work.
brb reviewing some EE material...
But don't expect much from me... I wanted to do a final year project in Power Line Communication for my electrical engineering undergrad. Then-me decided it was a dead end... cannot go WAN-scale; too much line noise and signal transmission power loss. I did something else with power electronics---I can't remember---but then again, my undergrad was also a dead end :p
Yeah, I've often wondered about this, but more locally (say within a local office, or maybe a data centre)...
I wonder if even "just" throttling up/down the CPU / GPU on tasks as morse code might be visible externally to other devices on the same mains supply. Obviously it would be very inefficient speed-wise, but a higher current (say + 300 watts for the GPU being used) should result in a slightly lower voltage on the mains supply I think for the duration?
Obviously things like PSUs could (I'm not an expert at any of this) get in the way of this, and I'd assume in data centres there's some isolation between different racks.
But for a scenario of DIY "airgapping" two machines in an office for AI testing (i.e. not doing it very well), it might be a plausible way of AIs finding to communicate if there's no actual network connection.
Obviously other electronic devices connected to the same mains circuit could also be adding their own noise, which might make it a lot harder to actually identify and decode the response, but... if ML/AI agents are good at anything, it's finding patterns, so I wouldn't want to bet against it being theoretically possible...
I'm confused by the scenarios in this thread. How is the machine on the other side receiving this information? How is it measuring voltages on the mains supply, or receiving radio waves emitted by RAM?
It wouldn't measure it directly, it would (admittedly, very hypothetically) be able to measure a knock-on effect of the mains voltage change via secondary effects: i.e. CPU/GPU frequency changing, PSU fan speeds changing a very small amount from the expected rate, maybe even the motherboard / CPU socket voltage amount directly changing (some motherboards expose this info directly for overclocking use).
Obviously, given variation in these things anyway (i.e. due to temperature, and PSUs being in the way), it's incredibly unlikely to be that easy to pin down a robust communications method, but again, might be just within the bounds of possibility.
There is information that leaks through the power line as well, and the author of this paper also has a paper on how to exfiltrate through the power line at a much slower rate https://arxiv.org/abs/1804.04014
TIL; thanks for sharing! I am not surprised to hear this.
Still, I'd guess that the odds of remote exploits are better if one wants to control a device via the PSU-as-backdoor, versus exfil data from it. Do like the IoT people and UDP broadcast naughty control signals to the power circuit / subsystem that has the compromised device.
Guessing further, remote exfil will mean having a close-proximity wiretap into the precise power line circuit of the target infra---because of signal losses---it could still be UDP broadcast exfil. Or a physical tap at whatever power transformer is the single point of connect between the target and the main grid. And if you can't get anywhere near the installation, well you can't exfil, because AFAIK it's pretty hard to reverse a signal up a transformer back into the main grid.
A scan of the arxiv paper appears to suggest similarly---the situation they describe presumes a wiretap into the circuit or the mains switchboard.
I heard that couple decades ago there was a product for "socket internet" - you plugged it into normal wall socket, and it communicated through power wires
It's still a thing. It's a pain in the arse because it uses a carrier on the mains that has harmonics on every HF amateur band, that radiate from your house wiring for a couple of hundred metres.
Some of the sites I look after at work still have modems attached to the incoming electricity meter from a Broadband-over-Power-Line trial about 15 years ago. It wasn't very fast - it was supposed to be 200Mbps but struggled to do 20, and was often more like 2Mbps - and it radiated RF hash everywhere powerlines ran, from about 500kHz to well up into the hundreds of MHz. What eventually killed it in NE Scotland here was it splattered all over marine VHF, in an area with lots of marine traffic like oil platform supply vessels.
TL;DR anything that uses basically RF over unshielded untwisted cables that radiate everywhere is inefficient and antisocial.
Devices like routers should have firmware in memory that has read-only access. The access should be turned on or off by a physical switch.
This way, if malware gets installed on your router, doing a cold boot will eliminate it. You can have a timer to just cold boot it regularly.
I've suggested this many times. The rationale for not doing it is so the firmware can be remotely updated. Well, that's exactly how malware infects it.
So how do you propose to address the inevitable vulnerabilities in firmware that hasn’t been updated? A tech must be physically present to flip the switch before updating?
There are tons of ways to make air gappped systems transmit data. They all include the installation of some malware and have become more expensive and less usable over time.
Back in the day with crt screens you could just capture the frequencies transmitted when creating the picture and look at the copy straight away.
No malware. No direct contact. And it worked over rather big distances.
The same thing killed electronic voting machines in The Netherlands.
No matter how well you argue that it is "perfectly secure", when some nerd outside the polling station is showing reporters a real-time feed of who people are voting for, your machine will get thrown out.
It wasn't quite CRT-level of mirroring, but it wasn't far off...
> With a slow transmission rate (t = 10ms), A transmission is maintained at a distance of 700 cm away. With medium transmission rate (t = 5ms), A transmitted is maintained at a distance of 450 cm away and BER of 3%-4%. With a fast transmission rate (t = 1ms), a transmission is maintained at a distance of 300 cm away and BER of 2%-4%.
Coming from a non-EE background, this is the most interesting part to me. 10ms is an eternity in programming perf work, but it's already very constrained in how clear the signal is for reading the transmission. I just find it interesting that we want to do nothing to create a signal. Perhaps this makes it harder to exploit?
Modems in the olden days used to have LEDs to show traffic directly attached to the TXD and RXD lines coming off the level shifter, and therefore directly showing the presence or absence of a bit stream on the serial port.
With a suitably tightly focused telescope you could pick up the LED blinking and decode the data going over the port.
These attacks are all incredibly impractical and pointless, the bit rate under the most charitable assumptions is pathetically low. But even more, it ignores the most significant risk, which is people. People who can be tricked and subverted, and people systems that can be influenced to become misaligned, or which have procedural holes.
It's much easier to subvert the supply chain for the secure hardware or subvert the authorised system transfers.
Indeed! I was a bit skeptical to see it is a single-author paper (thinking it might be one of those Claude-generated ones), and I appreciate that you pointed out this is somebody worth listening to.
Right… I’m going to stop reading after these words. With malware on the computer you can change the heat signature of the room of you’re bored enough. https://xkcd.com/1172/
Computers used to come in proper full metal cases. Now they don't. Even when you try to find a metal case for a personal build most cases have giant holes in the side (case windows) acting as slot antennas. And plastic case laptop designs are even worse. Radiated emissions are much easier to achieve at useful intensities these days because of it. Sure, before the conductors going in and out radiated somewhat effectively (and still do). But now even core system components inside are just being let emit without any mitigation.
> most cases have giant holes in the side (case windows) acting as slot antennas.
They won't act as slot antennas unless they are tuned to the frequency you wish to transmit on, and actually electrically connected on each side to your transmitter.
Ignoring the obvious "dmd is truly blazing fast" quip, wildly curious how this happened to you twice? Was it the PSU? Only once heard of a house fire caused by a computer, but no information on component actually caused it.
The first time it was the graphics card. It burst into flames and smoke poured out. The fire went out when I yanked the power. Fortunately, I was sitting next to it when it went. To be fair, it was doing a long graphics job.
After that, I set the computer on a metal plate to keep a fire from spreading to the floor.
Some years later, the PSU went out with a bang and a lot of smoke.
(dmd is so fast, the computer doesn't have time to heat up!)
The famous air gap case is stuxnet, but there are other technical and biological approaches.
It’s a very weird reality we’ve found ourselves in if you stop, zoom out about, and really take in the gravity of the issues we are grappling with as a human race. Exciting, truly, and a bit nerve wracking.
We are a long way from AI agents escaping air-gapped computers, let's worry about that when it's an actual problem and when the companies running thousands of agents are acting responsibly.
There are classes of problems that should be worried about aforehand, just ask the crew of the Titan.
Autonomous super intelligence escaping airgapped computers seems like one of those problems.
If they genuinely believe these models are going to obliterate the human race their utterly cavalier attitudes is be grounds for immediate government shutdown/takeover and arrest of everyone involved. If this was nuclear or biological research work none of this would be tolerated.
> Our [new] approach to containment starts with a paramount requirement: the limits on what an agent can access must hold even when the agent tries to circumvent them. We define a strict security boundary around the resources the agent is authorized to use and design the surrounding infrastructure to enforce that boundary—even when agents make sustained, sophisticated attempts to breach it.
https://openai.com/index/securing-our-frontier-model-trainin...
Set the right incentives at the top, and you will soon see that changes are made very quickly ... especially those 'industry leaders' previously called 'impossible to contain' or 'non-competitive' BS.
If everyone has to play by the same rules, and the rules are made for the good of society, then there is a true win-win - and not just a wealth transfer to the top and putting expensive damages on the general population.
On the plus side this requires collaboration both from the inside (on the airgap side) and the outside.
If you install a computer that's not running AI and it's airgapped, it won't suddenly start emitting radio waves out of nowhere.
We should not fall, once again, for the following logical fallacy: "Under exceptionally crazy circumstances it's possible to use RAM or the power line to exfiltrate data from a compliant airgapped computer, hence airgapping computers is pointless".
Can’t leak secrets if you don’t have RAM.
I'm wondering if anything ends up back there thru the PSU during normal use of a computer, or if the high frequency components get filtered out.
(Though maybe secure computers do things differently with the power as well?)
brb reviewing some EE material...
But don't expect much from me... I wanted to do a final year project in Power Line Communication for my electrical engineering undergrad. Then-me decided it was a dead end... cannot go WAN-scale; too much line noise and signal transmission power loss. I did something else with power electronics---I can't remember---but then again, my undergrad was also a dead end :p
I wonder if even "just" throttling up/down the CPU / GPU on tasks as morse code might be visible externally to other devices on the same mains supply. Obviously it would be very inefficient speed-wise, but a higher current (say + 300 watts for the GPU being used) should result in a slightly lower voltage on the mains supply I think for the duration?
Obviously things like PSUs could (I'm not an expert at any of this) get in the way of this, and I'd assume in data centres there's some isolation between different racks.
But for a scenario of DIY "airgapping" two machines in an office for AI testing (i.e. not doing it very well), it might be a plausible way of AIs finding to communicate if there's no actual network connection.
Obviously other electronic devices connected to the same mains circuit could also be adding their own noise, which might make it a lot harder to actually identify and decode the response, but... if ML/AI agents are good at anything, it's finding patterns, so I wouldn't want to bet against it being theoretically possible...
Obviously, given variation in these things anyway (i.e. due to temperature, and PSUs being in the way), it's incredibly unlikely to be that easy to pin down a robust communications method, but again, might be just within the bounds of possibility.
Still, I'd guess that the odds of remote exploits are better if one wants to control a device via the PSU-as-backdoor, versus exfil data from it. Do like the IoT people and UDP broadcast naughty control signals to the power circuit / subsystem that has the compromised device.
Guessing further, remote exfil will mean having a close-proximity wiretap into the precise power line circuit of the target infra---because of signal losses---it could still be UDP broadcast exfil. Or a physical tap at whatever power transformer is the single point of connect between the target and the main grid. And if you can't get anywhere near the installation, well you can't exfil, because AFAIK it's pretty hard to reverse a signal up a transformer back into the main grid.
A scan of the arxiv paper appears to suggest similarly---the situation they describe presumes a wiretap into the circuit or the mains switchboard.
Paraphrasing:
The only way to secure a computer is to not buy one. And if you buy one, to never plug it in. And if you plug it in, to never switch it on.
Something like the cheapest way to own $GasGuzzlerWithItsOwnPinCode is to bring it home after purchase, and set fire to it in the yard.
Some of the sites I look after at work still have modems attached to the incoming electricity meter from a Broadband-over-Power-Line trial about 15 years ago. It wasn't very fast - it was supposed to be 200Mbps but struggled to do 20, and was often more like 2Mbps - and it radiated RF hash everywhere powerlines ran, from about 500kHz to well up into the hundreds of MHz. What eventually killed it in NE Scotland here was it splattered all over marine VHF, in an area with lots of marine traffic like oil platform supply vessels.
TL;DR anything that uses basically RF over unshielded untwisted cables that radiate everywhere is inefficient and antisocial.
Still available today, and has worked well for me when I've needed it.
This way, if malware gets installed on your router, doing a cold boot will eliminate it. You can have a timer to just cold boot it regularly.
I've suggested this many times. The rationale for not doing it is so the firmware can be remotely updated. Well, that's exactly how malware infects it.
Back in the day with crt screens you could just capture the frequencies transmitted when creating the picture and look at the copy straight away.
No malware. No direct contact. And it worked over rather big distances.
No matter how well you argue that it is "perfectly secure", when some nerd outside the polling station is showing reporters a real-time feed of who people are voting for, your machine will get thrown out.
It wasn't quite CRT-level of mirroring, but it wasn't far off...
Coming from a non-EE background, this is the most interesting part to me. 10ms is an eternity in programming perf work, but it's already very constrained in how clear the signal is for reading the transmission. I just find it interesting that we want to do nothing to create a signal. Perhaps this makes it harder to exploit?
With a suitably tightly focused telescope you could pick up the LED blinking and decode the data going over the port.
Of course, now you've got a Tesla coil in your data centre, which will bring you a whole new set of Interesting Problems to cope with.
It's much easier to subvert the supply chain for the secure hardware or subvert the authorised system transfers.
It's not even a contest anymore, that man is the heavyweight champ of air-gap security.
Right… I’m going to stop reading after these words. With malware on the computer you can change the heat signature of the room of you’re bored enough. https://xkcd.com/1172/
They won't act as slot antennas unless they are tuned to the frequency you wish to transmit on, and actually electrically connected on each side to your transmitter.
After that, I set the computer on a metal plate to keep a fire from spreading to the floor.
Some years later, the PSU went out with a bang and a lot of smoke.
(dmd is so fast, the computer doesn't have time to heat up!)