> A chatbot account tied to Alfozan’s spouse, which shared identifiers with his own account, submitted prompts between Sept. 24 and Sept. 30 about building, installing, obscuring, and remotely accessing a recording device, prosecutors said. Queries included how to “build a continuous audio recorder” and “encrypt the SD card,” prosecutors claimed. On Sept. 27 and 28, the account asked about the specific device that was later found, including its unique MAC address.
Just to be clear, the difficult choice is not whether to use OpenAi/Anthropic/Grok/... versus a local LLM.
The choice is whether to hang out with people with whom you can't be sure they don't have a Plaud/Pocket/.. in their literal pocket which is continuously listening.
Unfortunately the set of such people will eventually be shrinking more and more until eventually only a few tinfoil folks are left. Dystopian? Just think back w.r.t. continuous tracking by Google/Facebook/.. Unthinkable 30 years ago, total standard today. Try to count the fraction of your emails that don't actually reside on a MSFT or Google server because the other end has an account there. That fight is over. You can just as well join them.
Prepare to have that dug up 20 years in the future during some court proceedings.
Or at the border where it is already routine to ask for social media accounts (especially foreign nationals) you might someday five the border agent your login.
Especially since you an just buy one, like Plaud and tape it to the underside of the desk, surreptitiously, instead of suspiciously accessing the back of the TV.
I am not sure whether it bodes well for humanity that the advice "If you want to break the law use local abliterated model or at least a Chinese one" actually has to be written down.
Recall that multiple of the Chinese LLM providers got caught sending at least some of their queries to Claude. That would be a pretty funny way to get caught in turn doing something by the US government.
"OpenAI personnel found the device Sept. 30. It was plugged into a USB port on the back of a television in an executive conference room and had been concealed in toilet paper,"
I wonder what the IQ of someone needs to be to think that covering it up with tp makes it less suspicious instead of more. And I also wonder what the hiring process was like to hire someone with that mental frame.
> And I also wonder what the hiring process was like to hire someone with that mental frame.
Remember, this is only the company responsible for potentially building and aligning AGI. No need to be setting high expectations toward their sandboxing or hiring capabilities.
Given that this is being described as an "executive conference room", it's more likely an attempt at insider trading than espionage. Decisions made in those rooms at OpenAI move markets (c.f. DRAM) for sure.
OpenAI is both a company that manages a lot of money and a lot of political power, that the target was an "executive conference room" gives no indication as to whether the motives were financial or political.
What might tilt the bayesian balance in favour of political intent would be the nationality of the individual, state or religious sponsoring become hypotheses.
Honestly I think you can rule out "political" already, no? It's far easier to just bribe politicians for that kind of intelligence. My point was that they weren't stealing models, they wanted decisionmaking intel.
> "Scott Schelble, special agent in charge of the FBI’s San Francisco field office, said the agency is committed to protecting sensitive research and proprietary technology."
While the headlines and quotes seem designed to imply sophisticated industrial espionage, potentially by a state-level actor, what's public so far isn't adding up that way.
1. You're not going to get much actionable research material on proprietary technologies by audio bugging what's said in an executive conference room. That's what you'd do if you were after economic or deal negotiation info.
2. Hard to imagine a state-level actor buying a ~$15-$35 hobbyist ESP32 board from Seeed Studio (I've ordered a couple from there for home automation myself) and then asking a public chatbot for set up instructions.
3. They say he was an "engineer" who'd been there ~15 months. If you're an actual engineer at OAI and past the 1 year cliff, you're probably already sitting on RSUs likely to be worth at least $1M after OAI's IPO - and maybe much more. That's a lot of incentive to not get fired and probably makes you pretty hard to recruit as an internal mole for cash.
A service you delegate your thinking to (Brain-aaS?), which the provider can also instruct to report anything suspicious... What a world we have now.
I wonder if the BaaS-providers monitor the accounts of decision makers. They could profit if they figure out what e.g. a central banker is going to decide. Or... if they're the ones deciding for him.
Hah, you can even streamline that by having an AI agent that autonomously makes decisions based on insider info...
> Remember kids, uncle Sam can see anything you tell your AI and will use that against you.
Not just your AI.
You could have stopped the sentence at "uncle Sam can see anything" and these days it's pretty much true. They will use all of it against you if they can. Unless you truly want to live off the grid, if the state takes a sufficient interest in you, expect everything you do to be seen.
Uncle Sam tried to get MLK to kill himself by sending him letters alleging knowledge of an affair [1]. Getting some idiot's wife's chatgpt logs represents a fairly routine law enforcement action these days, and as such is really rather unremarkable. The State is always something of a terror. Really under most (if not all) conceptions, it necessarily has to be. The monopoly of force is a comfort until the Leviathan knocks on your door.
> ... a chatbot account associated with Alfozan’s spouse (which shared multiple identifiers and attributes with Alfozan’s own chatbot account) submitted prompts between Sept. 24 and Sept. 30 related to building, customizing, installing, audio recording, testing, obfuscating, and remote accessing a device, including inquiries about “build[ing] a continuous audio recorder,” “customiz[ing] Wi-Fi . . . connectivity,” “improv[ing] audio and live listening,” and “encrypt[ing] the SD card,” among other things. The same chatbot account also submitted prompts specifically related to Recording Device 1, including its custom header and unique MAC address, between Sept. 27 and Sept. 28.
The justice.gov article doesn't specifically say it was an OpenAI account, but I think, based on the fact that they have logs including MAC address of the device, that we could probably assume that. The alternative is that other chatbot account providers are collaborating.
Regardless, doing a string search on a MAC address for a device you find inside your property doesn't strike me as an abuse of the data that is also your property.
Not the smartest engineer ever, that use case perfectly align with using a Chinese model! And anyway I have no idea how he was hoping not to be caught in the case the device would be discovered.
in addition to the whole crime thing, an 'openai engineer', apparently engineering lead according to his linkedin, having to ask chatgpt to tell him how to set up a recorder or how to encrypt a sd card is something. Truly the cream of the engineering crop at those top AI labs
I doubt it. Storage is expensive and costs add up. There are with great certainty many accounts that are marked as non-schmuck and have retain forever policy, but for the schmuck - I think that after year or two your Griffit/Fulgrim slash fiction will go down the drain. I am not sure how many petabytes per minute in logs openai and anthropic generate, but I would guess more than 1.
I actually am curious if they were able to volunteer the information to the government of their own volition, without a government subpoena first. I would think that could violate their own terms if done sloppily. Maybe different if an employee or maybe they had to invite the government to ask for them or something? In any case, the mechanism aside, the state would have asked anyway (say if he used another company's chatbot) so the fact that it's the "same company" is rather irrelevant. The evidence would have ended up in the state's hands one way or the other, and I would think a search warrant in this case would have been rather routine. Search warrants are infamously not a high bar to climb in most jurisdictions, as often the presiding judges tend to default to deferring to the state.
I used to joke that the nsa should offer a paid service for no-setup backups. You just tell them what files you've lost, and they can send you a copy for a fee.
This is a very weird take to me. They're demonstrating they're able to in-house this cybersecurity investigation, leading to the alleged perpetrator's arrest, and you think that makes them look "desperate and cheap?"
Yes? I mean, I don’t think it’s a character flaw to not know things, but their rivalry is extremely notorious and relevant. Someone who can't keep them straight won’t be able to understand what’s going on in AI.
It could be that the reports are accurate and the device simply wasn’t as sophisticated as the statement implies. You could if you wanted use an AI model to analyze a USB camera with device name “John_Doe_1”.
I mean really what's the big deal? Gathering information from unsuspecting parties who aren't aware it's happening? He was just doing what his employer does.
Thank god security at a company which...checks notes...has remote code execution on every developers laptop and access to practically unlimited secrets uploaded by its agents...is top notch.
Model "distillation" lets one model be trained on the outputs of another without having to directly steal it. It will be subject to constant arms race.
The alignment people would like you worry about the model exfiltrating itself, too.
This technique currently works because the architecture of the model is generally published and well known. But, that isn't necessarily going to be the case in the future. For example, if some new model type which is not a LLM is discovered which gets closer to AGI, just training an LLM on its outputs will not necessarily give you a competitive model
I could've sworn I read about Palmer Lucky making comments on hardware coming from China with wiretap implants in the last few days. Funny coincidence eh?
Free startup idea: Anti-espionage cameras for corporate offices. Put it behind TVs and plants etc. and it will notify the security team if it detects new objects.
> How did they get that capture of Alfozan looking behind the TV?
“OpenAI personnel found the device Sept. 3…On Oct. 5, the complaint says, Alfozan entered the conference room and appeared to look behind the television.”
Well, The Thing [0] could potentially show up on a probing sweep if you're careful.
Unless you mean, it happened to be actively transmitting during a(n investigative) sweep.
Still, raises some fun concepts. The sort of things where in this current level of technology you'd almost need to supercool a room for a sweep, blast various RF/sound/etc and look for anything that even remotely warms up, to detect (1) whether something detected something worth recording for later as well as (2) something that might be on the right wavelength/etc to trigger a send.
I have seem some articles which reference allegations from court documents / complaint, but I haven't been able to find the primary source. Do we know if that's publicly available yet?
Sounds to me like the feds got involved already, and they're definitely going to want to make a big noise about this as a national security thing:
> “For us it is a matter of national security,” U.S. Attorney Craig Missakian said in a statement. [...]
> The case is being prosecuted by the National Security, Cyber & Special Prosecutions Section of the U.S. attorney’s office for the Northern District of California.
(But that was after OpenAI "referred the matter to the FBI")
There's presumably a strong argument for making this stuff known internally, since you want other employees to a) be on the lookout for it and b) get scared into NOT doing it themselves because someone else got caught.
If you tell your employees, the outside world is going to hear about it too.
A common example is if a journalist gets a tip off and is about to run a story. Being a good old fashioned balanced-reporting type, they call you and ask you for a comment, but rather than giving them your side of story you slam the phone down and blast out a press release before the journalist can even publish.
(Or maybe you stall them on the phone while scribbling a note to your PA saying “BUG STORY LEAKED TO VALLEYWAG PUT OUT COUNTER PR GET AGEAD OF STORY” hoping to get a tweet out before the ValleyWag gumshoe gets off the phone.)
(Or maybe you keep a notion.so page pre-filled with all your secrets and each one has a BURN button next to it, ready for you to tap when ValleyWag call up for a comment on their upcoming exposé in re: PR/Disasters/20206090500039.md)
IPO remember? Our information is so valuable we have foreign actors trying to steal it. Our model is so good it managed to figure out what this recording device was.
Right. I can guarantee that there are tens of thousands of listening devices with embedded radios in the OpenAI office, right now. The only open question is who controls them.
> A chatbot account tied to Alfozan’s spouse, which shared identifiers with his own account, submitted prompts between Sept. 24 and Sept. 30 about building, installing, obscuring, and remotely accessing a recording device, prosecutors said. Queries included how to “build a continuous audio recorder” and “encrypt the SD card,” prosecutors claimed. On Sept. 27 and 28, the account asked about the specific device that was later found, including its unique MAC address.
“I did not kill my wife!”
“Cool so why did you search for how to hide a dead body? the same day she went missing.”
Ive seen people going through Plaud.AI conversation to find "on day X person Y said Z to person W", just to get some amunition against them.
It’s like thinking about edge cases in programming…
Most people are totally not like that. And it does force you into some difficult choices.
I gots to know...
The choice is whether to hang out with people with whom you can't be sure they don't have a Plaud/Pocket/.. in their literal pocket which is continuously listening.
Unfortunately the set of such people will eventually be shrinking more and more until eventually only a few tinfoil folks are left. Dystopian? Just think back w.r.t. continuous tracking by Google/Facebook/.. Unthinkable 30 years ago, total standard today. Try to count the fraction of your emails that don't actually reside on a MSFT or Google server because the other end has an account there. That fight is over. You can just as well join them.
Or at the border where it is already routine to ask for social media accounts (especially foreign nationals) you might someday five the border agent your login.
When the Soviets wanted to spy on the American embassy, they energized the steel rebars to act as antennas. The structure was so wired that the entire building was the bug. https://www.airandspaceforces.com/PDF/MagazineArchive/Docume...
Toilet paper, ESP-32 and Claude "make me a bug please"? Excuse me? How far have we fallen from grace?
https://en.wikipedia.org/wiki/Operation_Snow_White
> An unattended bag found in the company’s offices contained Alfozan’s wallet and identification, as well as a recording device
It’s almost like he wanted to be caught…
I wonder what the IQ of someone needs to be to think that covering it up with tp makes it less suspicious instead of more. And I also wonder what the hiring process was like to hire someone with that mental frame.
Remember, this is only the company responsible for potentially building and aligning AGI. No need to be setting high expectations toward their sandboxing or hiring capabilities.
What might tilt the bayesian balance in favour of political intent would be the nationality of the individual, state or religious sponsoring become hypotheses.
While the headlines and quotes seem designed to imply sophisticated industrial espionage, potentially by a state-level actor, what's public so far isn't adding up that way.
1. You're not going to get much actionable research material on proprietary technologies by audio bugging what's said in an executive conference room. That's what you'd do if you were after economic or deal negotiation info.
2. Hard to imagine a state-level actor buying a ~$15-$35 hobbyist ESP32 board from Seeed Studio (I've ordered a couple from there for home automation myself) and then asking a public chatbot for set up instructions.
3. They say he was an "engineer" who'd been there ~15 months. If you're an actual engineer at OAI and past the 1 year cliff, you're probably already sitting on RSUs likely to be worth at least $1M after OAI's IPO - and maybe much more. That's a lot of incentive to not get fired and probably makes you pretty hard to recruit as an internal mole for cash.
I expect Meta are running an audit right now to figure out if he was involved in espionage back then as well.
Remember kids, uncle Sam can see anything you tell your AI and will use that against you.
I wonder if the BaaS-providers monitor the accounts of decision makers. They could profit if they figure out what e.g. a central banker is going to decide. Or... if they're the ones deciding for him.
Hah, you can even streamline that by having an AI agent that autonomously makes decisions based on insider info...
Not just your AI.
You could have stopped the sentence at "uncle Sam can see anything" and these days it's pretty much true. They will use all of it against you if they can. Unless you truly want to live off the grid, if the state takes a sufficient interest in you, expect everything you do to be seen.
Uncle Sam tried to get MLK to kill himself by sending him letters alleging knowledge of an affair [1]. Getting some idiot's wife's chatgpt logs represents a fairly routine law enforcement action these days, and as such is really rather unremarkable. The State is always something of a terror. Really under most (if not all) conceptions, it necessarily has to be. The monopoly of force is a comfort until the Leviathan knocks on your door.
[1] https://en.wikipedia.org/wiki/FBI-King_letter
But that's crazy talk
It's AI inbreeding all the way down.
We also don’t know if this person did it under duress.
He was a Saudi national, so that is a possibility
There’s plenty of better people out there who just aren’t interested in that sort of stuff.
The justice.gov article doesn't specifically say it was an OpenAI account, but I think, based on the fact that they have logs including MAC address of the device, that we could probably assume that. The alternative is that other chatbot account providers are collaborating.
Regardless, doing a string search on a MAC address for a device you find inside your property doesn't strike me as an abuse of the data that is also your property.
They can confiscate your computers, too.
This person, of all people, would know this. He really thought he’d never get caught.
I actually am curious if they were able to volunteer the information to the government of their own volition, without a government subpoena first. I would think that could violate their own terms if done sloppily. Maybe different if an employee or maybe they had to invite the government to ask for them or something? In any case, the mechanism aside, the state would have asked anyway (say if he used another company's chatbot) so the fact that it's the "same company" is rather irrelevant. The evidence would have ended up in the state's hands one way or the other, and I would think a search warrant in this case would have been rather routine. Search warrants are infamously not a high bar to climb in most jurisdictions, as often the presiding judges tend to default to deferring to the state.
Did OpenAI do a marketing bit in the middle of an espionage accusation?
Plumb your video feed of your executive conference room into our DC and we'll tell you if anybody is spying on you.
Yes, they did. The people who felt it was totally normal to add that would no doubt be surprised how desperate and cheap it makes them look.
"almost surely made in china"
https://www.forth.news/lists/philapd/Cfs1w2A22JiL3fWSzA8Nx
So the back side of the TV and the entire room was under surveillance even before that?
Good time to start a company that builds giant Faraday cages over data centers.
The alignment people would like you worry about the model exfiltrating itself, too.
Free cooling, safe from missile attack, and safe from some types of electronic spying.
Still a significantly less dumb idea than space data centers.
https://www.propublica.org/article/saudi-fugitives-accused-o...
Certainly they would be transparent and very ethical with something like that
“OpenAI personnel found the device Sept. 3…On Oct. 5, the complaint says, Alfozan entered the conference room and appeared to look behind the television.”
The image was captured after the bug was found.
But no the boring answer is that after they found the device they left it in place and planted the camera to see who came looking for it.
in this context, they should be more specific about how terminal s/he is
Back of a TV is a decent hiding place. Wonder how they found it -was it actively transmitting?
Unless you mean, it happened to be actively transmitting during a(n investigative) sweep.
Still, raises some fun concepts. The sort of things where in this current level of technology you'd almost need to supercool a room for a sweep, blast various RF/sound/etc and look for anything that even remotely warms up, to detect (1) whether something detected something worth recording for later as well as (2) something that might be on the right wavelength/etc to trigger a send.
[0] - https://en.wikipedia.org/wiki/The_Thing_(listening_device)
I was making a joke, but it does look like they got their components from Seeed Studio based on photos.
https://assets.sfstandard.com/image/994911177489/image_k101c...
https://www.seeedstudio.com/XIAO-ESP32S3-Sense-p-5639.html
It looks like it’s the (SeeedStudio XIAO ESP32S3 Sense w/ WiFi, BLE, OV2640, Mic) https://ca.robotshop.com/products/seeedstudio-xiao-esp32s3-s...
Which is just an esp32s3 based device. I assume the word “sophisticated” in openAI report is just copy paste from device description.
Sounds like we know where their priorities lie, anyway.
...uh what
https://m.youtube.com/watch?v=1reman2waLs
> “For us it is a matter of national security,” U.S. Attorney Craig Missakian said in a statement. [...]
> The case is being prosecuted by the National Security, Cyber & Special Prosecutions Section of the U.S. attorney’s office for the Northern District of California.
(But that was after OpenAI "referred the matter to the FBI")
There's presumably a strong argument for making this stuff known internally, since you want other employees to a) be on the lookout for it and b) get scared into NOT doing it themselves because someone else got caught.
If you tell your employees, the outside world is going to hear about it too.
(Or maybe you stall them on the phone while scribbling a note to your PA saying “BUG STORY LEAKED TO VALLEYWAG PUT OUT COUNTER PR GET AGEAD OF STORY” hoping to get a tweet out before the ValleyWag gumshoe gets off the phone.)
(Or maybe you keep a notion.so page pre-filled with all your secrets and each one has a BURN button next to it, ready for you to tap when ValleyWag call up for a comment on their upcoming exposé in re: PR/Disasters/20206090500039.md)
This isn’t uncommon, but it’s rare this info is shared in such detail publicly..possibly to make an example.
Concealing a recording device in a conference room in toilet paper is a bit strange. Do executives meet in the toilets to discuss strategic moves?
Either way, there goes the false flag and the reason for companies to stop hiring humans over this espionage.