There's a joke about certain princes here but I just feel bad for the people getting scammed since I know firsthand how realistic these things can seem
This paper is the one and only source for every claim from the past 10 years that "scammers make mistakes in scams on purpose." Crucially, no actual scammer or former scammer or even acquaintance of a former scammer has ever corroborated it. The paper itself is pure speculation.
If anything I think that paper serves best as a reminder that just because an idea is clever and logical, that doesn't necessarily mean it's true.[1]
No. The paper is about reducing a set, as far as I understand. It's not about mistakes. It's about the deliberacy of basically squeezing out the individuals who are vulnerable and who do not see the signs of being scammed.
Yes but ascribing those motivations is a guess. There's no actual scammer saying they are doing that vs. it just being a fairly unsophisticated scam. It might be the reason, but the paper provides no actual proof, and as far as I've been able to find, neither has anyone else in the decade since. Just a whole lot of people repeating the theory as fact.
I used to bait 419 scammers. Weirdly enough I actually befriended a couple of them. That was about 20 years ago. They were full on and shameless but when exposed they became more human and even sympathetic and I dropped my guard and told them what I was doing.
One of them said, and I'll never forget, "Thank you for being reasonable".
> By sending an email that repels all but the most gullible the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.
There's less need to identify the most gullible when AI drops the incremental cost of responding closer to zero.
I had a bash with that paper. It is pure and utter tosh - I started with the ... start and swiftly moved to 6. Conclusion.
"Thinking like an attacker is a skill rightly valued among defenders."
and
"Attacking as a business must identify targets, and this is easy only if we believe that attackers have solved a problem that has vexed multiple communities for decades."
How so? For me the point of the paper is the elimination of the ones who will not fall for such a scam in the first place. The title of the paper says it all.
When the majority of a decades-long-scams' victims are Boomers getting hacked with AIs (clearing out your possibly-lesser inheritances)... does it still matter if the /u/usernametake29s of this world are still stuck thinking LLMs are "a scam" ? Still?
----
This thought-provoker is 100% human-derived.
#They'reMadeOutOfWeights
----
I didn't even realize how serious I was being, back in late 2022, when I was telling my Boomer friends (twice my age, but still closer than most "family") "make sure if you're grand-daugther calls 'begging for her life,' you need to ask the kidnappers something only she would know – and then *you can never ask that same question again (for verification, at least; ask whatever you want).
If someone who is as unpopular as Elon the cringelord Musk can convince the markets that Tesla is worth its current stock price, I doubt the AI bubble will burst.
It's already burst in South Korea when China showed they can make memory chips too. And it would have burst by now in the US if OpenAI and Anthropic were public.
It is interesting looking at a long term chart how obvious that it was due for correction. Having said that, it's easy to say that after the correction because 'the top' is impossible to predict.
But if investors hadn't been taking profits on the incline, then I have little sympathy for them skiing down the decline.
(This is a harsh opinion, but people getting into investing should be doing it actively, not passively, ie. do some research, make a plan. Also, for those who are invested passively, long term, via ETFs, then they're still up 55% as @Nition says, so they're all good)
Tesla is imo overvalued and has been for a long time. Its bankrupted a lot of pretty savvy investors in the last 10 years trying to short them because its obviously over valued. Market irrationality vs solvency and all of that.
While I agree with you in general, usernametaken29 made a specific prediction about OpenAI's IPO which is an event that will very likely happen in the next year or so.
That said, OpenAI isn't public, so you can't directly short it easily. I guess you could short Microsoft but I don't know if that gives you enough exposure to truly be worth it, even if you do genuinely believe what OOP claims.
While he was still living on his own, my grandfather struggled with people sending him scam letters in the mail. He sent a meaningful amount of money to obvious scams despite being a shrewd businessman all his life. AI will take these cons to another level.
Is there any way to help the elderly navigate this shift?
Easier said than done.
One of most popular scam here is a scammer using deepfake/ai calling victim saying there was serious accident to one of victim's relative and they need money for hospital (that's how it works here, money for proper caring). The victim would be panic and send the money.
Usually, the scammer will think of a scenario that give the victim very limited time to stop and think, or put them in the state that fear takes over their mind
It sounds extremely obvious and infinitely practical.
How often are you (or your parents) legitimately asked for money via email / internet? I mean, my answer is: never. My parents' answer is: never. My parents-in-law answer is: never.
That was mostly Bayesian probability filters, followed by SPF and more recently DKIM.. Maybe you could consider the Bayesian filter databases being trained as ML, but I feel it's a far way from our typical AI/LLM surge
The main fuel is the "need" to have browsers that can execute code, coupled with the ability to transfer money via the internet. JavaScript was a mistake, e-commerce was a mistake, and so is everything else that came after.
Let's ask Vint Cerf whether the web was meant to run logic, or just be a document format. I bet no matter what he answers, the world would be like the GIF pronunciation "I don't care if the original creator said it's pronounced like giraffe, I'm gonna say it like gift because I KnOw BeTtEr"
Internet 3 when (because I know that academia does already have an "Internet 2", although I bet due to Internet 1 security issues, Internet 2 is already compromised too)
Douglas Adams put the source of the rot a bit further back:
> This planet has, or had, a problem, which was this. Most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movements of small, green pieces of paper, which is odd, because on the whole, it wasn't the small, green pieces of paper which were unhappy. And so the problem remained, and lots of the people were mean, and most of them were miserable, even the ones with digital watches. Many were increasingly of the opinion that they'd all made a big mistake coming down from the trees in the first place, and some said that even the trees had been a bad move, and that no-one should ever have left the oceans.
Criminals are bad, evil, jail etc etc. I agree.
But while the core problem that creates this environment is not solved, nothing will fix it. Not even if you bomb the hell out of it.
Ethics, morals and whatever people call only exists if economy is stable. If you let this sort of opportunistic environment exist there will be people to work in it
Same for global migrants issues..as long as people in dirt poor/war torn/unstable countries can watch tik toks of fat people dancing and crying about first world problems while they starve on the streets or deal with gangs and infrastructure issues we will have migrant problem.. we live in a hyperconnected world these days
Viewing the world as haves and have nots and building giant Walls and moats and hyper-capitalism alone will never solve this..
> Ethics, morals and whatever people call only exists if economy is stable.
What is a "stable" economy? The US has the world's largest GDP, the highest nominal GDP per capita of any nation with a population over 10 million, the highest amount of investable and liquid private assets of any country in the world, more than a third of the world's millionaires, etc.
And yet I think you'd have a hard time convincing most people that the US is a bastion of ethics and morals.
Rich countries have created a definition that makes it a poor country thing, conveniently ignoring the type of corruption that is institutionalized (and in some cases even legalized) in many rich countries.
The point of institutionalizing corruption is to reduce competition from competing corruption entities. Which is bad but not as bad as an endemically corrupt society.
I think it can be true that we have in effect far more corruption in the west than what is readily recognized at the same time that it’s true that we have less corruption than most other countries.
> The point of institutionalizing corruption is to reduce competition from competing corruption entities. Which is bad but not as bad as an endemically corrupt society.
Do you even read what you write? If you institutionalize corruption to reduce competition between different corrupt factions, you end up with a not endemically corrupt society?
Here's what it actually means: if you institutionalize corruption, you end up with an endemically corrupt society that is incapable of even recognizing corruption for what it is.
> ...at the same time that it’s true that we have less corruption than most other countries.
No, the corruption is just more sophisticated so that we can pretend it isn't corruption. Just look at campaign finance. In many ways, it's even more insidious than companies handing over paper bags full of bribe money to politicians and government officials.
The amounts are bigger, the number of people who get a piece of the action is great and it's all "legal" so the corruption is more durable. And, again, everyone gets to pretend that it's not corruption.
A society that pretends it is not corrupt is better than one that has given up on that pretense. No matter how bad you think things are it can always get worse. I am not looking forward to the coming free for all.
I think what the Trump admin is doing is horribly corrupt and I hope the lot of them go to jail for that and other things, but in relative terms what Trump is doing is still less corrupt than what happens in many poor countries.
> ...but in relative terms what Trump is doing is still less corrupt than what happens in many poor countries.
In relative terms?
So what you're saying is that Trump and his family grifting Americans for a couple billion dollars so far in his second term is better than, say, Hun Sen and his family grifting Cambodians for somewhere between $500 million and $4 billion over ~40 years because the US economy is so much bigger than Cambodia's?
I would measure by relative destruction of wealth since corruption is an inefficient destructive process that can quite often yield very little left to steal.
Having run a small social media SaaS for a while, the amount of AI bots (and I say AI bots because the comments and direct messages sent by them definitely read like AI), was staggering.
I'm thankful for Cloudflare and I don't think I would have had the resources to deal with it any other way.
Cybercrime is helped by the massive amount of static identifiers big tech and the internet uses and their refusal to shift away from them.
Privacy focused solutions prevent the leaks which enable many scammers, and yet governments respond by demanding more identifiers be added everywhere, which perpetuates the leaks. I've not gotten a single spam message on Signal with their phone number privacy settings and username usage. You can change your username whenever you want.
Google should add email aliasing and encourage it. They could detect and alert users if an email that comes to an alias is from a different sender than the first email to that alias.
Got a scam call yesterday morning. An American-ish accent, but quite rusty, I suspected they were using an AI voice changer.
They said "Am I speaking to <my full name>?". I did not answer that and instead asked what company they were calling from, then they immediately hung up.
I guess people who ask that initial question are disqualified for their scam, they're looking for more gullible folks.
As if people weren't scamming before AI, with the billions of dollars seized from the Chinese scam dungeons in Cambodia [0] where they had kidnapped people and tied them to computers (including a few deaths, and another wave of arrests just recently)
Scammers gonna scam. At least if they can use AI for it then maybe they'll kidnap fewer people
I'm not sure that's a very comforting comparison: It's not clear that guns have that level of deterrent effect, and a world where everyone is shooting ar everyone else with guns is supremely wasteful.
Better to strike at the root of the LLM-scam business model wherever we can.
It's great to know that AI can be used for scams. Meanwhile, here I am paying for AI, and it accusing me of being either a pedophile or child abuser by asking it to change an image of my six-year-old into a "spy kid" like he requested.
So deepfakes and “AI was also used to generate synthetic identities that bypassed KYC checks, craft hyper-personalized phishing lures tailored to individual victims, and produce AI-driven malware capable of evading signature-based detection.”
Our most vulnerable must be getting absolutely hammered by scams. Is anyone working on protecting e.g. elderly people from social engineering? I feel like banks should be proactively suggesting people have a second family member or fiduciary sign off on risky transactions like wiring money to new accounts or pulling out a ton of cash.
I was recently on the phone with a very articulate member of my banks fraud department, or at least I believed they were up until they asked for my two factor code to reset my account. They had visibility into some past legitimate wire transfers which they asked me about along with a fictitious fraudulent one that they “blocked“, and when I pushed back against giving them my two factor code, they had a whole script prepared about needing to freeze my account until I went to a branch with four forms of identification (or just give us the two factor code and avoid the hassle). An acquaintance wired their down payment to a scammer after the real estate firms email system was compromised and scammer inserted their own wire instructions.
Not to mention reports of mythos directly hacking into banking systems. Scary times.
Dunno first time I have ever been on that domain, I think the direct link to the PDF gets you blacklisted, I can navigate the rest of the site just fine.
The PDF is linked here, which you can get to this article from the front page:
> You know about the values of a culture, if they call your old parents to rip them off.
Large groups of people in a poor country steal from rich countries, and your explanation is a morally corrupt culture? not the obviously extremely poor prospects of it's citizens?
> They have none, or worse they can talk themselves into this "Robin Hood" psychosis, where stealing the pension from the elderly is just fine, because they are part of the upper crust of the world.
You're describing a cartoon character. Most of these people are trying to survive, not make some kind of twisted moral stand. Please try to recognise the privilege your place of birth affords you and how much of a global minority you are in. Incentives are everything here.
So not only AI is consuming more energy than than the entire bitcoin network is [0], it is also accounts for more than half of cybercrimes and digital scams in Africa.
I don't think you can even dispute this. AI will be the one to cause the next financial crisis.
>AI will be the one to cause the next financial crisis.
So what's the implication here? It sounds like you want AI canceled or curtailed, and if you apply the same logic to the internet in the mid 90s, you would have correctly called out the dotcom bubble. Does that we mean we should have also canceled the internet?
Late 90s/early 00s internet would've probably been the sweet spot for utility while causing little harm. Sadly we can't just freeze the internet in time.
SWIM got scammed on poshmark a few days ago, as they were selling something they received an email from ethiopia and didn’t pay too much attention and lost $3k
There's a joke about certain princes here but I just feel bad for the people getting scammed since I know firsthand how realistic these things can seem
https://www.microsoft.com/en-us/research/wp-content/uploads/...
Edit: Recalled -> Recall
If anything I think that paper serves best as a reminder that just because an idea is clever and logical, that doesn't necessarily mean it's true.[1]
[1] As another example, see https://en.wikipedia.org/wiki/Nuclear_Gandhi
One of them said, and I'll never forget, "Thank you for being reasonable".
There's less need to identify the most gullible when AI drops the incremental cost of responding closer to zero.
"Thinking like an attacker is a skill rightly valued among defenders."
and
"Attacking as a business must identify targets, and this is easy only if we believe that attackers have solved a problem that has vexed multiple communities for decades."
Bollocks!
Are we talking about the same thing?
----
This thought-provoker is 100% human-derived.
#They'reMadeOutOfWeights
----
I didn't even realize how serious I was being, back in late 2022, when I was telling my Boomer friends (twice my age, but still closer than most "family") "make sure if you're grand-daugther calls 'begging for her life,' you need to ask the kidnappers something only she would know – and then *you can never ask that same question again (for verification, at least; ask whatever you want).
We are here.
https://en.wikipedia.org/wiki/Greater_fool_theory
Ah with a decade or two time horizon it shouldn’t matter how hard it crashes if Zitron & Doctorow are proven right… rightttt!
But if investors hadn't been taking profits on the incline, then I have little sympathy for them skiing down the decline.
(This is a harsh opinion, but people getting into investing should be doing it actively, not passively, ie. do some research, make a plan. Also, for those who are invested passively, long term, via ETFs, then they're still up 55% as @Nition says, so they're all good)
That said, OpenAI isn't public, so you can't directly short it easily. I guess you could short Microsoft but I don't know if that gives you enough exposure to truly be worth it, even if you do genuinely believe what OOP claims.
Not guaranteed, but the norm.
I've never tried to buy PUTs on a big IPO but I wouldn't be surprised that there's a premium on them, if options trading is even available.
Is there any way to help the elderly navigate this shift?
"Ignore anyone asking you for money unless you know them and they do it in-person."
Usually, the scammer will think of a scenario that give the victim very limited time to stop and think, or put them in the state that fear takes over their mind
It sounds extremely obvious and infinitely practical.
How often are you (or your parents) legitimately asked for money via email / internet? I mean, my answer is: never. My parents' answer is: never. My parents-in-law answer is: never.
AI definitely makes the scams more believable, a Nigerian Prince can now easily forge documents.
But at the same time AI is a double edged sword, it can be used for both offence and defense.
You don't exactly want convincing evidence; you want a good story.
Let's ask Vint Cerf whether the web was meant to run logic, or just be a document format. I bet no matter what he answers, the world would be like the GIF pronunciation "I don't care if the original creator said it's pronounced like giraffe, I'm gonna say it like gift because I KnOw BeTtEr"
Internet 3 when (because I know that academia does already have an "Internet 2", although I bet due to Internet 1 security issues, Internet 2 is already compromised too)
That’s a really extreme take that lacks any nuance.
Might as well blame the OS as well for allowing any custom code to be executed.
> This planet has, or had, a problem, which was this. Most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movements of small, green pieces of paper, which is odd, because on the whole, it wasn't the small, green pieces of paper which were unhappy. And so the problem remained, and lots of the people were mean, and most of them were miserable, even the ones with digital watches. Many were increasingly of the opinion that they'd all made a big mistake coming down from the trees in the first place, and some said that even the trees had been a bad move, and that no-one should ever have left the oceans.
Tim invented the web: HTML, HTTP, URLs.
Vint did a lot of things, but most of it had to do with lower-level internet infrastructure, not the web per se.
Here is a story which describes one such letter from 1913: https://web.archive.org/web/20020815055745/http://www.samizd...
Ethics, morals and whatever people call only exists if economy is stable. If you let this sort of opportunistic environment exist there will be people to work in it
Viewing the world as haves and have nots and building giant Walls and moats and hyper-capitalism alone will never solve this..
What is a "stable" economy? The US has the world's largest GDP, the highest nominal GDP per capita of any nation with a population over 10 million, the highest amount of investable and liquid private assets of any country in the world, more than a third of the world's millionaires, etc.
And yet I think you'd have a hard time convincing most people that the US is a bastion of ethics and morals.
It is true that we see correlation between richer countries and lower level of crimes - across all layers of society.
- tamper with index fund rules so that the 401k's of the masses are bearing the risk of your risky investment
- bribe a border agent to let you through to see a friend
Rich countries have created a definition that makes it a poor country thing, conveniently ignoring the type of corruption that is institutionalized (and in some cases even legalized) in many rich countries.
I think it can be true that we have in effect far more corruption in the west than what is readily recognized at the same time that it’s true that we have less corruption than most other countries.
Do you even read what you write? If you institutionalize corruption to reduce competition between different corrupt factions, you end up with a not endemically corrupt society?
Here's what it actually means: if you institutionalize corruption, you end up with an endemically corrupt society that is incapable of even recognizing corruption for what it is.
> ...at the same time that it’s true that we have less corruption than most other countries.
No, the corruption is just more sophisticated so that we can pretend it isn't corruption. Just look at campaign finance. In many ways, it's even more insidious than companies handing over paper bags full of bribe money to politicians and government officials.
The amounts are bigger, the number of people who get a piece of the action is great and it's all "legal" so the corruption is more durable. And, again, everyone gets to pretend that it's not corruption.
In relative terms?
So what you're saying is that Trump and his family grifting Americans for a couple billion dollars so far in his second term is better than, say, Hun Sen and his family grifting Cambodians for somewhere between $500 million and $4 billion over ~40 years because the US economy is so much bigger than Cambodia's?
I'm thankful for Cloudflare and I don't think I would have had the resources to deal with it any other way.
Privacy focused solutions prevent the leaks which enable many scammers, and yet governments respond by demanding more identifiers be added everywhere, which perpetuates the leaks. I've not gotten a single spam message on Signal with their phone number privacy settings and username usage. You can change your username whenever you want.
Google should add email aliasing and encourage it. They could detect and alert users if an email that comes to an alias is from a different sender than the first email to that alias.
They said "Am I speaking to <my full name>?". I did not answer that and instead asked what company they were calling from, then they immediately hung up.
I guess people who ask that initial question are disqualified for their scam, they're looking for more gullible folks.
As if people weren't scamming before AI, with the billions of dollars seized from the Chinese scam dungeons in Cambodia [0] where they had kidnapped people and tied them to computers (including a few deaths, and another wave of arrests just recently)
Scammers gonna scam. At least if they can use AI for it then maybe they'll kidnap fewer people
[0] https://news.ycombinator.com/item?id=45580981
Chinese models are going to keep getting stronger. Maybe the Free World™ should increase their people's access to and education about AI too.
Better to strike at the root of the LLM-scam business model wherever we can.
Why would the scammer go thru your detector?
Phone? The network could spot sources exhibiting robocall patterns. AI could analyze live conversations and warn the recipient on certain patterns.
Instant messaging? There's spam and abuse screening built into most IMs already. AI could [locally/privately] analyze conversations.
Email? Ditto; analyze conversations, a bit deeper than basic junk mail filters already do.
Unless you ban mentioning other IM platform or communication channel, they will always try to get you on something less monitored
Page 24 is where the AI stuff starts being described in detail.
Our most vulnerable must be getting absolutely hammered by scams. Is anyone working on protecting e.g. elderly people from social engineering? I feel like banks should be proactively suggesting people have a second family member or fiduciary sign off on risky transactions like wiring money to new accounts or pulling out a ton of cash.
I was recently on the phone with a very articulate member of my banks fraud department, or at least I believed they were up until they asked for my two factor code to reset my account. They had visibility into some past legitimate wire transfers which they asked me about along with a fictitious fraudulent one that they “blocked“, and when I pushed back against giving them my two factor code, they had a whole script prepared about needing to freeze my account until I went to a branch with four forms of identification (or just give us the two factor code and avoid the hassle). An acquaintance wired their down payment to a scammer after the real estate firms email system was compromised and scammer inserted their own wire instructions.
Not to mention reports of mythos directly hacking into banking systems. Scary times.
Is what I see. Which makes sense I guess. Maybe just tell me how you got to the page?
The PDF is linked here, which you can get to this article from the front page:
https://www.interpol.int/News-and-Events/News/2026/INTERPOL-...
Large groups of people in a poor country steal from rich countries, and your explanation is a morally corrupt culture? not the obviously extremely poor prospects of it's citizens?
> They have none, or worse they can talk themselves into this "Robin Hood" psychosis, where stealing the pension from the elderly is just fine, because they are part of the upper crust of the world.
You're describing a cartoon character. Most of these people are trying to survive, not make some kind of twisted moral stand. Please try to recognise the privilege your place of birth affords you and how much of a global minority you are in. Incentives are everything here.
It's nearly as if there was a lesson here about some individuals in the human race?
What was that yet about psychos? That 1% of all humans, no matter the socio-economic class, no matter the parents, are just full-on psychopaths.
The question is: what do we do, as a society, about those who only want to harm others?
I don't think you can even dispute this. AI will be the one to cause the next financial crisis.
[0] https://www.cell.com/joule/abstract/S2542-4351(25)00142-4
So what's the implication here? It sounds like you want AI canceled or curtailed, and if you apply the same logic to the internet in the mid 90s, you would have correctly called out the dotcom bubble. Does that we mean we should have also canceled the internet?
You should also be careful about taking imagined positions to the extreme ..."Does that we mean we should have also canceled the internet?"
It does, otherwise what are you arguing for, that financial crises are bad?
>You should also be careful about taking imagined positions to the extreme ..."Does that we mean we should have also canceled the internet?"
Right, which is why I posed it as a question.
In retrospect, I think the world would be a much better place if we'd just turned it off in 1998.