how does something like this pass code and design reviews. is this code written by ai (the buggy code). Not only was the key that signs the cookie inside the cookie it self but it's using pickle which even model weights are moved away from. I'm a little flabbergasted at the noob nature of this root cause.
Do you have any advice for aspiring bug-bounty hunters? Do you have some method for being able to find vulnerabilities like this, or lots of past experience?
> Do you have any advice for aspiring bug-bounty hunters?
We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
most people who care about legality don't work in that field either. whoever gets into this probably starts by being ok with doing shady stuff, I imagine.
Many start in the military and eventually move to private sector. However, an ever growing percentage just straight up go to college to learn the skills.
Cyber companies are in the fortune 100 now, and it's not like it used to be back when you actually had to be interested in it and dedicate your life to learning the skillz from shady forums. Now it's just a job to most practitioners.
there are enough people operating in countries that would not charge them for hacking an American company for it to matter, $90k can be a life-changing sum
Sorry for the meta, but the discourse in many of the comments is nuts to me.
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
If we're doing this standard, then let's do it for everything. Claude coded that, not you. Claude built that website, not you. Might make the industry less exhausting tbh.
Yeah some people come off as envious they didn't have Claude try it first.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
Nmap doesn't write its own blog posts. That's not where I draw the line, but the point isn't at all about "drawing the line." (In other words, don't get caught up in where the line is. There is one, but we're not sitting in some gray area, we've blown past it by hundreds of kilometers by now.)
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
People who believe AI is an abstraction layer on top of compilers, are in that middle ground where they know computer science but are far from an expert. Which makes them very susceptible to the Dunning–Kruger effect. Throw in AI sycophantcy and you've got yourself takes like the one above.
If this were Microsoft, they would have received a coupon for 10% off a Thanksgiving turkey, only redeemable in 2030 and only in the month of March, must be purchased with cash, non-transferrable.
It's hard to get a $90k bounty from Google, even with Claude in hand, so we shouldn't be dismissive of that, but it is pretty annoying to read write-ups written entirely in LLM voice.
An llm is a tool, I had access to Claude too but I didn’t find the bug. The author found it.
Research taste, among other things are still abilities that differentiate among humans.
In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.
The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’
I bet, I could take "Trading places" style, anyone off the street, give them a 100 "Secret" word list and they could spend 6 months discovering all sorts of things.
I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.
handing clients access to an LLM gets them beginner level code, easily. same with beginner level exploits. an RCE against google is not a "beginner level exploit". If you've seen AI generated code, I can tell you that AI generated security findings also have that "beginner" smell. an expert who knows what they're doing has to drive them to look at the right places, and use the experts preferred method.
Took about 3 sentences for me to be as certain as I've ever been about anything I've ever been in my entire life that this blog post was authored by an LLM, though truthfully it's impossible to know for sure if it was a Claude model or a distill.
We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
Is this really true? After all, reselling the exploit would be illegal, and most people don't want to break the law to earn income.
Cyber companies are in the fortune 100 now, and it's not like it used to be back when you actually had to be interested in it and dedicate your life to learning the skillz from shady forums. Now it's just a job to most practitioners.
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
I suspect a lot of things fall into the latter. It is a common neg to defer everything to the former though.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
It is fair an interesting to disclose tools used, but I am with GP in leaving agency with humans, at least for now.
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
Research taste, among other things are still abilities that differentiate among humans.
In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.
The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’
I do have to send input to the thing to get a response, no?
No thank you says pretty much everyone.
I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.
Same old tired complaining about people not using technology correctly, or the way a _real_ coder would do it.
I also am wondering if Claude will proceed on a prompt which is about finding a bug like that? I am not sure, has anyone tried?
The cookie contained a pickle-serialized payload, "which was bad enough on its own." maybe not best practice but not bad on it's own
Flask session cookies are signed", not encrypted," so the... LLMs love to tell you what things aren't, as an emphasis on what they are
;)