Wow, this is serious. Makes you think, that even though QubesOS attack surface is so tiny (well-designed to be secure) there are still vulnerabilities to be found.
Worth noting that (as I understand) this vulnerability occurs only when doing copy-to-VM from Dom0:
>Note that the VM variant of `qvm-copy-to-vm` is not affected, as its
version of the error reporting function does not use `system()`:
Since you should not use Dom0 for regular work, and definitely not for interacting with likely-to-be-infected VMs, the scope of this attack is smaller than it sounds. On the flip side, when it works, it elevates privileges straight to Dom0.
While I suppose this doesn't say the design isn't secure, system() is one of those calls that has no place in modern code. It is insecure by its design.
> Any user input that is employed as part of `command` should be carefully sanitized, to ensure that unexpected shell commands or command options are not executed. Such risks are especially grave when using system() from a privileged program.
The "system" function (and also "popen" function) are helpful, although they should be used carefully. You should only use it where the entire input (rather than merely a part of it) comes from a trusted source from the local user (and documented in a clear way that it does this), such as being entered interactively or from a user configuration file, and to ensure that the security boundary is correct (e.g. you should probably avoid it if it is setuid). (If it is necessary to pass additional data then you might use environment variables, pipes, temporary files, etc.)
You are right, copying to dom0 is not best practices and warned against since anno dazumal, but given the user groups I remember not always being technically minded (journalists, dissidents, etc.) and ensuring qubeses isolation holds even when users do things they are discouraged from has always been part of the philosophy. Don’t trust users, don’t trust userland, don’t trust software and all that yazz.
I believe this is a vulnerability that occurs when copying data from dom0, which is a more common task. Generally the Qubes model recommends copying data from more trusted VMs to less trusted VMs, and dom0 still runs some system-wide processes in many default configurations.
For example when you take a screenshot with xfce4-screenshooter, the file is saved to dom0, and you have to use qvm-copy-to-vm to move it to a (less trusted) qube to do something with it. That's the most frequent use case, at least for me.
Theos is a very opinionated and not necessarily wrong position, but I feel also a bit too reductive given we are eternally having to deal with compromises of some form. Also, lest we forget, it has been two decades in the interim and oh so much has changed. In any case, this originated from their code, not virtualization, so it doesn’t really apply either way…
I think it's pretty much spot on myself and applies to more than virtualization based on the last point. It really suggests that further complexity and abstraction is not a good security posture. And I agree with this from extensive experience (embedded, defence).
Regarding the two decades since and the numerous exploitable x86-64 and hypervisor bugs suggests he wasn't wrong and that the tone was appropriate for the severity of the problem.
Sadly complexity is unavoidable for most real-world usecases. Just see how so many people interested in de-Googled phones balk at the prospect of losing access to banking apps. Telling them to go use an OS that doesn't support Bluetooth isn't going to work, but hopefully you wouldn't say that means they should give up all hope of security.
Abstraction has served us well in managing complexity before. I wouldn't abandon it out of misplaced idealism.
> You are absolutely deluded, if not stupid, if you think that a
worldwide collection of software engineers who can't write operating
systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.
This comment wouldn't survive HN scrutiny. It's a strawman argument, and doesn't address the core point at all: does virtualization improve or degrade security, when taken as a whole?
Adam's response is great, and TDR is smart, but he's been a lightning rod for 20 years for a reason: he was doing hot takes before they were even called that, and this is a great example.
Yeah, I'm with Theo on this one. Conventional OS security between Ring-0 and everything else is well understood; the problem has become too much code in Ring-0, a great fraction of which has its own interfaces across the security boundary, and the Unix security model just doesn't scale.
No capabilities, or even a sane and useful way of adding capabilities with everything in ring 0, and the flat integer namespacing of users and groups just doesn't work for what userspace needs to do today - hence namespaces, which have introduced their own problems, because (no surprise) trying to graft a tree structure onto a flat integer namespace after the fact is a mess.
Virtualization tried to sidestep all that, but to make it fast the cost has been more driver interfaces to host ring-0 - remember what the original was? - and screwing around a whole bunch with particularly arcane facets of the core ring-0 security boundary, e.g. page tables.
The bug here is not related to virtualization, but a footgun as old as C stdlib: system() that doesn't take arguments separately, and instead relies on shell escaping by the application.
Being right is not the most important part. If you're right but don't convince anyone, you've made no difference.
Theo was right that virtualization is a comparatively shoddy security boundary. At the same time, it's flexible and capable in ways that now define the shape of modern IT.
Could we have replicated that by other means? If yes, then it's on Theo and other knee-jerk critics that they never proposed a better approach and settled for insulting people. If not, then maybe virtualization was a necessary evil. Or maybe everyone else is an irredeemable idiot, but again - if we reach that conclusion, is the world better off?
I did not claim it was most important, but he clearly called out issues that haunt us still, because we did not fix the basic initial issues ( and we are actively making it worse now with AI slop -- we because, while I use llms, I am not arrogant enough to offer my contributions to emulation, virtualization and other foundational stuff ). The point is that he called it and all of us should do some soul searching as to why his words were ignored. Because if he was ignored because 'he was mean about it', the we are kinda screwed long, because anyone with a brain that can comprehend the issue, will likely have little patience for anyone, who can't see it or won't want see to see it for some contrived semi-social reason.
They aren't, they're one of the people who latch on longstanding and neutral syntax (em-dashes, "it's not $foo") as a proof of LLM text. I don't blame them because HN has a lot of people trying to pass gen AI stuff as their own, and you need quick heuristics... but I'd encourage people at least do it right. Pay for Pangram or something.
"HN: Professionally identifying 230% of posts from 2017 as LLM generated".
That said Panagram is trash for the opposite reason. Not that some people talked like LLMs before LLMs, but now a lot of people talk like LLMs because of LLMs.
I don't really know as the argument is mainly about how stupid people are... The technical argument is one paragraph ended with an insult, not much to make an educated and civilized opinion.
<< You are absolutely deluded, if not stupid, if you think that a
worldwide collection of software engineers who can't write operating
systems or applications without security holes, can then turn around
and suddenly write virtualization layers without security holes.
Allow me to start by saying that you are wrong about 'mainly about'. The argument starts with an insult, but insult is minimal and it moves directly into the technical details. But, note that how much you are wrong about the paragraph, because what you want to focus is the insult. You are so wrong about it you effectively invert the proportions of presence of technical data to insult..
But is the world a better place with that particular understanding being the norm. Would we, as a whole, be better served if function was what actually mattered? Wouldn't that be nice?
I hope when people read this though they understand this is a communication style; they're clearly trying to strongly discourage people from thinking they are suddenly protected. Effective? Maybe at one time, where "macho dev energy" was a thing. Today, not so much. You can tell they mean well because the intro sentence is actually pretty cheeky!
This is less of a virt/x86 bug and more of a "don't call system() on arbitrary user input" bug.
.. incidentally, OpenBSD also provides one of the clearest examples of how the excuse "calling system() is fine in my case, its totally not arbitrary user input" is deluded just the same, see CVE-2020-8794.
It has? News to me. Go on any major thread on this page, you’ll witness similarly strong pushback visa-vi buying into corporate backed hype, akin to the overconfidence in virt security he pointed at back then.
The founder Joanna Rutkowska left QubesOS in 2018. All the code involved in this bug was committed by her successor Marek Marczykowski-Górecki.
Joanna seems to be a genuine good guy, she once wrote a paper titled "Intel x86 considered harmful". That's why Huawei and the Chinese government aren't even trying any more to make western CPU architectures secure, it's a hopeless cause.
I am still impressed by QubesOS track and I use it for my dedicated 'financials' laptop.
IMHO the thing that is holding back QubesOS is the lack of hardware acceleration for graphics - maybe now when dual monitor setups are getting popular this could be a workaround for the security considerations?
I dropped QubesOS once exactly for that reason in the past but now I’m running it again on a separate computer.
Even with all its drawbacks, there is something really nice about being able to run different applications over Tor, VPN or plain internet simultaneously, the ability to isolate non-safe binaries and being able to backup your VMs easily.
I wish a similar distro would be made based on KVM so that the standard kernel could be used. It would be great for compatibility.
> being able to run different applications over Tor, VPN or plain internet simultaneously, the ability to isolate non-safe binaries and being able to backup your VMs easily.
These are all possible using light containers. For example, on FreeBSD I will spin up a jail which runs wireguard, and then I'll bridge that to another a jail. That 2nd jail is running entirely off wireguard without any other way to access the network. Since it is a jail, it is isolated. And backing up is as simple as a zfs snapshot and zfs send. I assume the same is possible on Linux.
Problem is not that nobody wants to do it, it is that GPU stack is exactly the kind of enormous driver surface Qubes exists to keep away from dom0. Second monitor does not really change that, somebody still has to trust the driver.
Mini tangent: Could someone explain to me why Qubes is used for security, when (from what I understand) Jails on BSD is significantly more robust/safe/has a much smaller exposed area? Is it just "everyone's using linux already; here's a safer linux"?
Qubes can be viewed as a Xen distribution, rather than a Linux distribution [1]. You may find the Qubes FAQ a good starting point (I'm reading it now because of your question, so thanks).
The worst part is that in 2020 they explicitly documented that the remote filename is attacker controlled,but still allowed it to reach system()
That is C security 101: never pass untrusted input through a shell. This should have been caught in review!
That is sphincter tightening to read. Have to point out how amazingly well their bulletins handle communication. Clearly describes the issues, how users are to act, etc. in, what I feel, is an easy to grasp language, even if one’s not in the weeds that much. In fairness though, I do still have some past memories concerning Qubes architecture from way back, so maybe my assessment is wrong and this is still not that straight forward to grasp for most.
> Important: At this point, you still don’t know whether the key you just imported is the genuine QMSK or a forgery. In order for this entire procedure to provide meaningful security benefits, you must authenticate the QMSK out-of-band. Do not skip this step! The standard method is to obtain the QMSK fingerprint from multiple independent sources in several different ways and check to see whether they match the key you just imported. For more information, see How to import and authenticate the Qubes Master Signing Key.
It looks like Qubes is ran by people who take security seriously, which is refreshing.
I own a Qubes T-shirt which I bought in person at FOSDEM. The design on the T-shirt consists of many copies of the QMSK in hexadecimal. All of their merch is like this.
The worst part is that in 2020 they explicitly documented that the remote filename is attacker controlled,but still allowed it to reach system()
This is C security 101: never pass untrusted input through a shell. This should have been caught in the review!
Isn't it the case for all bugs? If they appear in the production software, it means that they passed the review. And obviously bugs shouldn't pass the review, but that's easier said than done.
Another example for why system() is so dangerous to use.
I also don't understand why it needs to show the dialog in dom0. If you have the option to handle attacker controlled input on the unprivileged side, you should do that instead of putting a lot of logic on the privileged side.
The code is sloppy. They check existance of kdialog binary using full path; next step they rely on PATH search by shell. If would've been much safer to just do execve directly.
If you're going to put the graphics and NIC into separate VMs, surely the secure screen can be another of those semi-privileged VMs rather than part of dom0
Peculiar stuff. I'm always skeptical of these security Linux distributions, but this bug is so bad that it seems like an infiltration of Qubes at best or Qubes being a honeypot at worst.
Worth noting that (as I understand) this vulnerability occurs only when doing copy-to-VM from Dom0:
>Note that the VM variant of `qvm-copy-to-vm` is not affected, as its version of the error reporting function does not use `system()`:
Since you should not use Dom0 for regular work, and definitely not for interacting with likely-to-be-infected VMs, the scope of this attack is smaller than it sounds. On the flip side, when it works, it elevates privileges straight to Dom0.
While I suppose this doesn't say the design isn't secure, system() is one of those calls that has no place in modern code. It is insecure by its design.
> Any user input that is employed as part of `command` should be carefully sanitized, to ensure that unexpected shell commands or command options are not executed. Such risks are especially grave when using system() from a privileged program.
(—man 3 system)
For example when you take a screenshot with xfce4-screenshooter, the file is saved to dom0, and you have to use qvm-copy-to-vm to move it to a (less trusted) qube to do something with it. That's the most frequent use case, at least for me.
Theos is a very opinionated and not necessarily wrong position, but I feel also a bit too reductive given we are eternally having to deal with compromises of some form. Also, lest we forget, it has been two decades in the interim and oh so much has changed. In any case, this originated from their code, not virtualization, so it doesn’t really apply either way…
Regarding the two decades since and the numerous exploitable x86-64 and hypervisor bugs suggests he wasn't wrong and that the tone was appropriate for the severity of the problem.
Abstraction has served us well in managing complexity before. I wouldn't abandon it out of misplaced idealism.
TDR said:
> You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.
This comment wouldn't survive HN scrutiny. It's a strawman argument, and doesn't address the core point at all: does virtualization improve or degrade security, when taken as a whole?
Adam's response is great, and TDR is smart, but he's been a lightning rod for 20 years for a reason: he was doing hot takes before they were even called that, and this is a great example.
No capabilities, or even a sane and useful way of adding capabilities with everything in ring 0, and the flat integer namespacing of users and groups just doesn't work for what userspace needs to do today - hence namespaces, which have introduced their own problems, because (no surprise) trying to graft a tree structure onto a flat integer namespace after the fact is a mess.
Virtualization tried to sidestep all that, but to make it fast the cost has been more driver interfaces to host ring-0 - remember what the original was? - and screwing around a whole bunch with particularly arcane facets of the core ring-0 security boundary, e.g. page tables.
It is a mess.
Theo was right that virtualization is a comparatively shoddy security boundary. At the same time, it's flexible and capable in ways that now define the shape of modern IT.
Could we have replicated that by other means? If yes, then it's on Theo and other knee-jerk critics that they never proposed a better approach and settled for insulting people. If not, then maybe virtualization was a necessary evil. Or maybe everyone else is an irredeemable idiot, but again - if we reach that conclusion, is the world better off?
Stares at Go as about the only step in that direction...
"HN: Professionally identifying 230% of posts from 2017 as LLM generated".
That said Panagram is trash for the opposite reason. Not that some people talked like LLMs before LLMs, but now a lot of people talk like LLMs because of LLMs.
Allow me to start by saying that you are wrong about 'mainly about'. The argument starts with an insult, but insult is minimal and it moves directly into the technical details. But, note that how much you are wrong about the paragraph, because what you want to focus is the insult. You are so wrong about it you effectively invert the proportions of presence of technical data to insult..
Especially as more and more virtualization functions move into hardware, not using them as a second security barrier seems foolish.
I hope when people read this though they understand this is a communication style; they're clearly trying to strongly discourage people from thinking they are suddenly protected. Effective? Maybe at one time, where "macho dev energy" was a thing. Today, not so much. You can tell they mean well because the intro sentence is actually pretty cheeky!
.. incidentally, OpenBSD also provides one of the clearest examples of how the excuse "calling system() is fine in my case, its totally not arbitrary user input" is deluded just the same, see CVE-2020-8794.
Joanna seems to be a genuine good guy, she once wrote a paper titled "Intel x86 considered harmful". That's why Huawei and the Chinese government aren't even trying any more to make western CPU architectures secure, it's a hopeless cause.
> she
IMHO the thing that is holding back QubesOS is the lack of hardware acceleration for graphics - maybe now when dual monitor setups are getting popular this could be a workaround for the security considerations?
Even with all its drawbacks, there is something really nice about being able to run different applications over Tor, VPN or plain internet simultaneously, the ability to isolate non-safe binaries and being able to backup your VMs easily.
I wish a similar distro would be made based on KVM so that the standard kernel could be used. It would be great for compatibility.
These are all possible using light containers. For example, on FreeBSD I will spin up a jail which runs wireguard, and then I'll bridge that to another a jail. That 2nd jail is running entirely off wireguard without any other way to access the network. Since it is a jail, it is isolated. And backing up is as simple as a zfs snapshot and zfs send. I assume the same is possible on Linux.
There is an Issue for that: https://github.com/QubesOS/qubes-issues/issues/7051
Maybe someday AI-assisted killchains will be so widespread that Qubes is the minimal level for the (few?) still-local users of compute.
I would not have copied anything from dom0 to any another qube, the impact is low.
[1] https://doc.qubes-os.org/en/latest/introduction/faq.html#is-...
> Important: At this point, you still don’t know whether the key you just imported is the genuine QMSK or a forgery. In order for this entire procedure to provide meaningful security benefits, you must authenticate the QMSK out-of-band. Do not skip this step! The standard method is to obtain the QMSK fingerprint from multiple independent sources in several different ways and check to see whether they match the key you just imported. For more information, see How to import and authenticate the Qubes Master Signing Key.
It looks like Qubes is ran by people who take security seriously, which is refreshing.
I also don't understand why it needs to show the dialog in dom0. If you have the option to handle attacker controlled input on the unprivileged side, you should do that instead of putting a lot of logic on the privileged side.
I think it's the "secure screen" that cannot be manipulated by the malware in a VM. I'd expect a password entry dialog to also be handled like that.