Hey this is my work, although the exploits etc aren't my ideas by any means.
This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
Yeah I hate the patched drivers too, I have a CMP170 as well I want to see if I can find a way to get fully working the same way without or minimal driver patching.
4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)
Can't wait, I have two 48G modded ada 4090's and 4 stock Blackwell 4000's (though I doubt those will be as easy). The 4090's alone would be an amazing thing to have boosted, they're my main workhorses and effectively any improvement there will immediately make my days more productive. If I can help out with testing, my email is in my profile.
Yes, but it's pinned to a driver version that I unfortunately can not use. That's precisely why I love your solution, it is the best way to do this.
I'm running 8 GPUs in a single box and that driver is a pretty essential part of getting this to work with P2P between as many cards as possible. The modded 4090 is a problem because the BAR1 it advertises (32G) is smaller than the VRAM so not all software is able to use it (you need two memory segments rather than a single 48G one to make this work). That driver is part of the solution for this problem.
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
It's quite nuanced. In a nutshell the unlock just flips the 2 bits that halves how often the core can issue the specific instructions Nvidia locks to full rate on the professional and datacenter cards.
Obviously doing more compute requires more power but what it does is shift the card down on the frequency curve to keep the wattage in check.
So you're right that at stock power limits it hits a cap where the full issuing rate isn't reached but it depends heavily on the exact workload how much juice is left.
The frequency curve also takes heat into account so in general you end up with a nice perf boost for a slightly hotter card.
I've been running this on my 2 3090s for a month now with no issues but they are also water cooled :-)
Ah, that's a twist I didn't see coming, all my GPUs are aircooled. I can see how with watercooling you'd be able to get more performance out of the GPU before hitting the limits.
> - you will need to run your card hotter than you might be comfortable with
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.
I put it together myself. It was a lot of work and I learned a lot of nasty lessons along the way. The short version of the recipe:
- get the largest PSU you can find (1500W at a minimum, more is better)
- get the ADT PLX Quad PCIe backplane. Run 4 thin (single slot) GPUs inside your main enclosure, stick the ADT interface board in between two of the thin boards (don't stack them close to each other or they won't be able to breathe).
- ensure you have as much or more RAM as you do VRAM
- ditch the cables that come with the ADT board, they may work but there are much better ones out there and they're not expensive, this will save you days of troubleshooting
- Build your own enclosure rather than trying to adapt a standard one, it won't work anyway (the board pitch is wrong, as is the placement of the mounting holes)
- Add another 3x 140 mm fan to the front of the case so you get good airflow without the feeling of being next to a Harrier at take-off
- Make sure the board is properly grounded to the enclosure
- Power on the ADT board, wait three seconds then power on the main machine. Enjoy your VRAM :)
Yes, but you can get up to a 40-50% increase in some workloads even with the same power limit, and some cards have it locked anyway. I just felt like it was a litle misleading for you to mention the power in your original post, since the unlock works regardless of that.
Interesting, I have 7 3090's here of different plumage and they all require a power increase to get > 25% gain with this modification, would you mind sharing the type of card you have and how it is cooled?
Obviously, if there are 3090's out there that respond differently then I will be happy to retract that.
This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-)
There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
Happy to answer questions!
edit: now do the 4090 ;)
4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)
If you can't wait for 4090 someone has a fork with a seemingly working unlock :D
I'm running 8 GPUs in a single box and that driver is a pretty essential part of getting this to work with P2P between as many cards as possible. The modded 4090 is a problem because the BAR1 it advertises (32G) is smaller than the VRAM so not all software is able to use it (you need two memory segments rather than a single 48G one to make this work). That driver is part of the solution for this problem.
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
Obviously doing more compute requires more power but what it does is shift the card down on the frequency curve to keep the wattage in check. So you're right that at stock power limits it hits a cap where the full issuing rate isn't reached but it depends heavily on the exact workload how much juice is left.
The frequency curve also takes heat into account so in general you end up with a nice perf boost for a slightly hotter card.
I've been running this on my 2 3090s for a month now with no issues but they are also water cooled :-)
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
- get the largest PSU you can find (1500W at a minimum, more is better)
- get the ADT PLX Quad PCIe backplane. Run 4 thin (single slot) GPUs inside your main enclosure, stick the ADT interface board in between two of the thin boards (don't stack them close to each other or they won't be able to breathe).
- ensure you have as much or more RAM as you do VRAM
- ditch the cables that come with the ADT board, they may work but there are much better ones out there and they're not expensive, this will save you days of troubleshooting
- Build your own enclosure rather than trying to adapt a standard one, it won't work anyway (the board pitch is wrong, as is the placement of the mounting holes)
- Add another 3x 140 mm fan to the front of the case so you get good airflow without the feeling of being next to a Harrier at take-off
- Make sure the board is properly grounded to the enclosure
- Power on the ADT board, wait three seconds then power on the main machine. Enjoy your VRAM :)
Obviously, if there are 3090's out there that respond differently then I will be happy to retract that.
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada
On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).
I also wonder how this affects power consumption.
If its memory bandwidth bound you need to raise your batch sizes to leverage more parallelism.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-) There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
Also this one for 4070s shows that its for fp8 as well:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
There's at least 3080 Ti, 3070 Ti, 3070, 3060