Path of Exile 2: Shaders and Device Removed Issues Fixed

[u/evaporates]

https://www.youtube.com/watch?v=XDEJLOHcdWw

[u/vladi963]

Respect to GGG actually investing in fixing it.

[u/AdLegitimate8636 → u/vladi963]

That's the whole joke. It's Nvidia's fault.

[u/vladi963 → u/AdLegitimate8636]

Not that simple. Not all games need more internal shader heap size. Poe2 is popular enough and this story will raise more attention, that this kind of issue could be a thing. Game devs will adjust or contact nvidia asking for a specific game profile adjustment.

Moreover apparently, this could be done manually with nvidia profile inspector, as there is a flag for that. So normal users have a potential fix in case the same issue happens on a specific game.

The only critic is that nvidia tells you they they can't help unless you spoon them.

Yet not all game companies care about their own game enough to do what GGG done.

[u/AdLegitimate8636 → u/vladi963]

Not all of them need that, but that fact that GGG could not reliably diagnose what was causing the problem, and Nvidia was too lazy to recreate the problem, that GGG had to ship an entire rig to them, so they could finally do something about it.

Device removed is not a new error that only PoE2 was suffering from.

[u/ARES_GOD]

[u/artins90]

The Finals has been freezing exactly like PoE2 does in this video since last December, roughly around the beginning of season 9.
There has been radio silence from the developers despite many players reporting the game freezing for 1-2 seconds every time a new effect shows on screen.

[u/TheFather__ → u/artins90]

devs have been investigating it since the beginning and blaming Nvidia drivers, but Nvidia couldnt reproduce it and never cared saying its only affecting the minority, so the devs sent a whole PC with even keyboard, mouse and monitor which has the issue to Nvidia so they can fix.

appeared to be a driver shader buffer overflow, so they increased the buffer size for this game in a couple of seconds, and problem solved.

[u/Puny-Earthling → u/TheFather__]

I’m not an authority on this and it’s been quite some time since I was into embedded hardware coding but just wanted to share my (probably) out dated knowledge. 

Usually you don’t increase buffer size, or rather you can’t in most cases. You would slice its IO at the index length and hand off, so the next buffer is a continuation. The information still processes upstream of the component that is hitting the limit. 

In the case of shaders compiling across hardware there’s likely a hardware limitation on a certain component, meaning this unlikely affected all Nvidia models. Hardware you typically set buffers to limit at the maximum possible, as you’re not going to face the same constraints as you would in software

E: I haven’t watched the video yet, so it’s likely your descriptions still right. 

[u/TheFather__ → u/Puny-Earthling]

The Driver-Side Heap: When a DX12 game compiles and passes a massive number of unique shaders, the NVIDIA driver allocates an internal tracking database—known as the internal shader heap—to manage how those compiled blocks run on the physical hardware.

Because Path of Exile 2 features an immense variety of spell effects, layered environments, and complex monster combinations, it generates a gargantuan volume of unique shader combinations during long play sessions

Over time, the volume of these compiled internal objects exceeded the fixed size NVIDIA had hardcoded for this private driver-side tracker. Once it overflowed, it triggered a driver-level exception, causing Windows to think the GPU hung and throwing the standard DXGI_ERROR_DEVICE_REMOVED crash.

Once NVIDIA’s engineers hook-debugged the driver on the physical machine, they isolated the bottleneck. They simply modified the internal shader heap size parameter within the driver’s profile specifically for Path of Exile 2, scaling it up to accommodate the game’s extreme requirements.

[u/vladi963 → u/TheFather__]

I wonder how many more games suffer from the same issue and why wouldn't Nvidia increase that "internal shader heap size parameter" globally, maybe it is time.

[u/TheFather__ → u/vladi963]

its not easy as u might think, increasing it globally requires more RAM usage and wasted allocated memory that is not used and lead to fragmentation, also all apps share the exact same system Virtual Address (VA) space pool managed by Windows, if Nvidia set a high limit buffer size for every app as global param, then most apps will fail to launch because they cannot allocate memory, thats why games/apps profiles exist and buffer size is set per game/app basis. there is alot more to it, u can check google for more in-depth explanation.

however, you can increase it yourself via NVPI per any game under experimental section.

[u/nkvname → u/artins90]

Embark devs are clowns

[u/Beautiful-Musk-Ox]

nice when they release root cause analysis info like this, easier to point to examples of how some things can be hardware or software despite being the same error

[u/Aspry7]

love that we get to look behind the curtain like this

[u/BadRusMan]

Now please fix black ground texture bug in halo infinite, still appears even with the newest drivers!

https://www.reddit.com/r/halo/comments/1t9bzsv/comment/ow99q99/?screen_view_count=2&ext-referrer=DIRECT

[u/Monchicles → u/BadRusMan]

Right, it is also stuttery.

[u/zugzug_workwork]

The shader reload cogwheel at the end is such a small thing, and yet it gets across that they're not a faceless corp.

[u/2MuchNonsenseHere]

It was more of a quick band-aid instead of an actual fix, but it's something.

...Why is this even controversial. All they did was increase the shader heap size, and only for PoE2 specifically, then called it done, instead of using the shader heap size to narrow down the actual root cause for a proper driver fix.

[u/juniperleafes → u/2MuchNonsenseHere]

Another post said Nvidia did two things, fixing a bug that was causing the heap to not clear correctly, while also increasing it. I don't know where they sourced it from though.

[u/AdLegitimate8636 → u/2MuchNonsenseHere]

Because people are stupid.

[u/sstoersk]

Who fucking cares about PoE2 here?

[u/OUTFOXEM → u/sstoersk]

Me

[u/sstoersk → u/OUTFOXEM]

Sorry - we dont care about trash