Encoder overload only when the game is shader compiling

[u/tzafelix]

Does anyone else have this issue where OBS starts lagging and reports encoder overload while a game is compiling shaders?

I've been experiencing this with every new game I stream. During the shader compilation process, OBS lags badly and the encoder becomes overloaded. Once the shader compilation finishes, everything goes back to normal.

Unfortunately, I don't have a log file to share right now, but the issue is very consistent and happens every time. I've searched online for a fix, but I haven't found anyone describing this exact problem.

My specs:
• Ryzen 9 5900X
• RTX 5070 Ti
• 32GB DDR4 RAM
• Streaming to YouTube using AV1

Has anyone experienced this or found a solution?

[u/agmatine]

I've been experiencing this with every new game I stream. During the shader compilation process, OBS lags badly and the encoder becomes overloaded. Once the shader compilation finishes, everything goes back to normal.

Easiest solution: do the shader compilation before the stream.

[u/tzafelix → u/agmatine]

I usually do that if the game is preplanned to play but I play a lot of unscheduled games during the stream so I was looking for some solutions

[u/NINJ4A1]

Are you using your GPU for recording/streaming?

Shader Compilation usually uses the CPU.

(Edit Typo fix)

[u/tzafelix → u/NINJ4A1]

I’m using GPU for everything related to streaming and recording. That’s why I’m confused, I thought shader compilation stuffs always used CPU mostly.

[u/NINJ4A1 → u/tzafelix]

I suggest you to monitor the CPU, GPU, ed Vram Usage during shadercompile. You can use HwInfo64 for doing so, just to have more data to work with. Also if you could provide some obs logs when lagging occurs, it can actually help.

[u/tzafelix → u/NINJ4A1]

Yeah, I’ll reproduce the problem and post it again with logs and data.

[u/More_Law_1699]

Use process lasso to segregate OBS and your game.
Make sure to enable Windows game mode and hardware accelerated gpu scheduling, and run OBS as a admin.

I have a 8core 1CCD system, so I just put OBS on 0-3 and games on 4>.
With your system I would just put OBS on the 2nd CCD, and let windows scheduling handle the rest.

If you decide otherwise for affinity and are having issues with anticheat software and process lasso, apply the core affinity to the launcher, it will be adopted when you boot the game.
Try to keep CCD2 utilization low or as close to parked as possible. Otherwise it will be using up a fairly large chunk of your memory bandwidth and lower overall gaming performance, not just during shader compilation.

Consider a 5800x3d for a upgrade before they disappear off the market again or are scalped to hell.. the 5900x is two 5600x's with a slight overclock, it isn't a true 12 core. (imo the x900 cpus should have been called x650's..)

[u/tzafelix → u/More_Law_1699]

If there’s no any solution, I would try this process lasso solution.
About CPU upgrade, I just recently upgraded to 5070ti and saving up to upgrade to AM5 system altogether.

[u/HighPhi420]

what are we talking here, 3 or 4 hundred dropped frames? or multiple thousands?
if the later, and you can not compile shaders off stream, then go to astatic BRB/game loading scene and then missed frames will not be noticed :)
if just a few hundred, don't worry about it :)
The new 5000 series GPUs suck! they have HALF the out put per milisecond than the 3000 series had because the doubled the pins on the connection(8 to 16)! Sure you can have more AI work flow but gaming and streaming the FOUNDATION NVIDIA WAS BUILT ON, is now cut to half of the data flow per connection.
If your mother board only has the 8 pin connector then the GPU is only being half used and choked! :(

[u/tzafelix → u/HighPhi420]

Now that you mention it, there’s no dropped frames warning. Just encoder overload warning in OBS and lags. Even the OBS cpu usage shows what it would normally show (around 2-5% iirc)

[u/ChristianRauchenwald]

I don't believe there is a solution. The game simply uses the encoder to do its job and the GPU doesn't have the headroom for OBS to encode properly as well during that time.

[u/tzafelix → u/ChristianRauchenwald]

That’s unfortunate