Why is Plex Media Server trying to load CUDA DLL's for any hardware transcoding? No nVIDIA hardware here!

[sheik124]

Server Version#: 1.16.2.1321
Player Version#: 3.99.2 (Web) - countless others tried, Android, Android TV, Xbox

Summary: Plex insists (and fails) to initiliaze CUDA DLL’s even if you have an AMD card or are using Intel Quick Sync.

These logs are from my server running Windows Server 2019 with an AMD Radeon Pro WX5100. All of the latest Windows Updates are installed. I also have the latest Radeon Pro drivers. Media Foundation is installed, yada yada yada - I tried a software utility (StaxRip) that can utilize AMD’s encode hardware. It works. I tried forcing LAV to decode AVC and HEVC in hardware in MPC-HC. That works. I tried to get Plex to hardware transcode. That works once or twice immediately following a reboot, then it stops working.

A friend called me and told me they had really high CPU usage on their tiny Plex server, i3 3220 in an ITX case. Obviously, he was expecting QSV to be doing the heavy lifting. I fired up Handbrake and tried to do a simultaneous QSV decode/encode. 100+ FPS. I closed that and tried to get something in 1080p to transcode, CPU usage instantly at 100%, I checked his logs and saw the same exact thing there as well that I saw in my own. Firefox choked and crashed before we could copy them, guess it wasn’t happy about PlexTranscoder.exe taking over.

Plex’s own page on hardware transcoding has the “pecking order” for the different libraries:

It skips straight to CUDA on both machines, except on mine when it is randomly working.

Neither machine has never had nVIDIA hardware in it, nor have we ever installed their drivers.

DXVA Checker reports an all clear for both machines. Mine shows H.264, VC-1, H.265, and a bunch of other encode/decode support. The i3/QSV machine shows MPEG2/H.264/VC-1 hardware decode support. Both say DXVA2/D3D11, and dxdiag.exe doesn’t show anything weird on either server too.

What gives? Handbrake is using ffmpeg - we know the features work in other software. Why isn’t it working in the one instance we actually paid for it?

Jul 19, 2019 21:24:23.556 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.556 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.578 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.578 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.590 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.591 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:23.594 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.594 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.618 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.618 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.629 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.629 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:23.633 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.633 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.653 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.653 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.667 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.667 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:23.670 [13012] Info — [FFMPEG] - MFT name: 'Microsoft AAC Audio Encoder MFT'

Jul 19, 2019 21:24:23.672 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.673 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.688 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.689 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.701 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.701 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:23.703 [13012] Info — [FFMPEG] - MFT name: 'Microsoft AAC Audio Encoder MFT'

Jul 19, 2019 21:24:23.706 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.706 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.723 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.723 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.736 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.736 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:23.739 [13012] Error — [FFMPEG] - Cannot load nvcuda.dll

Jul 19, 2019 21:24:23.739 [13012] Warning — avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jul 19, 2019 21:24:23.754 [13012] Error — [FFMPEG] - could not create MFT

Jul 19, 2019 21:24:23.754 [13012] Warning — avcodec_open2 returned -40 for encoder 'h264_mf'
Jul 19, 2019 21:24:23.766 [13012] Error — [FFMPEG] - Could not create the surfaces

Jul 19, 2019 21:24:23.766 [13012] Error — [FFMPEG] - Error creating an internal frame pool

Jul 19, 2019 21:24:29.149 [14876] Warning — Held transaction for too long (..\Statistics\StatisticsManager.cpp:248): 0.156250 seconds

[sheik124]

Nothing?

[sheik124]

This is driving me crazy. Was hardware transcoding on my Windows 10 desktop with an RX580 a fluke? Is it not working because I’m on Windows Server 2019, or because of the Radeon Pro WX cards? Is Windows Media Foundation transcoding just not supported anymore?

I tried rolling back to a much older version of PMS that I know was working with Win10 and the RX580 and I still only get software transcoding. I was able to successfully use DXVAChecker2 to start a MF (Media Foundation) transcode thread using the AMD…filter/DLL/whatever and it did in fact utilize my GPU. But no matter what I do I see those failed CUDA calls in the console.

Plex devs, if there’s anything specific I can do to provide you with more information, please let me know.

[anon18523487]

Please provide the entire pms log after recreating the issue.

[sheik124]

I have been unable to recreate the issue on the server with the AMD cards, particularly after downgrading back to 1.14.xx - and reinstalling Tautulli to deal with not having the new server statistics, but I hope this small piece of information helps.

If I reboot the server and physically log into it, hardware transcoding works. As in completely works, with no interruption. It will HW Decode everything, and HW Encode back to H.264.

The second I use Microsoft RDP, the logs start to show the bogus nvcuda.dll, h264_nvenc, and h264_mf errors. Even after disconnecting from Remote Desktop and physically logging back into the server. The only fix at that point is to reboot it and log into it locally.

Will using TeamViewer avoid this issue? I am assuming it’s happening because of the virtual display adapter RDP creates. The annoying thing is that Plex is the only affected program when it comes to no longer being able to use AMD VCE / MF transcoding - others will correctly default to the physical GPU even while RDP is still connected.

[anon18523487]

Oh, you were using RDP at the time. We’ve seen reports of other strange issues with PMS when RDP is involved. Don’t use RDP, it causes some strange things to happen that break stuff.

[sheik124]

I wish there was some rhyme or reason to it. Being able to select a GPU somewhere in Plex’s settings would go a long way towards eliminating most if not all of my issues with transcoding.

[anon18523487]

Why are you using RDP? If you’ve enabled remote access, you can access your server directly without needing RDP.

[sheik124]

I might try to run Plex on a 2nd user account that automatically logs in (locally) to see if RDP’ing into a different user account will still knock the main HW GPU down a peg on Plex’s “list of GPUs” it is silently trying for HW Transcoding. Would it be too much to ask for Plex to be hard-coded to ignore “Microsoft Remote Display Adapter”? Or for a “Select GPU” option for people who might have an integrated Intel or AMD GPU alongside something like a Quadro?

Plex isn’t the only role this server’s got! It’s a chunky 4U boi with 128GB of RAM. Kinda regretting my decision to run WS2019 bare metal because Hyper-V doesn’t want to pass any GPU’s through to…anything. Too lazy to trash the setup and switch to Proxmox.

I’ve found a workaround because calling it a solution would be silly. I can use the (crappy) IPMI to remotely access it, with some caveats. The Radeon is the main GPU and the onboard graphics are limited to like 1024x768 and on a “2nd monitor”, IPMI can only access the built-in graphics and its respective display. I have to Win+RightArrow everything over when I log on and deal with 90’s resolution. Or I can just use IPMI to reboot after using RDP, but that’s even sloppier.