Cannot enable HW transcoding (nvidia) on existing PMS install

Server Version#: 1.21.0.3616
Player Version#: 4.47.1

Yesterday I purchased a gtx 1660 super to add to my existing plex server. This server is currently running Ubuntu 20.04 (upgraded from 18.04).

Upon installing the card & NVIDIA drivers, I verified the driver through nvidia-smi and everything appeared to be good. However, when I start a file and transcode it, I am not seeing the file transcode (hw designation). I have double checked that transcoding options are on in PMS. I do see the following two errors in the console when trying to transcode:
Codecs: testing h264_nvenc (encoder)
Debug — [Transcode] Codecs: hardware transcoding: testing API nvenc
[FFMPEG] - cu->cuInit(0) failed
Error — [Transcode] [FFMPEG] - -> CUDA_ERROR_UNKNOWN: unknown error
Error — [Transcode] [FFMPEG] -
Debug — [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred
Debug — [Transcode] Codecs: testing h264 (decoder) with hwdevice vaapi
Debug — [Transcode] Codecs: hardware transcoding: testing API vaapi
Error — [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Error — [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Debug — [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error

I did find that plex did not have access to the render group, which I updated. I have reinstalled the CUDA toolkit directly from nvidia using their deb(local) instructions as well as reinstalling the drivers. I’ve made sure that I have the supplemental codecs installed.

I’m not the greatest at Linux, but I’m completely stumped. Any ideas before I just return the card?

You should not return the card.

The first thing you need to confirm is that the command nvidia-smi works.

That will confirm drivers are installed and visible to apps.

If/once nvidia-smi is working, then attach full logs, plex web - settings - troubleshooting - download logs - attach whole zip to reply here.

Thank you. Yes, the nvidia-smi command does work and shows 455.32.00 driver and 11.1 cuda libraries. I have attached my server log.(File removed)

mde: lines show why something is transcoding

Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: Selected protocol dash; container: mp4
Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: analyzing media item 120359
Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: E2 - Far from Home: Direct Play is disabled
Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: E2 - Far from Home: media must be transcoded in order to use the dash protocol
Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: E2 - Far from Home: no direct play video profile exists for http/mkv/h264
Nov 25, 2020 12:19:35.421 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: E2 - Far from Home: no direct play video profile exists for http/mkv/h264/ac3
Nov 25, 2020 12:19:35.422 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: Cannot direct stream audio stream due to codec ac3 when profile only allows aac
Nov 25, 2020 12:19:35.422 [0x7f28bf7fe700] DEBUG - [Transcode] MDE: Star Trek: Discovery - S3 E2 - Far from Home: selected media 0 / 120359
Nov 25, 2020 12:19:35.422 [0x7f28bf7fe700] DEBUG - [Transcode] Streaming Resource: Added session 0x7f28a8170c50:y0xkdpgm76joci4gmmodp0p1
Nov 25, 2020 12:19:35.422 [0x7f28bf7fe700] DEBUG - [Transcode] Streaming Resource: Reached Decision id=104889 codes=(General=1001,Direct play not available; Conversion OK. Direct Play=3000,App cannot direct play this item. Direct play is disabled. Transcode=1001,Direct play not available; Conversion OK.) media=(id=120359 part=(id=125425 decision=transcode container=mp4 protocol=dash streams=(Video=(id=266949 decision=copy width=1280 height=720) Audio=(id=266950 decision=transcode bitrate=256 encoder=aac channels=2 rate=48000))))

tpu: lines show how something is transcoded

Nov 25, 2020 12:19:35.536 [0x7f28b5ffb700] DEBUG - [Transcode] TPU: hardware transcoding: enabled, but no hardware decode accelerator found
...
Nov 25, 2020 12:19:41.387 [0x7f28b7fff700] DEBUG - [Transcode] TPU: hardware transcoding: final decoder: , final encoder: 

[ffmpeg] lines show errors

Nov 25, 2020 12:19:40.879 [0x7f28beffd700] ERROR - [Transcode] [FFMPEG] - cu->cuInit(0) failed
Nov 25, 2020 12:19:40.879 [0x7f28beffd700] ERROR - [Transcode] [FFMPEG] -  -> CUDA_ERROR_UNKNOWN: unknown error
Nov 25, 2020 12:19:40.879 [0x7f28beffd700] ERROR - [Transcode] [FFMPEG] - 
Nov 25, 2020 12:19:40.880 [0x7f28beffd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred

looks like plex can’t access the gpu, so likely a permissions issue somewhere.

is this in a container, or did you manually install with dpkg ?

also, you do not need the whole cuda framework, only the nvidia drivers.

are the drivers installed from ubuntu package manager, or from the nvidia .run installer?

there is a good list of nvidia drivers @ GitHub - keylase/nvidia-patch: This patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs.

This PMS is not in a container and is installed with dpkg.

I have installed the Nvidia driver’s every way I know how. The first time was from the package manager and then I went back and reinstalled them directly from Nvidia via terminal.

when you install plexmediaserver via dpkg -i /path/to/pms.deb did/does the installer detect HW ?

it should show both on the terminal and the installer logs, I can’t remember the file name, but might still be in the /tmp folder.

edit: /tmp/plexinstaller.log on my system

# Plex Media Server installation configuration info:  Mon 23 Nov 2020 05:55:37 AM CST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=1
OverrideFile="/etc/systemd/system/plexmediaserver.service.d/override.conf"
PlexUser="plex"
PlexGroup="dvrapps"
VideoGroup="render"
AppSuppDir="/dataplex/pms/Library/Application Support"
PlexTempDir="/transcode"
LangEncoding="en_US.UTF-8"
ExistingVersion=12100
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
Havei915=0
HaveBeignet=1
HaveOpenCL=1
Platform="VB1416"
Processor="Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz"
Distro="Debian GNU/Linux bullseye/sid"
Kernel="Linux proximo 5.8.0-3-amd64 #1 SMP Debian 5.8.14-1 (2020-10-10) x86_64 GNU/Linux"
Memory="64377 5543 5783"

what does ls -lah /dev/dri show ?

tekno@proximo:~$ ls -lah /dev/dri
total 0
drwxr-xr-x   3 root root        120 Nov 10 04:28 .
drwxr-xr-x  18 root root       4.5K Nov 10 04:30 ..
drwxr-xr-x   2 root root        100 Nov 10 04:28 by-path
crw-rw----+  1 root render 226,   0 Nov 23 05:55 card0
crw-rw----+  1 root render 226,   1 Nov 23 05:55 card1
crw-rw----+  1 root render 226, 128 Nov 23 05:55 renderD128

also, what kernel are you running, kernel 5.9 is incompatible with nvidia drivers currently

uname -a

tekno@proximo:~$ uname -a
Linux proximo 5.8.0-3-amd64 #1 SMP Debian 5.8.14-1 (2020-10-10) x86_64 GNU/Linux

also, what kernel are you running, kernel 5.9 is incompatible with nvidia drivers currently

BINGO!

I had no idea of that. It’s one of those Ubuntu/Linux things that I’m just not used to after fooling with a PC and Mac all my life. I found a PS/2 adapter for my keyboard, got into the advanced grub mode and went back to 5.8.18. Fired up a 4K movie and transcoded it down to 1080p without Plex even breaking a sweat (I realize that’s not a good thing to do, just wanted to try).

Thank you for your help. I was definitely in over my head!

1 Like

The kernel incompatible is rare, apparently nvidia is/was having some problems getting their end straightened out.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.