Help fixing my broken nvidia nvenc driver/config (ubuntu 20.04)

Plex Media Server
1

Server Version#: 1.20.2.3343
Player Version#: 1.19.0.1399-31d8c803

So I’ve been struggling to figure out why my server is using my nvidia GPU to decode video, but not to encode. My server has a AMD Ryzen Threadripper CPU and Nvidia P4000 GPU, running Ubuntu 20.04. Digging through the logs I see the following:

Sep 28, 2020 17:49:01.793 [0x7f67627fc700] DEBUG - [Transcode] [FFMPEG] - Loaded lib: libnvidia-encode.so.1
Sep 28, 2020 17:49:01.793 [0x7f67627fc700] DEBUG - [Transcode] [FFMPEG] - Loaded sym: NvEncodeAPICreateInstance
Sep 28, 2020 17:49:01.793 [0x7f67627fc700] DEBUG - [Transcode] [FFMPEG] - Loaded sym: NvEncodeAPIGetMaxSupportedVersion
Sep 28, 2020 17:49:01.793 [0x7f67627fc700] ERROR - [Transcode] [FFMPEG] - Driver does not support the required nvenc API version. Required: 10.0 Found: 9.1
Sep 28, 2020 17:49:01.793 [0x7f67627fc700] ERROR - [Transcode] [FFMPEG] - The minimum required Nvidia driver for nvenc is 418.30 or newer
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] WARN - [Transcode] Codecs: avcodec_open2 returned -38 for encoder 'h264_nvenc'
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice vaapi
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice nvdec
Sep 28, 2020 17:49:02.160 [0x7f67627fc700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvdec

For some reason the logs are saying I have an old driver / NVENC API version, but as far as I can tell, I have the latest nvidia drivers installed. When I run nvidia-smi, it returns the following:

Mon Sep 28 17:55:21 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro P4000        Off  | 00000000:21:00.0 Off |                  N/A |
| 50%   48C    P0    30W / 105W |     26MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1458      G   /usr/lib/xorg/Xorg                             9MiB |
|    0      1838      G   /usr/bin/gnome-shell                           2MiB |
+-----------------------------------------------------------------------------+

and if I look at the driver modules I get this:

find /usr/lib/modules -name nvidia.ko -exec modinfo {} \;
filename:       /usr/lib/modules/5.4.0-47-generic/kernel/nvidia-440/nvidia.ko
alias:          char-major-195-*
version:        440.100
supported:      external
license:        NVIDIA
srcversion:     79812FEC91C545B3F516DF2
alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
depends:        ipmi_msghandler
retpoline:      Y
name:           nvidia
vermagic:       5.4.0-47-generic SMP mod_unload
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        60:9F:61:6A:D4:C9:3C:64:35:01:B1:13:6F:DD:36:46:DB:88:1D:84
sig_hashalgo:   sha512
signature:      4E:71:7F:56:8E:28:8D:6D:0A:25:73:46:9C:92:60:3F:CC:9A:28:77:
                EA:28:E9:6A:34:A0:65:8B:1F:3F:4C:0B:AD:C9:5F:A1:2D:65:B1:0A:
                D4:12:AE:F9:3D:0E:74:23:1A:75:FF:BB:E6:D6:D0:58:4E:42:FB:A6:
                13:14:14:A5:AA:EB:F5:71:C5:7A:C7:46:FD:22:60:DB:6F:4F:28:BB:
                03:C9:08:62:DD:44:D4:39:4A:BB:21:E5:5B:AB:37:00:6E:23:8C:33:
                B0:8A:90:EF:F2:98:65:58:52:D2:11:9F:C0:15:08:68:0D:C4:17:98:
                B5:A6:45:CD:CE:0E:DB:2A:76:A4:70:FD:73:03:80:C3:F0:3B:2F:16:
                B7:E6:74:E8:68:94:9B:81:1B:2E:0D:29:AE:B7:00:C9:A6:7F:0F:B9:
                C8:E2:46:51:BF:3E:93:AC:F4:DC:83:80:B0:71:B8:79:A9:BA:6C:21:
                AC:72:B8:A6:B0:87:D3:C5:5F:41:53:B8:B2:58:9A:AA:DF:15:BF:FC:
                71:AF:B0:57:2B:F7:87:4D:C6:66:F2:83:D2:E1:7C:26:C7:3C:77:FC:
                7E:DC:59:2C:C9:29:93:F8:8F:70:10:46:67:7E:A7:C6:62:0A:EC:75:
                5A:B7:CC:53:95:E7:BC:09:C9:83:68:3E:9C:17:1E:1B:26:81:04:33:
                D4:D0:93:CD:E7:23:4E:95:87:0B:AE:E4:99:88:F4:63:0E:61:1C:6C:
                0B:49:37:C5:45:CC:15:1C:BC:6D:A8:DE:85:27:6E:A8:37:8B:3D:A2:
                E7:7E:A1:20:0E:5D:4C:CB:76:71:10:D2:BE:69:F7:4D:D7:D6:12:1F:
                D6:2D:A3:88:AF:6C:2D:13:A7:32:D8:12:CC:8E:9D:5D:64:0C:50:46:
                56:E3:30:41:C2:B6:5C:B9:76:D5:CD:BC:44:9A:0A:33:E6:11:A7:87:
                41:47:CC:5C:76:DF:67:97:BF:9E:3B:AC:57:68:7E:89:B2:D1:75:45:
                EE:A4:0C:D0:1C:50:30:FE:8E:B8:3D:E0:5C:C8:2A:0A:DF:E9:B7:B3:
                A5:57:1C:2F:AA:90:18:77:7C:1C:AA:A3:7F:FC:EF:18:A9:69:FE:28:
                70:7C:31:F0:CA:8C:0D:A4:83:2E:60:AA:FC:38:F3:3E:33:DB:B6:61:
                58:B7:09:B7:00:A1:F7:03:25:AB:37:86:E3:26:4C:B1:C9:06:24:C2:
                E6:B4:C4:13:E4:0E:D5:76:C9:15:7C:4C:6E:87:3A:92:16:04:B1:85:
                33:FC:A6:9F:EE:7D:88:BE:03:28:80:C1:DB:CB:26:B0:29:C6:00:BB:
                91:FA:56:BA:73:F3:C3:38:09:11:19:3F
parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
parm:           NVreg_Mobile:int
parm:           NVreg_ResmanDebugLevel:int
parm:           NVreg_RmLogonRC:int
parm:           NVreg_ModifyDeviceFiles:int
parm:           NVreg_DeviceFileUID:int
parm:           NVreg_DeviceFileGID:int
parm:           NVreg_DeviceFileMode:int
parm:           NVreg_InitializeSystemMemoryAllocations:int
parm:           NVreg_UsePageAttributeTable:int
parm:           NVreg_MapRegistersEarly:int
parm:           NVreg_RegisterForACPIEvents:int
parm:           NVreg_EnablePCIeGen3:int
parm:           NVreg_EnableMSI:int
parm:           NVreg_TCEBypassMode:int
parm:           NVreg_EnableStreamMemOPs:int
parm:           NVreg_EnableBacklightHandler:int
parm:           NVreg_RestrictProfilingToAdminUsers:int
parm:           NVreg_PreserveVideoMemoryAllocations:int
parm:           NVreg_DynamicPowerManagement:int
parm:           NVreg_EnableUserNUMAManagement:int
parm:           NVreg_MemoryPoolSize:int
parm:           NVreg_KMallocHeapMaxSize:int
parm:           NVreg_VMallocHeapMaxSize:int
parm:           NVreg_IgnoreMMIOCheck:int
parm:           NVreg_NvLinkDisable:int
parm:           NVreg_RegisterPCIDriver:int
parm:           NVreg_RegistryDwords:charp
parm:           NVreg_RegistryDwordsPerDevice:charp
parm:           NVreg_RmMsg:charp
parm:           NVreg_GpuBlacklist:charp
parm:           NVreg_TemporaryFilePath:charp
parm:           NVreg_AssignGpus:charp
filename:       /usr/lib/modules/5.4.0-48-generic/updates/dkms/nvidia.ko
alias:          char-major-195-*
version:        440.100
supported:      external
license:        NVIDIA
srcversion:     79812FEC91C545B3F516DF2
alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
depends:        ipmi_msghandler
retpoline:      Y
name:           nvidia
vermagic:       5.4.0-48-generic SMP mod_unload
sig_id:         PKCS#7
signer:         skynet Secure Boot Module Signature key
sig_key:        21:AF:51:58:09:F3:C1:D5:9C:B8:AB:FB:83:49:0D:A7:CA:28:3B:E9
sig_hashalgo:   sha512
signature:      07:6E:72:DA:E9:8D:2E:C0:A9:2B:D3:6A:25:AD:15:B4:1F:4D:35:40:
                C6:CE:01:61:AE:74:29:07:97:95:44:AB:62:7B:73:5E:CB:BC:A6:B3:
                F9:9E:BD:FA:0E:1D:20:F6:5B:34:75:6B:E8:BE:C2:22:16:82:FC:48:
                77:82:38:81:DD:7E:0D:13:B4:F6:C5:CC:2A:D1:A4:3E:C8:4D:5F:D3:
                AF:E2:DE:8C:CD:6E:22:EF:75:04:37:FB:2C:AB:3B:FF:29:0E:BE:2E:
                20:E2:8F:C0:C2:11:B3:B4:B5:89:53:6F:7B:83:3C:48:C8:2D:A8:86:
                8F:D1:32:D3:DA:8E:86:86:F5:E2:37:5B:5C:85:9E:9B:DD:17:CC:F9:
                49:24:17:7D:B6:E8:24:75:74:F8:BD:FA:BD:8C:F4:03:88:DF:BB:03:
                00:DA:FB:2E:EC:10:7C:2B:2D:25:F6:F8:D7:39:27:AC:B2:6D:48:4F:
                3B:04:23:97:2E:E5:F2:DE:51:93:E6:74:8D:CF:F3:01:CD:41:9C:39:
                2B:FC:85:7A:88:C4:3A:C0:D6:C7:14:8D:4A:81:A1:CA:FC:85:6E:60:
                9D:08:99:A4:CA:57:D9:0C:8D:4C:CA:97:13:3B:FC:0F:C7:E5:B3:9F:
                92:92:16:DF:B1:7E:02:D7:A0:24:66:F3:AB:4A:4B:6C
parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
parm:           NVreg_Mobile:int
parm:           NVreg_ResmanDebugLevel:int
parm:           NVreg_RmLogonRC:int
parm:           NVreg_ModifyDeviceFiles:int
parm:           NVreg_DeviceFileUID:int
parm:           NVreg_DeviceFileGID:int
parm:           NVreg_DeviceFileMode:int
parm:           NVreg_InitializeSystemMemoryAllocations:int
parm:           NVreg_UsePageAttributeTable:int
parm:           NVreg_MapRegistersEarly:int
parm:           NVreg_RegisterForACPIEvents:int
parm:           NVreg_EnablePCIeGen3:int
parm:           NVreg_EnableMSI:int
parm:           NVreg_TCEBypassMode:int
parm:           NVreg_EnableStreamMemOPs:int
parm:           NVreg_EnableBacklightHandler:int
parm:           NVreg_RestrictProfilingToAdminUsers:int
parm:           NVreg_PreserveVideoMemoryAllocations:int
parm:           NVreg_DynamicPowerManagement:int
parm:           NVreg_EnableUserNUMAManagement:int
parm:           NVreg_MemoryPoolSize:int
parm:           NVreg_KMallocHeapMaxSize:int
parm:           NVreg_VMallocHeapMaxSize:int
parm:           NVreg_IgnoreMMIOCheck:int
parm:           NVreg_NvLinkDisable:int
parm:           NVreg_RegisterPCIDriver:int
parm:           NVreg_RegistryDwords:charp
parm:           NVreg_RegistryDwordsPerDevice:charp
parm:           NVreg_RmMsg:charp
parm:           NVreg_GpuBlacklist:charp
parm:           NVreg_TemporaryFilePath:charp
parm:           NVreg_AssignGpus:charp

Any idea what I’m doing wrong and what I can do to fix it? Thanks very much in advance for any assistance.

2

You can find out what it thinks it’s looking at and go from there using ldd.
I would recommend searching for nvidia

Example using libGL:

$ ldd /usr/bin/glxinfo | grep -i libGL
libGL.so.1 => /usr/lib/libGL.so.1 (0x00007f34ff796000)

ldconfig -p will let you look at all the places it knows for library files.

3

Thanks Chuck. Results:

$ ldd /usr/bin/glxinfo | grep -i nvidia
ldd: /usr/bin/glxinfo: No such file or directory

and

$ ldconfig -p | grep -i nvidia
        libnvidia-tls.so.440.100 (libc6,x86-64, OS ABI: Linux 2.3.99) => /lib/x86_64-linux-gnu/libnvidia-tls.so.440.100
        libnvidia-tls.so.440.100 (libc6, OS ABI: Linux 2.3.99) => /lib/i386-linux-gnu/libnvidia-tls.so.440.100
        libnvidia-rtcore.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-rtcore.so.440.100
        libnvidia-ptxjitcompiler.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1
        libnvidia-ptxjitcompiler.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-ptxjitcompiler.so.1
        libnvidia-ptxjitcompiler.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so
        libnvidia-ptxjitcompiler.so (libc6) => /lib/i386-linux-gnu/libnvidia-ptxjitcompiler.so
        libnvidia-opticalflow.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1
        libnvidia-opticalflow.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-opticalflow.so.1
        libnvidia-opticalflow.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-opticalflow.so
        libnvidia-opticalflow.so (libc6) => /lib/i386-linux-gnu/libnvidia-opticalflow.so
        libnvidia-opencl.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-opencl.so.1
        libnvidia-opencl.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-opencl.so.1
        libnvidia-ml.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ml.so.1
        libnvidia-ml.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-ml.so.1
        libnvidia-ml.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ml.so
        libnvidia-ml.so (libc6) => /lib/i386-linux-gnu/libnvidia-ml.so
        libnvidia-ifr.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ifr.so.1
        libnvidia-ifr.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-ifr.so.1
        libnvidia-ifr.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-ifr.so
        libnvidia-ifr.so (libc6) => /lib/i386-linux-gnu/libnvidia-ifr.so
        libnvidia-gtk3.so.440.82 (libc6,x86-64) => /lib/libnvidia-gtk3.so.440.82
        libnvidia-gtk2.so.440.82 (libc6,x86-64) => /lib/libnvidia-gtk2.so.440.82
        libnvidia-glvkspirv.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.440.100
        libnvidia-glvkspirv.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-glvkspirv.so.440.100
        libnvidia-glsi.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-glsi.so.440.100
        libnvidia-glsi.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-glsi.so.440.100
        libnvidia-glcore.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-glcore.so.440.100
        libnvidia-glcore.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-glcore.so.440.100
        libnvidia-fbc.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-fbc.so.1
        libnvidia-fbc.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-fbc.so.1
        libnvidia-fbc.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-fbc.so
        libnvidia-fbc.so (libc6) => /lib/i386-linux-gnu/libnvidia-fbc.so
        libnvidia-fatbinaryloader.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.440.100
        libnvidia-fatbinaryloader.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-fatbinaryloader.so.440.100
        libnvidia-encode.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-encode.so.1
        libnvidia-encode.so.1 (libc6) => /lib/i386-linux-gnu/libnvidia-encode.so.1
        libnvidia-encode.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-encode.so
        libnvidia-encode.so (libc6) => /lib/i386-linux-gnu/libnvidia-encode.so
        libnvidia-eglcore.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-eglcore.so.440.100
        libnvidia-eglcore.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-eglcore.so.440.100
        libnvidia-container.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-container.so.1
        libnvidia-compiler.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-compiler.so.440.100
        libnvidia-compiler.so.440.100 (libc6) => /lib/i386-linux-gnu/libnvidia-compiler.so.440.100
        libnvidia-cfg.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-cfg.so.1
        libnvidia-cfg.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-cfg.so
        libnvidia-cbl.so.440.100 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-cbl.so.440.100
        libnvidia-allocator.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-allocator.so.1
        libnvidia-allocator.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libnvidia-allocator.so
        libGLX_nvidia.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libGLX_nvidia.so.0
        libGLX_nvidia.so.0 (libc6) => /lib/i386-linux-gnu/libGLX_nvidia.so.0
        libGLESv2_nvidia.so.2 (libc6,x86-64) => /lib/x86_64-linux-gnu/libGLESv2_nvidia.so.2
        libGLESv2_nvidia.so.2 (libc6) => /lib/i386-linux-gnu/libGLESv2_nvidia.so.2
        libGLESv1_CM_nvidia.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1
        libGLESv1_CM_nvidia.so.1 (libc6) => /lib/i386-linux-gnu/libGLESv1_CM_nvidia.so.1
        libEGL_nvidia.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
        libEGL_nvidia.so.0 (libc6) => /lib/i386-linux-gnu/libEGL_nvidia.so.0

Seems like the encode library file is /lib/x86_64-linux-gnu/libnvidia-encode.so.1

$ ls -l /lib/x86_64-linux-gnu/libnvidia-encode.so.*
lrwxrwxrwx 1 root root     27 May 29 03:14 /lib/x86_64-linux-gnu/libnvidia-encode.so.1 -> libnvidia-encode.so.440.100
-rw-r--r-- 1 root root 103632 May 29 01:11 /lib/x86_64-linux-gnu/libnvidia-encode.so.440.100
4

install latest drivers from list @ https://github.com/keylase/nvidia-patch

you may also need to google on how to disable nouveau driver, if you haven’t already.

1 Like
5

I don’t need the patch (Quadro P4000 has unlocked encodes) but I will look into updating the driver, as I notice 450.66 is available.

As far as I can tell, I don’t have the nouveau drivers installed:

$ sudo lshw -C display
  *-display
       description: VGA compatible controller
       product: GP104GL [Quadro P4000]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:21:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:129 memory:c0000000-c0ffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:1000(size=128) memory:c1000000-c107ffff

I’ll report back if I still have the issue after updating the drivers. Other ideas and root cause thoughts welcome! Thanks!

1 Like
6

Update: I am back in business. Performing a ‘sudo apt dist-upgrade’ correctly upgraded my nvidia driver to 450.66, and following a reboot Plex is now hardware transcoding again without error.

Thank you all for the help, very much appreciated!

7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.