Ubuntu 18.04 stopped HW transcoding

[Mi_Key]

Server Version#: 1.14.0.5465
Player Version#: ALL

nvidia-smi 390.87 / driver 410.78
also tried driver 390.87

A brand new Ubuntu 18.04 server (headless) build with a nvidia p2000, i7-8700 CPU was transcoding with ease yesterday. Today not one HW transcoding can be initiated and the cpu is saturated with 8-9 streams.

Logs show:

Nov 17, 2018 10:13:47.392 [0x7f10363f4700] DEBUG - TPU: hardware transcoding: enabled, but no hardware decode accelerator found
Nov 17, 2018 10:13:47.401 [0x7f10363f4700] DEBUG - TPU: hardware transcoding: zero-copy support not present
Nov 17, 2018 10:13:47.401 [0x7f10363f4700] DEBUG - TPU: hardware transcoding: final decoder: , final encoder: 
Nov 17, 2018 10:13:58.769 [0x7f103b3fe700] DEBUG - TPU: hardware transcoding: enabled, but no hardware decode accelerator found
Nov 17, 2018 10:13:58.779 [0x7f103b3fe700] DEBUG - TPU: hardware transcoding: zero-copy support not present
Nov 17, 2018 10:13:58.779 [0x7f103b3fe700] DEBUG - TPU: hardware transcoding: final decoder: , final encoder:

Linux minibeast 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

The only changes that occurred within the last 24hrs was me plugging in a 4k dummy plug instead of monitor and a reboot. I also enabled liveTV via my hd homerun.
I have since swapped back to a 4k monitor and the problem exists.

root@minibeast:~# ls -ahl /dev/dri
total 0
drwxr-xr-x   3 root root       100 Nov 17 05:10 .
drwxr-xr-x  20 root root      4.1K Nov 17 10:29 ..
drwxr-xr-x   2 root root        80 Nov 17 05:10 by-path
crw-rw----+  1 root video 226,   0 Nov 17 05:10 card0
crw-rw----+  1 root video 226, 128 Nov 17 05:10 renderD128

Nov 17, 2018 10:14:04.675 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Nov 17, 2018 10:14:05.187 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: testing API vaapi
Nov 17, 2018 10:14:05.187 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Nov 17, 2018 10:14:05.692 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: testing API vaapi
Nov 17, 2018 10:14:05.692 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Nov 17, 2018 10:14:06.224 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: testing API vaapi
Nov 17, 2018 10:14:06.224 [0x7f105bfff700] DEBUG - Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error

Nov 17, 2018 10:32:46.995 [0x7f1033bef700] ERROR - [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).

Any advice to rectify this would be appreciated.

Cheers!

[Mi_Key]

I’ve upgraded to nvidia drivers 415.13 and performed a few nvidia driver purges and reboots which seems to have allowed some form of transcoding to work.
My bios had onboard iGPU disabled and set to PCIX slot only. It is now switched back to Auto.

I suspect its back to using the onboard iGPU, rather than P2000 GPU.

Nov 17, 2018 23:10:17.916 [0x7f2fca3ff700] DEBUG - TPU: hardware transcoding: using hardware decode accelerator vaapi
Nov 17, 2018 23:10:17.916 [0x7f2fca3ff700] DEBUG - TPU: hardware transcoding: zero-copy support present
Nov 17, 2018 23:10:17.916 [0x7f2fca3ff700] DEBUG - TPU: hardware transcoding: using zero-copy transcoding
Nov 17, 2018 23:10:17.916 [0x7f2fca3ff700] DEBUG - TPU: hardware transcoding: final decoder: vaapi, final encoder: vaapi

This output now shows both devices, whereas previously only showed the nvidia card.

root@minibeast:/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Logs# lshw -c video
  *-display                 
       description: VGA compatible controller
       product: GP106GL
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:16 memory:a3000000-a3ffffff memory:90000000-9fffffff memory:a0000000-a1ffffff ioport:3000(size=128) memory:a4000000-a407ffff
  *-display
       description: VGA compatible controller
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       version: 00
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
       configuration: driver=i915 latency=0
       resources: irq:139 memory:a2000000-a2ffffff memory:80000000-8fffffff ioport:4000(size=64) memory:c0000-dffff

intel_gpu_top shows:

[Mi_Key]

Somehow its back to working again utilising the P2000. i did the following, but unsure why it came good. Now on nvidia 415.13

dkms remove nvidia/415.13 -k 4.15.0-39-generic
apt purge nvidia-*
apt remove nvidia*
apt autoremove 

reboot

sudo add-apt-repository --remove ppa:graphics-drivers/ppa
add-apt-repository ppa:graphics-drivers/ppa

apt install ubuntu-drivers-common
apt update
apt upgrade
ubuntu-drivers devices
ubuntu-drivers autoinstall
reboot

Only one device is here now, rather than including the i915.

root@minibeast:~# lshw -c video         
  *-display                 
       description: VGA compatible controller
       product: GP106GL
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:155 memory:a2000000-a2ffffff memory:90000000-9fffffff memory:a0000000-a1ffffff ioport:3000(size=128) memory:c0000-dffff

root@minibeast:~# nvidia-smi 
Sun Nov 18 05:57:38 2018       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 415.13       Driver Version: 415.13       CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro P2000        Off  | 00000000:01:00.0 Off |                  N/A |
| 50%   40C    P0    18W /  75W |     72MiB /  5056MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     31526      C   /usr/lib/plexmediaserver/Plex Transcoder      62MiB |
+-----------------------------------------------------------------------------+

Nov 18, 2018 05:38:00.096 [0x7fcd793ff700] DEBUG - TPU: hardware transcoding: enabled, but no hardware decode accelerator found
Nov 18, 2018 05:38:00.096 [0x7fcd793ff700] DEBUG - TPU: hardware transcoding: zero-copy support not present
Nov 18, 2018 05:38:00.097 [0x7fcd793ff700] DEBUG - TPU: hardware transcoding: final decoder: , final encoder: nvenc