Hardware encoding fails for several attempts then works

Plex Media Server
1

Server Version#: Version 1.32.0.6973
Ubuntu 22.04
Running on ESX 70U3
Player MS Edge#:
NVIDIA-SMI 515.105.01 Driver Version: 515.105.01 CUDA Version: 11.7

I’m testing an install of Plex on ubuntu with an NVIDIA 620 so that i do hardware encoding on a smaller form esxi server.

Ive successfully got this going on my native install on a NUC, but ive now got a video card and want to move plex into my esxi server.

Ive followed steps found online which range from a couple of config entries on the esxi VMX file to pass through the GPU, to full on block built in nivida drivers and install only v515 and many other hacks as well.
Where i’m at is that it will do hardware encoding but it will take a few internal attempts at checking the API vaapi and getting a few failures, before it streams. so it takes maybe 20 seconds to start streaming.
On my current production plex, there are no failed attempts with the onboard intel card.
Ive found an entry that sets the preferred video card in the plex XML which ive set, but that makes no difference either way

Any thoughts?

Six of these:

root@plex2:~# tail -f “/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Logs/Plex Media Server.log” | grep hardware
May 06, 2023 15:17:20.066 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:20.066 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:20.066 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for h264_vaapi
May 06, 2023 15:17:20.067 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:20.792 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:20.792 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:20.792 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:20.793 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:21.216 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:21.629 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:21.629 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:21.629 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:21.630 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:21.838 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:22.174 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:22.347 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:22.347 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:22.347 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:22.348 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:22.791 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:22.939 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:22.939 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:22.939 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:22.940 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:23.340 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:23.501 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:23.501 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:23.501 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:23.502 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:23.900 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:24.064 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:24.064 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:24.064 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:24.065 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:24.336 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:24.336 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:24.336 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:24.337 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:24.340 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:24.807 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:25.014 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:25.014 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:25.014 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:25.015 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:25.222 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:25.222 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:25.222 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:25.223 [0x7fb9162f6b38] DEBUG - [Req#1e7/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:25.356 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:25.838 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:25.838 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:25.838 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:25.839 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:26.047 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:26.279 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:26.279 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:26.279 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:26.279 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:26.487 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvenc
May 06, 2023 15:17:26.719 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 15:17:26.719 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 15:17:26.719 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Could not create hardware context for hevc
May 06, 2023 15:17:26.720 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] Codecs: hardware transcoding: testing API nvdec
May 06, 2023 15:17:26.927 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] TPU: hardware transcoding: using hardware decode accelerator nvdec
May 06, 2023 15:17:26.927 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] TPU: hardware transcoding: zero-copy support present
May 06, 2023 15:17:26.927 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] TPU: hardware transcoding: using zero-copy transcoding
May 06, 2023 15:17:26.932 [0x7fb9156e4b38] DEBUG - [Req#1f4/Transcode] TPU: hardware transcoding: final decoder: nvdec, final encoder: nvenc

2

Sorry, but it’s not possible to diagnose that grep output.

Will need to see the full logs ZIP file and all the entire HW probe process.

3

Thank you for taking the time to answer this chuck
I think these are what you’re after from the HW probe;
lspci;

00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (                                                                                                                                               rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (re                                                                                                                                               v 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 1                                                                                                                                               0)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ul                                                                                                                                               tra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)

**surplius PCI briges removed here****

00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:02.0 SATA controller: VMware SATA AHCI controller
03:00.0 VGA compatible controller: NVI
[Plex Media Server Logs_2023-05-06_18-41-43.zip|attachment](upload://8iqAvu0tGTtnOELB5f95V1qE58R.zip) (892.7 KB)
DIA Corporation GP107GL [Quadro P620] (rev                                                                                                                                                a1)
03:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controlle                                                                                                                                               r (rev a1)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: VMware USB3 xHCI 1.0 Controller

image
image654×765 61.2 KB

Ive attempted several videos during the capture of these zip files.
all eventually used hardware encoding and all suffered the same
error several times;

May 06, 2023 18:36:57.346 [0x7f60aaf87b38] DEBUG - [Req#6f/Transcode] Codecs: hardware transcoding: testing API vaapi
May 06, 2023 18:36:57.346 [0x7f60aaf87b38] ERROR - [Req#6f/Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
May 06, 2023 18:36:57.346 [0x7f60aaf87b38] ERROR - [Req#6f/Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
May 06, 2023 18:36:57.346 [0x7f60aaf87b38] DEBUG - [Req#6f/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
May 06, 2023 18:36:57.346 [0x7f60aaf87b38] DEBUG - [Req#6f/Transcode] Could not create hardware context for hevc
May 06, 2023 18:36:57.346 [0x7f60aaf87b38] DEBUG - [Req#6f/Transcode] Codecs: testing hevc (decoder) with hwdevice nvdec
May 06, 2023 18:36:57.347 [0x7f60aaf87b38] DEBUG - [Req#6f/Transcode] Codecs: hardware transcoding: testing API nvdec```
4

@bashers

May I have the ZIP logs please?

I’m trying to help but I need to look at everything. What you’re presenting are only tiny snippets of what I look at.

The best logs ZIP for me is:

  1. Restart PMS
  2. Wait 60-ish seconds for it to stabilize
  3. Start playback
  4. Play for 20 seconds
  5. Stop playback
  6. Wait 15 seconds more for logs to settle
  7. Download the logs (ZIP file)
  8. Attach that ZIP file here for me.

This will let me see what PMS sees (hardware at startup), the probe process, and all intermediate output from the time PLAY is commanded until stopped/fails.

5

Sorry, i attached them but there not here.
Lets do that again
Plex Media Server Logs_2023-05-06_18-41-43.zip (892.7 KB)

6

@bashers

I’m going to suggest to you as I have to others over the past few days.

Please try reverting to 1.29.2.6364. I can provide the DEB file if you don’t have it.

I’m trying to identify the root cause while Engineering is away but haven’t so far.

The symptom i’m seeing is common:

  1. Hardware probe is successful
  2. Playback starts
  3. Tonemapping memory (CUDA) fails to setup and start correctly. (bug)
  4. Entire process falls back to Software DECODING (incorrect behavior)

I figured this out using Plex/web and the iOS client.
PMS 1.29.2.6364 was the only solution.

7

Ive installed the version 1.29 you recommended, but I’m still seeing 6 failures before successful testing of NVDEC API
.
I hoped this might also be fixed as ive read that installing PLEX after GPU drivers is more successful than the other way round.

Plex Media Server Logs_2023-05-07_08-37-35.zip (1.9 MB)

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.