"Conversion failed" on multiple clients

[nicholush_gmail_com]

Server Version#: 1.32.0.6918
Player Version#: Web 4.100.1 / Android TV 9.20.0.806

Hardware transcoding fails - getting error “Conversion failed. The transcoder exited due to an error.” Getting this error, or similar errors, on multiple devices. This was previously working before, then the transcoding started to get very slow and
choppy. I updated the server to the latest verion, and started getting this error.

My server has and uses a quadro P400, or at least it used to. I never had problems transcoding before.

Plex Media Server Logs_2023-04-15_23-06-44.zip (4.6 MB)

[ChuckPa]

1. Your user-supplied certificate needs updating

Apr 15, 2023 22:56:17.964 [0x7f4a53d7da90] DEBUG - FeatureManager: Using cached data for features list
Apr 15, 2023 22:56:18.154 [0x7f4a53d7da90] ERROR - [CERT] PKCS12_parse failed: error:0308010C:digital envelope routines::unsupported
Apr 15, 2023 22:56:18.154 [0x7f4a53d7da90] ERROR - [CERT] Found a certificate, but couldn't install it.
Apr 15, 2023 22:56:18.154 [0x7f4a53d7da90] DEBUG - MyPlex: mapping state set to 'Unknown'.
Apr 15, 2023 22:56:18.155 [0x7f4a53d7da90] DEBUG - Relay: read 35 cached entries from hosts file
Apr 15, 2023 22:56:18.156 [0x7f4a53d7da90] DEBUG - [CERT] CERT: Forcing refresh.
Apr 15, 2023 22:56:18.159 [0x7f4a53d7da90] ERROR - [CERT] PKCS12_parse failed: error:0308010C:digital envelope routines::unsupported
Apr 15, 2023 22:56:18.159 [0x7f4a53d7da90] DEBUG - [CERT] CERT: Unable to parse existing certificate, fetching a new one.

Linux Tips

OpenSSL v3.0.0 and PMS. For those using their own domains and certificates with PMS , PMS 1.32.0.6865 and above updates OpenSSL from v1.1.1 to v3.0.0 . (changed the numbering scheme in the process) The consequence of this is OpenSSL v3 removed several “less secure” encryption methods. The impact on you, if you’re using an older distributions where openssl v3 is not the default, if not already doing so, you will need pay special attention to how your certificates are generated As example…

2. I see where you’re directing to /dev/dri/renderD129.

Apr 15, 2023 22:57:04.135 [0x7f4a4a117b38] DEBUG - [Req#383/Transcode] TPU: hardware transcoding: final decoder: , final encoder: vaapi
Apr 15, 2023 22:57:04.135 [0x7f4a4a117b38] DEBUG - [Req#383/Transcode/JobRunner] Job running: FFMPEG_EXTERNAL_LIBS='/var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Codecs/e51a01b-4528-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx46ce-ae97-538446215785 "/usr/lib/plexmediaserver/Plex Transcoder" -codec:0 h264 -codec:1 ac3 -analyzeduration 20000000 -probesize 20000000 -i "/home/nicholush/Videos/TV/Parks and Recreation/Season 1/Parks and Recreation - S01E01 - Make My Pit a Park WEBDL-1080p.mkv" -filter_complex "[0:0]scale=w=1920:h=1080:force_divisible_by=4[0];[0]format=pix_fmts=nv12[1];[1]hwupload[2]" -map "[2]" -codec:0 h264_vaapi -b:0 20000k -r:0 23.975999999999999 -force_key_frames:0 "expr:gte(t,n_forced*1)" -filter_complex "[0:1] aresample=async=1:ochl='stereo':rematrix_maxval=0.000000dB:osr=48000[3]" -map "[3]" -metadata:s:1 language=eng -codec:1 aac -b:1 256k -f dash -seg_duration 1 -dash_segment_type mp4 -init_seg_name 'init-stream$RepresentationID$.m4s' -media_seg_name 'chunk-stream$RepresentationID$-$Number%05d$.m4s' -window_size 5 -delete_removed false -skip_to_segment 1 -time_delta 0.0625 -manifest_name "http://127.0.0.1:32400/video/:/transcode/session/pptm6wy0hcc53yeeox6loc12/72ba05a8-6082-4c7f-874d-1bd14d0272fc/manifest?X-Plex-Http-Pipeline=infinite" -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -vsync cfr -init_hw_device vaapi=vaapi:/dev/dri/renderD129 -filter_hw_device vaapi -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/pptm6wy0hcc53yeeox6loc12/72ba05a8-6082-4c7f-874d-1bd14d0272fc/progress

But the output is Vaapi not Nvidia.

If it were nvidia, it would state so. (nvdec / nvenc )

what driver version is installed ?

[nicholush_gmail_com]

Thanks so much for the response!

Regarding the first…I, uh…hmm. I have no idea what’s going on there. I don’t know what user-supplied certificate is being used, I frankly don’t even have much of an idea of what this even constitutes.

The server does connect through a VPN. That’s the only thing that I can think of that might have anything to do with anything else.

When I open the “Network” section of server settings on the dashboard, I do see input boxes, for “custom certificate location”, “custom certificate encryption key” and “custom certificate domain.” All of these boxes are empty.

With the second…I’m not sure why it’s going to VAAPI. I certainly didn’t change any setting or try to set it that way. How do I set it back?

The nvidia driver is 525.105.17.

[nicholush_gmail_com]

Any other help to be had here?

[ChuckPa]

@nicholush_gmail_com

The server does connect through a VPN.

That explains it.

When PMS gets the data returned through the VPN the VPN provider’s domain and certifcate got attached along the way.

PMS is very specific here. It won’t let any unknown domain be referenced in connections.

Historically, VPS and PMS never work well.

[nicholush_gmail_com]

@ChuckPa

That explains it.

When PMS gets the data returned through the VPN the VPN provider’s domain and certifcate got attached along the way.

PMS is very specific here. It won’t let any unknown domain be referenced in connections.

Historically, VPS and PMS never work well.

OK. Turned off the VPN, restarted the machine. Same error

I’m confused as to why it is trying to use the VAAPI instead of nvidia. My processor is a 4th gen Intel without quicksync.

Plex Media Server Logs_2023-05-02_23-51-49.zip (4.5 MB)

[ChuckPa]

This is very much an odd one. Thanks for making my day interesting

1. The CPU is an i3-4xxx which does have QSV capability (1080p only)
2. Looking at your logs,
   – It finds both capabilities (Nvidia and QSV)
   – It validates the Nvidia
   – It selects the Nvidia for decode
   – It selects QSV for the encode (incorrectly) when it should have selected the Nvidia because that CPU does not have sufficient OpenCL capability.

May 02, 2023 23:48:23.089 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android] [FFMPEG] - Format 0x30313050 -> p010le.
May 02, 2023 23:48:23.089 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android] [FFMPEG] - Created surface 0x4000000.
May 02, 2023 23:48:23.089 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android] [FFMPEG] - Direct mapping possible.
May 02, 2023 23:48:23.089 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android] TPU: hardware transcoding: final decoder: nvdec, final encoder: vaapi
May 02, 2023 23:48:23.090 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android/JobRunner] Job running: EAE_ROOT=/tmp/pms-8233ef27-9f14-433b-b691-82e5c8ce21aa/EasyAudioEncoder FFMPEG_EXTERNAL_LIBS='/var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Codecs/e51a01b-4528-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4592-8a15-9bc64eca217b "/usr/lib/plexmediaserver/Plex Transcoder" -codec:0 hevc -hwaccel:0 nvdec -hwaccel_fallback_threshold:0 10 -threads:0 1 -hwaccel_device:0 cuda -codec:1 truehd_eae -eae_prefix:1 8c85087b4294f9cc-com-plexapp-android_ -analyzeduration 20000000 -probesize 20000000 -i "/home/nicholush/Videos/Movies/Dune (2021)/Dune (2021) Remux-2160p.mkv" -filter_complex "[0:0]scale=w=3840:h=2160:force_divisible_by=4[0];[0]format=p010,tonemap=mobius[1];[1]format=pix_fmts=nv12[2];[2]hwupload[3]" -map "[3]" -metadata:s:0 language=eng -codec:0 h264_vaapi -b:0 62208k -maxrate:0 82944k -bufsize:0 165888k -r:0 23.975999999999999 -force_key_frames:0 "expr:gte(t,n_forced*1)" -filter_complex "[0:1] aresample=async=1:ochl='7.1':rematrix_maxval=0.000000dB:osr=48000[4]" -map "[4]" -metadata:s:1 language=eng -codec:1 libopus -b:1 1024k -segment_format matroska -f ssegment -individual_header_trailer 0 -flags +global_header -segment_header_filename header -segment_time 1 -segment_start_number 0 -segment_copyts 1 -segment_time_delta 0.0625 -segment_list "http://127.0.0.1:32400/video/:/transcode/session/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c/manifest?X-Plex-Http-Pipeline=infinite" -segment_list_type csv -segment_list_size 5 -segment_list_separate_stream_times 1 -segment_list_unfinished 1 -segment_format_options output_ts_offset=10 -max_delay 5000000 -avoid_negative_ts disabled -map_metadata:g -1 -map_metadata:c -1 -map_chapters -1 "media-%05d.ts" -start_at_zero -copyts -vsync cfr -init_hw_device vaapi=vaapi:/dev/dri/renderD129 -filter_hw_device vaapi -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c/progress
May 02, 2023 23:48:23.090 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android/JobRunner] In directory: "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions/plex-transcode-8c85087b4294f9cc-com-plexapp-android-9151ba3c-d4af-4680-9b69-89fd6e17ec0c"
May 02, 2023 23:48:23.090 [0x7f9d8325cb38] DEBUG - [Req#701b/Transcode/8c85087b4294f9cc-com-plexapp-android/JobRunner] Jobs: Starting child process with pid 600263
May 02, 2023 23:48:23.101 [0x7f9d8de02b38] DEBUG - Request: [127.0.0.1:44680 (Loopback)] PUT /video/:/transcode/session/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c/progress?status=startup (15 live) #67ac Signed-in Token (nicho555) (range: bytes=0-) 
May 02, 2023 23:48:23.102 [0x7f9d908acb38] DEBUG - Completed: [127.0.0.1:44680] 204 PUT /video/:/transcode/session/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c/progress?status=startup (15 live) #67ac 0ms 203 bytes (pipelined: 1) (range: bytes=0-) 
May 02, 2023 23:48:23.109 [0x7f9d8de02b38] ERROR - [Req#704d/Transcode/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c] [AVHWDeviceContext @ 0x7fb0c1915e00] Failed to initialise VAAPI connection: -1 (unknown libva error).
May 02, 2023 23:48:23.110 [0x7f9d8de02b38] ERROR - [Req#7052/Transcode/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c] Device creation failed: -5.
May 02, 2023 23:48:23.110 [0x7f9d8de02b38] ERROR - [Req#7055/Transcode/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c] Failed to set value 'vaapi=vaapi:/dev/dri/renderD129' for option 'init_hw_device': I/O error
May 02, 2023 23:48:23.110 [0x7f9d8de02b38] ERROR - [Req#7057/Transcode/8c85087b4294f9cc-com-plexapp-android/9151ba3c-d4af-4680-9b69-89fd6e17ec0c] Error parsing global options: I/O error
May 02, 2023 23:48:23.111 [0x7f9d90b25b38] DEBUG - Jobs: '/usr/lib/plexmediaserver/Plex Transcoder' exit code for process 600263 is 1 (failure)
May 02, 2023 23:48:24.480 [0x7f9d908acb38] DEBUG - Auth: authenticated user 201271834 as NickKim

Notice how it’s attempting to use the QSV for tonemapping and fails (as it will).
However – It shouldn’t be attempting to use it

The logs now show it attempting VAAPI on the Nvidia (/dev/dri/renderD129) which will obviously fail.

My first questions are:

1. Are all the nvidia drivers loaded, specifically nvidia-decode and nvidia-encode ?
2. Do those versions match the main Nvidia drivers which are installed ?
   dpkg -l | grep nvidia

[nicholush_gmail_com]

@ChuckPa

This is very much an odd one. Thanks for making my day interesting

Happy to be of assistance! I really appreciate the help.

1. Are all the nvidia drivers loaded, specifically nvidia-decode and nvidia-encode ?
2. Do those versions match the main Nvidia drivers which are installed ?
   dpkg -l | grep nvidia

I think so…?

…but maybe not? I don’t know. Clearly, I’m not an expert.

[ChuckPa]

Thank you for providing the listing.

You have two sets of invidia drivers installed (515 and 525). You only should have one.

Once the libnvidia-encode is working correctly, it won’t try to tonemap on a CPU which is incapable.

Here are my Nvidia drivers, inclusive of development tools (you can ignore).

[chuck@lizum ~.2000]$  dpkg -l | grep nvidia
ii  gpustat                               0.6.0-1                           all          pretty nvidia device monitor
ii  libnvidia-cfg1-525-server:amd64       525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-525-server           525.105.17-0ubuntu0.20.04.1       all          Shared files used by the NVIDIA libraries
ii  libnvidia-compute-525-server:amd64    525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA libcompute package
ii  libnvidia-decode-525-server:amd64     525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-encode-525-server:amd64     525.105.17-0ubuntu0.20.04.1       amd64        NVENC Video Encoding runtime library
ii  libnvidia-extra-525-server:amd64      525.105.17-0ubuntu0.20.04.1       amd64        Extra libraries for the NVIDIA Server Driver
ii  libnvidia-fbc1-525-server:amd64       525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-525-server:amd64         525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-ml-dev                      10.1.243-3                        amd64        NVIDIA Management Library (NVML) development files
ii  nvidia-compute-utils-525-server       525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA compute utilities
ii  nvidia-cuda-dev                       10.1.243-3                        amd64        NVIDIA CUDA development files
ii  nvidia-cuda-doc                       10.1.243-3                        all          NVIDIA CUDA and OpenCL documentation
ii  nvidia-cuda-gdb                       10.1.243-3                        amd64        NVIDIA CUDA Debugger (GDB)
ii  nvidia-cuda-toolkit                   10.1.243-3                        amd64        NVIDIA CUDA development toolkit
ii  nvidia-dkms-525-server                525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA DKMS package
ii  nvidia-driver-525-server              525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA Server Driver metapackage
ii  nvidia-kernel-common-525-server       525.105.17-0ubuntu0.20.04.1       amd64        Shared files used with the kernel module
ii  nvidia-kernel-source-525-server       525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA kernel source package
ii  nvidia-opencl-dev:amd64               10.1.243-3                        amd64        NVIDIA OpenCL development files
ii  nvidia-prime                          0.8.16~0.20.04.2                  all          Tools to enable NVIDIA's Prime
ii  nvidia-profiler                       10.1.243-3                        amd64        NVIDIA Profiler for CUDA and OpenCL
ii  nvidia-utils-525-server               525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA Server Driver support binaries
ii  nvidia-visual-profiler                10.1.243-3                        amd64        NVIDIA Visual Profiler for CUDA and OpenCL
ii  xserver-xorg-video-nvidia-525-server  525.105.17-0ubuntu0.20.04.1       amd64        NVIDIA binary Xorg driver
[chuck@lizum ~.2001]$ 

Please clean out your older drivers and the i386 modules. Everything is 64 bit.

Don’t forget to update the initram-fs when done.

[nicholush_gmail_com]

Continuing the discussion from "Conversion failed" on multiple clients:

Thanks for walking through this with me.

I uninstalled as much as I could…

Restarted, and installed a fresh set of nvidia drivers - (sudo apt install nvidia-driver-530)

Getting the same error / behavior.

Plex Media Server Logs_2023-05-05_00-10-28.zip (2.8 MB)

For what it’s worth, I do run a Jellyfin server side-by-side with Plex, and the hardware transcoding is currently working. Transcoded video plays fine, and there is an ffmpeg process running in nvidia-smi when it is playing. Not trying to start ■■■■, or whatever, just trying to provide diagnostic information.

[ChuckPa]

That output isn’t right. un should not be listed.

dpkg -l | grep nvidia

is what you want.

Install this. It’s an Engineer’s build with the updated drivers installed.

plexmediaserver_1.32.2.6997-416face2b_amd64.deb

What bothers me – It knows /dev/dri/renderD129 is the nvidia but trying to test as VAAPI.

Do you have libva manually installed ?

Don’t remember asking but is this in a VM ?

In your logs, this is where it goes wrong.

May 05, 2023 00:05:04.873 [0x7f6a39f4db38] DEBUG - [Req#bc3/Transcode] Starting a transcode session 1fskf6hms9wpp19sx2htcr3o at offset -1.0 (state=3)
May 05, 2023 00:05:04.874 [0x7f6a39f4db38] DEBUG - [Req#bc3/Transcode] TPU: hardware transcoding: using hardware decode accelerator nvdec
May 05, 2023 00:05:04.874 [0x7f6a39f4db38] DEBUG - [Req#bc3/Transcode] TPU: hardware transcoding: zero-copy support not present

zero-copy should be present. I don’t know if a card limitation or what.
How much memory?

[nicholush_gmail_com]

I don’t recall installing libva manually. This machine serves almost exclusively as a Plex server, I can’t think of any reason I would have installed that manually.

Nothing is running through a VM.

I will install the Engineer’s build and report back. Thanks.

[ChuckPa]

@nicholush_gmail_com

I’ve done some research about the Nvidia cards .

Look at the P400 .

Video Encode and Decode GPU Support Matrix

According to the matrix, you can transcode HEVC 4:2:0 encoded but not HEVC 4:4:4

Check the files you’re using.