Hardware transcoding issue

Hi Chuck,

This is great information! I didn’t know it was possible to execute the DEB file within the container. I’ll go ahead and do that and respond on this thread with my results in testing.

For everyones information, I have dual Xeon 2670 CPU’s, with 128gb RAM, and a GTX 1070 for hardware transcoding.

Thank you,
Shehwaz

I’ve already been beating on it too.

2690 v4, 256 GB, and P2200

and even a NUC3i7BNK haha

I hope this helps someone until the version that fixes this issue is released, I realized that if I set my transcoder quality to “Prefer higher speed encoding” I don’t encounter any issues in my transcoding.

image

1 Like

Hmm, I tried it now (with version 1.31. and 1.31.2) with Chrome and Edge and somehow it behaves really weirdly now.

Starting a file in Edge or Chrome works depending on the source but changing the quality does not work. It’s stuck with a black screen. I monitored the behavior of the GPU with nvtop while doing this. When changing the quality a second transcoding process is generated (as I would expect) - the initial transcoding process is killed then, the new process survives a second and then dies => In the Plex console I could see this =>
Jobs: '/usr/lib/plexmediaserver/Plex Transcoder' exit code for process 11591 is -9 (signal: Killed)
I have no idea if this “kills” the old initial quality transcode or the new one… Sometimes out of nowhere the transcode comes back up and the video continues to play just fine… Chrome and Edge behave the same in this regard. (except hevc files)

(file ac3 audio and no subs)
Chrome:
1080p@35mbit → 1080p@12mbit: not working
2160p@70mbit → 1080p@12mbit: not working

Edge:
1080p@35mbit → 1080p@12mbit: not working
2160p@70mbit → 1080p@12mbit: directly starts as a transcode but quality change not working (behavior as described above)

I then tried the Plex for Desktop client as it was more reliable in testing… Most of the time changing the quality works fine but with the exception of 2160p files =>

Plex for Desktop
1080p@35mbit → 1080p@12mbit: working + quality change also working (almost instant!!!)
2160p@70mbit → 1080p@12mbit: not working (well, working for a second but then the video collapses like the transcoder process does)

Plex.log (1,0 MB)
Plex Media Server Logs_2023-02-18_13-34-56.zip (3,8 MB)

1 Like

Looks like the “kill” is from a an old “test” session

Feb 18, 2023 13:32:16.282 [0x7fe82ab4db38] DEBUG - [Req#147e2/Transcode] TPU: hardware transcoding: final decoder: nvdec, final encoder: nvenc
Feb 18, 2023 13:32:16.283 [0x7fe82ab4db38] DEBUG - [Req#147e2/Transcode/JobRunner] Job running: CUDA_CACHE_PATH="/mnt/appdata1/Library/Application Support/Plex Media Server/Cache/Shaders/CUDA" FFMPEG_EXTERNAL_LIBS='/mnt/appdata1/Library/Application\ Support/Plex\ Media\ Server/Codecs/2584b51-4483-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4da2-a71e-fcb2db0fd60d "/usr/lib/plexmediaserver/Plex Transcoder" -codec:0 hevc -hwaccel:0 nvdec -hwaccel_fallback_threshold:0 10 -threads:0 1 -hwaccel_output_format:0 cuda -hwaccel_device:0 cuda -codec:2 ac3 -ss 209 -analyzeduration 20000000 -probesize 20000000 -i "/mnt/array/Filme/4K/Three Thousand Years of Longing/Three Thousand Years of Longing (2022)_4K.mkv" -filter_complex "[0:0]hwupload[0];[0]scale_cuda=w=1920:h=1080:format=p010[1];[1]tonemap_cuda=mobius:nv12[2]" -map "[2]" -metadata:s:0 language=eng -codec:0 h264_nvenc -b:0 8184k -maxrate:0 10912k -bufsize:0 21824k -forced-idr:0 1 -r:0 23.975999999999999 -filter_complex "[0:2] aresample=async=1:ochl='5.1':rematrix_maxval=0.000000dB:osr=48000[3]" -map "[3]" -metadata:s:1 language=deu -codec:1 libopus -b:1 444k -f segment -segment_format matroska -segment_format_options live=1 -segment_time 1 -segment_header_filename header -segment_start_number 0 -segment_list "http://127.0.0.1:32400/video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/manifest?X-Plex-Http-Pipeline=infinite" -segment_list_type csv -segment_list_unfinished 1 -segment_list_size 5 -segment_list_separate_stream_times 1 -avoid_negative_ts disabled -map_metadata:g -1 -map_metadata:c -1 -map_chapters -1 "chunk-%05d" -start_at_zero -copyts -init_hw_device cuda=cuda: -filter_hw_device cuda -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress
Feb 18, 2023 13:32:16.283 [0x7fe82ab4db38] DEBUG - [Req#147e2/Transcode/JobRunner] In directory: "/mnt/filedump1/transcode/Transcode/Sessions/plex-transcode-m7o1y828vjowfj6dhk3k1me0-e5891345-3645-4251-bb55-153eb895ff07"
Feb 18, 2023 13:32:16.283 [0x7fe82ab4db38] DEBUG - [Req#147e2/Transcode/JobRunner] Jobs: Starting child process with pid 18002
Feb 18, 2023 13:32:16.288 [0x7fe82a747b38] DEBUG - Request: [127.0.0.1:60098 (Loopback)] PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=startup (15 live) #147ec Signed-in Token (Mitzsch) (range: bytes=0-) 
Feb 18, 2023 13:32:16.289 [0x7fe8301f6b38] DEBUG - Completed: [127.0.0.1:60098] 204 PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=startup (15 live) #147ec 0ms 203 bytes (pipelined: 1) (range: bytes=0-) 
Feb 18, 2023 13:32:16.516 [0x7fe82b18eb38] DEBUG - Request: [127.0.0.1:60098 (Loopback)] PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=startup (15 live) #147f1 Signed-in Token (Mitzsch) (range: bytes=0-) 
Feb 18, 2023 13:32:16.516 [0x7fe82fff3b38] DEBUG - Completed: [127.0.0.1:60098] 204 PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=startup (15 live) #147f1 0ms 203 bytes (pipelined: 2) (range: bytes=0-) 
Feb 18, 2023 13:32:16.516 [0x7fe82cc7bb38] DEBUG - Request: [127.0.0.1:60098 (Loopback)] PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=opening (15 live) #147f2 Signed-in Token (Mitzsch) (range: bytes=0-) 
Feb 18, 2023 13:32:16.517 [0x7fe82fff3b38] DEBUG - Completed: [127.0.0.1:60098] 204 PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=opening (15 live) #147f2 0ms 203 bytes (pipelined: 3) (range: bytes=0-) 
Feb 18, 2023 13:32:16.518 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] Codecs: testing hevc (decoder) with hwdevice vaapi
Feb 18, 2023 13:32:16.518 [0x7fe82b4aab38] DEBUG - Request: [127.0.0.1:60098 (Loopback)] PUT /video/:/transcode/session/m7o1y828vjowfj6dhk3k1me0/e5891345-3645-4251-bb55-153eb895ff07/progress?status=opened (15 live) #147f3 Signed-in Token (Mitzsch) (range: bytes=0-) 



Feb 18, 2023 13:32:17.775 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] Codecs: dummy-frame test succeeded
Feb 18, 2023 13:32:17.812 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] MDE: Three Thousand Years of Longing (2022): Audio Direct Streaming is disabled, so video's audio stream will be transcoded
Feb 18, 2023 13:32:17.812 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] MDE: Cannot direct stream audio stream due to profile or setting limitations
Feb 18, 2023 13:32:17.812 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] MDE: Three Thousand Years of Longing (2022): selected media 0 / 21189
Feb 18, 2023 13:32:17.812 [0x7fe82b6adb38] DEBUG - [Req#147dc/Transcode] Killing job.
Feb 18, 2023 13:32:17.812 [0x7fe82b6adb38] DEBUG - [Req#147dc/Transcode] Signalling job ID 18002 with 9
Feb 18, 2023 13:32:17.813 [0x7fe82b8b0b38] DEBUG - [Req#147dc/Transcode] Streaming Resource: Reached Decision id=10488 codes=(General=1001,Direct play not available; Conversion OK. Direct Play=3000,App cannot direct play this item. Direct play is disabled. Transcode=1001,Direct play not available; Conversion OK.) media=(id=21189 part=(id=23554 decision=transcode container=mkv protocol=http streams=(Video=(id=55899 decision=transcode bitrate=10912 encoder=h264_nvenc width=1920 height=1080) Audio=(id=55901 decision=transcode bitrate=444 encoder=libopus channels=6 rate=48000))))
Feb 18, 2023 13:32:17.813 [0x7fe82b6adb38] DEBUG - [Req#147dc/Transcode] Job was already killed, not killing again.
Feb 18, 2023 13:32:17.813 [0x7fe82b6adb38] DEBUG - [Req#147dc/Transcode] Stopping transcode session m7o1y828vjowfj6dhk3k1me0
Feb 18, 2023 13:32:17.813 [0x7fe80ffffb38] DEBUG - [Req#147dc/Transcode] Cleaning directory for session m7o1y828vjowfj6dhk3k1me0 (/mnt/filedump1/transcode/Transcode/Sessions/plex-transcode-m7o1y828vjowfj6dhk3k1me0-e5891345-3645-4251-bb55-153eb895ff07)
Feb 18, 2023 13:32:17.814 [0x7fe82b6adb38] DEBUG - [Req#147dc/Transcode] Transcoder: Cleaning old transcode directories.



Feb 18, 2023 13:32:17.816 [0x7fe82b4aab38] DEBUG - [Req#14801] [Now] Adding 1 sessions.
Feb 18, 2023 13:32:17.818 [0x7fe8301f6b38] DEBUG - Completed: [192.168.1.110:55534] 200 GET /status/sessions (14 live) #14802 TLS GZIP 1ms 4728 bytes (pipelined: 25)
Feb 18, 2023 13:32:17.818 [0x7fe8301f6b38] DEBUG - Completed: [192.168.1.110:55547] 200 GET /status/sessions (14 live) #14801 TLS GZIP 2ms 4728 bytes (pipelined: 15)
Feb 18, 2023 13:32:17.869 [0x7fe8303f9b38] DEBUG - Jobs: '/usr/lib/plexmediaserver/Plex Transcoder' exit code for process 18002 is -9 (signal: Killed)
Feb 18, 2023 13:32:17.869 [0x7fe827fbeb38] DEBUG - Notifying consumer that data is complete.
Feb 18, 2023 13:32:17.870 [0x7fe82a94ab38] DEBUG - [TranscodeOutputStream] Input processing thread exited after writing 590579 bytes, m_closed=0, m_endOfFileReached=0, session->isStopped()=1
Feb 18, 2023 13:32:17.870 [0x7fe82a94ab38] DEBUG - Cleaning directory for session m7o1y828vjowfj6dhk3k1me0 (/mnt/filedump1/transcode/Transcode/Sessions/plex-transcode-m7o1y828vjowfj6dhk3k1me0-e5891345-3645-4251-bb55-153eb895ff07)
Feb 18, 2023 13:32:17.900 [0x7fe8301f6b38] ERROR - [TranscodeOutputStream] Session appears to have died from under us
Feb 18, 2023 13:32:18.150 [0x7fe82ab4db38] DEBUG - Request: [192.168.1.110:55534 (Allowed Network (Subnet))] GET /:/timeline?ratingKey=10488&key=%2Flibrary%2Fmetadata%2F10488&playbackTime=5098&playQueueItemID=25413&state=stopped&hasMDE=1&context=home%3Ahub.continueWatching&row=0&col=0&time=209000&duration=6534000 (12 live) #14814 TLS GZIP Signed-in Token (Mitzsch) (HomePC)
Feb 18, 2023 13:32:18.151 [0x7fe82ab4db38] DEBUG - [Req#14814] Client [a0hegwppzhlbs7amzmftbq6q] reporting timeline state stopped, progress of 209000/6534000ms for guid=, playbackTime=5098ms ratingKey=10488 url=, key=/library/metadata/10488, containerKey=, metadataId=10488, source=
Feb 18, 2023 13:32:18.155 [0x7fe82ab4db38] DEBUG - [Req#14814] Play progress on 10488 'Three Thousand Years of Longing' - got played 209000 ms by account 1!
Feb 18, 2023 13:32:18.165 [0x7fe82ab4db38] DEBUG - [Req#14814] It took 10.000000 ms to retrieve 164 items.

I am not surprised.

I have yet to see any PMS version + any Xeon CPU + any NVIDIA where changing quality works in Chrome.

Is this just all a known issue then? I’ve upgraded the plexinc/pms-docker image within the docker container to v1.31.1.6716 using the deb provided and I still cannot transcode anything using Plex for Windows client / Android client and Google Chrome. I bought the GPU as transcoding stopped for me months ago (I barely noticed as I direct play locally most of the time) but it seems like that’s just compounded the problem.

Specs below (edited to add kernel / CUDA versions):

Plex: v1.31.2.6715
OS: UnRAID v6.11.5
Kernel: 5.19.17
CPU: Intel Xeon CPU E3-1265L V2
GPU: Nvidia P400
GPU Driver: v525.89.02
CUDA Version: 12.0

Today I did testing with yet another friend’s equipment. Per usual, changing quality in Chrome does not work on the Xeon machine, works on everything else.

For ALL:
PMS: 1.31.0.6654-02189b09f
GPU: NVIDIA

Server 1
CPU: Xeon E5-2683 v3 (Intel® Xeon® Processor E5-2683 v3)
Motherboard: X10DRU-i+ (X10DRU-i+ | Motherboards | Products | Super Micro Computer, Inc.)
Results: Fails

Server 2
CPU: Intel Core i7-13700K
Results: Works

Server 3
CPU: AMD Ryzen 9
Results: Works

I find it unlikely that Plex would do hardware fingerprinting to try and artificially limit capabilities, the likely culprit is the Nvidia driver is either behaving differently on Xeons or there is some instruction set difference between consumer and enterprise CPUs that is triggering something. This is definitely a very weird issue!

3 Likes

FOLKS:

May I ask why folks continuing to test with anything other than

and

The fixes/changes made for Nvidia are in this engineering build and now on their way to QA & being released.

For those with LinuxServer and Plex Docker, You can use DPKG and install this in your existing container at the command line

We will see these changes in the next full build of PMS (likely next week?) with 1.31.2 or proper 1.32.x – I don’t know what the new version number will be)

Hi Chuck,

I did some testing today and all the files that were previously failing to run are now running appropriately!

Thank you,
Shehwaz

I’m continuing to try and understand this mess

As I always feared, the kernel and kernel drivers are a big part of the root cause.

My machine doesn’t have a problem whatsoever with Nvidia P2200 and CUDA 12.0 on PMS.

Many others do

I am running Ubuntu 20.04.5 LTS on the machine – DEFAULT 5.4.0-139 KERNEL!!!

Here’s an example which is also the one that got me thinking.

My Question to all

  1. How many are running a kernel version greater than 5.10 and have problems with CUDA 12.0 ?

  2. What is the DEFAULT kernel for your distro at this time?

Hi Chuck,

I’m running Ubuntu 22.04 with Plex running in the official Plex docker image (Plex-inc/PMS).

My kernel version is: 5.15.0-60-generic

My Nvidia info: NVIDIA-SMI 525.78.01 Driver Version: 525.78.01 CUDA Version: 12.0

Video card: GTX 1070

CPU: Intel 2670 v2

With the latest build you provided, 1.31.2.6715, everything is working as it should now for me at least.

@Shehwaz1

Thanks.

From trying to take in all the reports
+
All the changes to the kernel across the 5.x releases
+
All the i915 driver changes to support video

I am coming to a conclusion of sorts.

I don’t have a final recommendation as of yet but GUC and HUC are important here.
Once I understand it better, it’s my hope we can banter it back & forth to iterate on a solution which will work for all the kernels/drivers from 5.04 and above.

It is understood, having chatted with the engineer, the update coming in the next release of PMS is the formalized implementation of the patch we’re testing in build 6715

I will know more when the upcoming release is in Alpha test (I can grab that early and test drive)

For what its worth I’m running Ubuntu 22.04.2 LTS, i7-9750H CPU, RTX 2080 Mobile GPU (525.78.01 drivers) and kernel updated to 5.19.0-32 in the past week or so as the new default.

I’ve no problems with HW transcoding BTW …

OpenSuse Leap 15.4:

Defualt Kernel: 5.14.21-150400.24.46-default

Havent tested plexmediaserver-1.31.2.6715-ac8ee76c0.x86_64.rpm yet

@10Seconds

I’ve seen where 5.15 works for VAAPI (QSV) on my desktop.
It will be interesting if it’s still good for your machine.

So I just tested plexmediaserver-1.31.2.6715-ac8ee76c0.x86_64.rpm and my server seems to work just fine now. It has an AMD 3900X processor and GTX 1660 Ti. So the intel stuff wouldn’t really apply to it but that patch did fix the nvidia problems it appears.

I am also running nvidia driver 525.89.02 for reference.

You just verified.

  1. The fix
  2. Nvidia GPU
  3. 5.14 kernel
  4. Nvidia 525 drivers (CUDA 12.0 base)

Thanks.

One correction. I have a 5.14 kernel. I can’t quite verify 5.15. Not sure if there are really any major differences there but just wanted to point that out.

1 Like