Converting to a lower quality fails with Hardware-Accelerated Streaming in Plex Web

@ChuckPa,

Thank you for taking the time to do that. I know the logs are very lengthy.
Generally the subtitles are not being burned in. When they are (SyncLounge Player) I will start to see very high CPU load across the system’s 24 threads. I’m not testing media with subtitles to burn currently as I know that presents its own issues. The full length Sintel movie does not force subtitles.

I was just inspired to try something else. I am able to use optimized versions to test the GPU conversion as well. It works flawlessly.
image

On software, I only get <5.x speed with very high load on the system



nvidia smi not populating
image

On hardware, I get 15x+ speed with nvidia-smi populating.


low system load (probably audio transcoding driving cores 1,13 up)

nvidia-smi populating
image

Optimized version plays back flawlessly.

My problem seems to lie in live transcoding on web based Plex Players.

Thoughts?

1 Like

Steps I’ve taken since last update

  • blacklisted integrated controller mgag200 (lshw showed unclaimed pci display device)
    • completely disabled the integrated graphics controller in bios afterwards
      • lshw only shows nvidia now
  • stress test GPU with gpu_burn
    • (10 minutes, 100% utilization, 4.5GB/5GB mem used, 0 errors, 71C max temp)
  • connected a DP=>HDMI adapter to the GPU (shouldn’t need a dummy/display, but just in case)
  • Tested on multiple players (web based ones fail ~30-50% of the time)

To answer your question, yes I still see messages like

[0x7fb59effd700] DEBUG - TPU: hardware transcoding: final decoder: , final encoder:

But I only see it on plex web based players
evidence-Plex-Media-server.log (501.5 KB) ,
never comes up when I transcode on Android.

(Tested by cycling transcoder presets while tailing log)
tail -F Plex\ Media\ Server.log | grep '\ \,'

Are there any other things I can try?
Is there a path of escalation for support @ChuckPa ? Or are the forums the only way?

1 Like

The logs you’re showing me keep reinforcing the Nvidia is not up to the task.

what does /dev/dri look like ?

How can it not be up to the task? This is the part I don’t understand.

$ ls -ls /dev/dri
total 0
0 drwxr-xr-x 2 root root        80 Jul 29 12:51 by-path
0 crw-rw---- 1 root video 226,   0 Jul 29 12:51 card0
0 crw-rw---- 1 root video 226, 128 Jul 29 12:51 renderD128

Here it is doing 9 simultaneous transcodes on synclounge.

https://imgur.com/a/LwN3UG3

And here it is failing to transcode 1 quality change on Plex web

1 Like

Now I’m confused.

Even with your video sample, I can’t reproduce the failure in Plex/web. I’ve tried on FireFox and Chrome.

I’m sorry but I have a NUC8 (SkullCanyon) which is smaller than most Nvidia cards :confused:

@ChuckPA,

Wouldn’t this indicate some problem with the Media Decision Engine?
I tried the fixes / startup parameters people claim worked for them

  • kernel 5.7.8 (current kernel)
  • kernel 5.8
  • kernel 5.4
  • kernel 4.4
  • nvidia-440
  • nvidia-450.51
  • nvidia-450.57 (current driver)

I haven’t tried nvidia-335.21 yet, but since this seems to be specifically related to Plex Web, I’m not sure where to go here.

I wrote a Python shim that intercepts Plex Transcoder just so I could render out the transcoder flags to a shell script and call the Transcoder as a child process - it works, but it is not very enlightening.

Transcode success on synclounge vs plexweb

Things left to try (grasping at straws here)

  • Fresh Plex installation
    • slightly worried I had incompatible legacy settings
  • distro hop (ubuntu to arch perhaps)
    • plex on docker may make this process easier

The MDE is an arbitrator.

  1. Given the DNA of the video to be played
  2. Playback capabilities of the player
  3. Player settings applying any restrictions or specific instructions (burning subtitles, etc)
  4. Server bandwidth or other playback restrictions on the per-account basic.

The sequence is:

  1. The initial playback hope is to DirectPlay,
  2. The client provides its capabilities
  3. PMS applies those capabilities (limitations) if applicable
  4. The client’s desired (user options) about the playback are applied (burn / bitrate)
  5. Server limits.

As has been demonstrated above, the fault appears more with the particular player and not the MDE

Concur?

@ChuckPa,

Thank you for taking the time to look into this. I know you spent a while on it.

Yes that sounds about right. Is there anything I need to provide or will a developer look at this at some point?

I just don’t want to be in a situation where I have to toggle hardware transcoding off for some situations and toggle it back on for others.

Anyway, thanks again for your valued time.

How big is the file you have?

In cases like this, having the content in-hand to examine with our tools goes a long way.

With that data, the team can then decide what & if there is something to change as well as HOW to change it .

While not at liberty to disclose much, there are changes coming

  1. Plex/Web
  2. Smart TV app

From what I’ve been told , and read so far, this could be one of those cases which becomes a non-issue.

This is why having a sample (about 2 minutes is great if possible) helps the most.

Would that be possible to get a sample of it?

Thank you for letting me know. I’m glad to hear it!

The full movie is creative commons and therefore free and legal to distribute.

Here is a link to the full 1GB movie I’ve been testing with. I’m able to replicate this problem with pretty much everything in my library. I chose this movie for testing because of the permissive license.

Here is the 2 minute clip you asked for. Clipped with ffmpeg -ss 60 -i $FILENAME.mkv -c copy -t 120 $OUTFILE.mkv. It should be identical container/stream format due to -c flag.
Sorry the domain name looks sketchy, it’s an open source file host with a command line API. My computers are currently boxed up as I am moving.

.

I now have the file.

  1. Playback and downgrade to lower quality in Chrome

  1. Log file
Jul 31, 2020 00:10:03.032 [0x7ffa8b159700] DEBUG - [FFMPEG] - Direct mapping possible.
Jul 31, 2020 00:10:03.032 [0x7ffa8b159700] DEBUG - TPU: hardware transcoding: final decoder: vaapi, final encoder: vaapi
Jul 31, 2020 00:10:03.033 [0x7ffa8b159700] DEBUG - Job running: EAE_ROOT='/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/tmp/pms-47a69fa0-0ca9-4803-a5e6-e17f039d54a1/EasyAudioEncoder' FFMPEG_EXTERNAL_LIBS='/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/Plex\ Media\ Server/Codecs/5f603a2-3204-linux-x86_64/' XDG_CACHE_HOME='/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/Cache' XDG_DATA_HOME='/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Resources' X_PLEX_TOKEN='xxxxxxxxxxxxxxxxxxxx' '/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Plex Transcoder' '-codec:0' 'h264' '-hwaccel:0' 'vaapi' '-hwaccel_fallback_threshold:0' '10' '-hwaccel_output_format:0' 'vaapi' '-codec:1' 'ac3' '-analyzeduration' '20000000' '-probesize' '20000000' '-i' '/share/qa/samples/Sintel (2010)/Sintel.2010.1080p.mkv' '-filter_complex' '[0:0]hwupload[0];[0]scale_vaapi=w=718:h=306:format=nv12[1];[1]hwupload[2]' '-filter_complex' '[0:1] aresample=async=1:ocl='\''stereo'\'':rematrix_maxval=0.000000dB:osr=48000[3]' '-map' '[2]' '-metadata:s:0' 'language=eng' '-codec:0' 'h264_vaapi' '-b:0' '951k' '-maxrate:0' '1268k' '-bufsize:0' '2536k' '-r:0' '24' '-force_key_frames:0' 'expr:gte(t,0+n_forced*8)' '-map' '[3]' '-metadata:s:1' 'language=eng' '-codec:1' 'aac' '-b:1' '159k' '-f' 'dash' '-seg_duration' '8' '-init_seg_name' 'init-stream$RepresentationID$.m4s' '-media_seg_name' 'chunk-stream$RepresentationID$-$Number%05d$.m4s' '-window_size' '5' '-delete_removed' 'false' '-skip_to_segment' '1' '-time_delta' '0.0625' '-manifest_name' 'http://127.0.0.1:32400/video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/manifest?X-Plex-Http-Pipeline=infinite' '-avoid_negative_ts' 'disabled' '-map_metadata' '-1' '-map_chapters' '-1' 'dash' '-start_at_zero' '-copyts' '-vsync' 'cfr' '-y' '-init_hw_device' 'vaapi=vaapi:' '-hwaccel_device' 'vaapi' '-filter_hw_device' 'vaapi' '-nostats' '-loglevel' 'quiet' '-loglevel_plex' 'error' '-progressurl' 'http://127.0.0.1:32400/video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress'
Jul 31, 2020 00:10:03.033 [0x7ffa8b159700] DEBUG - Jobs: Starting child process with pid 24915
Jul 31, 2020 00:10:03.034 [0x7ffae2552700] DEBUG - Request: [127.0.0.1:37744 (Loopback)] PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=startup (12 live) Signed-in Token (ChuckPA) (range: bytes=0-) 
Jul 31, 2020 00:10:03.034 [0x7ffaf95b4700] DEBUG - Completed: [127.0.0.1:37744] 204 PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=startup (12 live) 0ms 203 bytes (pipelined: 1) (range: bytes=0-) 
Jul 31, 2020 00:10:03.044 [0x7ffa88e31700] DEBUG - Request: [127.0.0.1:37744 (Loopback)] PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=startup (12 live) Signed-in Token (ChuckPA) (range: bytes=0-) 
Jul 31, 2020 00:10:03.045 [0x7ffaf92c6700] DEBUG - Completed: [127.0.0.1:37744] 204 PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=startup (12 live) 0ms 203 bytes (pipelined: 2) (range: bytes=0-) 
Jul 31, 2020 00:10:03.045 [0x7ffaf8fd8700] DEBUG - Request: [127.0.0.1:37744 (Loopback)] PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=opening (12 live) Signed-in Token (ChuckPA) (range: bytes=0-) 
Jul 31, 2020 00:10:03.045 [0x7ffaf95b4700] DEBUG - Completed: [127.0.0.1:37744] 204 PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=opening (12 live) 0ms 203 bytes (pipelined: 3) (range: bytes=0-) 
Jul 31, 2020 00:10:03.045 [0x7ffae0806700] DEBUG - Request: [127.0.0.1:37744 (Loopback)] PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=opened (12 live) Signed-in Token (ChuckPA) (range: bytes=0-) 
Jul 31, 2020 00:10:03.045 [0x7ffaf92c6700] DEBUG - Completed: [127.0.0.1:37744] 204 PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress?status=opened (12 live) 0ms 203 bytes (pipelined: 4) (range: bytes=0-) 
Jul 31, 2020 00:10:03.046 [0x7ffae2552700] DEBUG - Request: [127.0.0.1:37744 (Loopback)] PUT /video/:/transcode/session/dtyw0f62k7uxfzxi4rvtrl6r/310a9600-b8f9-451f-8ce4-c8346efa9340/progress/stream?index=0&id=0&codec=h264&type=video (12 live) Signed-in Token (ChuckPA) (range: bytes=0-) 

What am I missing with this please?

Is this the Nvidia fails to work or any hardware?? This is QSV working

I don’t have a quicksync (vaapi) Intel to test, only have the P2200 Quadro. My Xeon x5650s are too old for vaapi to work.

Plex Server in my configuration will attempt to open vaapi, fails, tries nvdec/nvenc, succeeds, repeats trying vaapi again, and then Plex Server kills the transcoder with a -9 signal. Finally, plex web will sit there polling for a transcoder route and get a 404.

I have done everything to ensure there is no Intel graphical device on my server including disabling the onboard graphics in the bios. Plex will still attempt vaapi.

Yuck.

I have no way of testing that either.

The GTX-1050 from the lab is dedicated to another task
My QNAP has a stock power supply (250w) so can’t accept a card.

@chrisallen

Can you help here? Do you have a p2000 ?

I do have a P2000. I’ll have a look next week at trying to reproduce.

Were you able to test @chrisallen?
This is still a problem I am having with the latest release (Version 1.20.4.3508).

Just chiming in to say I also have this issue with Plex Web. Running Proxmox with plex in an LXC, the 1650 Super is able to transcode on Android and roku TV just fine, transcode quality can be adjusted on a whim. In web, if you start playback in original and then lower to force a transcode the transcoder runner fails. However, starting the stream in a lower quality does work and is hw transcoded. Switching quality from there sometimes works, other times it fails. I can get more info and logs tomorrow, but this is on the latest nvidia driver and ubuntu 20.04.

1 Like

Thank you @jpurpura ! I am no longer alone with this issue!
I have been upgrading every package I can think of, Plex is up to date

Plex on Bare Metal (systemd)
Plex version 1.20.4.3517
Plex web 4.45.0
Ubuntu 20.04.1 LTS
Kernel 5.4.0-52-generic
nvidia-headless-455.28

I have seen a couple other threads here and there about this issue:

This guy says his was fixed, but no luck for me. I’ve attached some debug logs from mine, at 15:42:11 I chose a lower quality from a file playing in original quality and it just froze with a black screen.

Proxmox 6.2-15 (kernel 5.4.65-1-pve)
Plex inside an unprivileged Ubuntu 20.04.1 LXC, 1650 Super passed through with Proxmox’s nvidia hook
Plex server 1.20.4.3517
nvidia driver 455.38

(File removed)

@jpurpura

That would be an interesting “gotcha”, but I really doubt that a PCI x4 port would cause hardware transcoding to fail. The link speed should not interfere with hardware transcoding as you’d have to saturate many gigabytes per second over the port – and unless you’re live transcoding lossless 8k 60fps HDR video, nobody should be running into it.

In any case this is definitely not my problem as evidenced by my motherboard manual and verified with lspci

$ lspci -nvvv -s 06:00.0 | grep -E '(nvidia|LnkCap)'
                LnkCap: Port 0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
        Kernel driver in use: nvidia
        Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

x16 slot being the important bit.

I’ve done all sorts of benchmarking and tests on my card and can verify that it

  1. doesn’t overheat or have physical problems (gpu-burn cuda test for hours)
  2. can handle up to 10 1080p -> 720p transcodes simultaneously (synclounge for long periods of episodes)

Yeah I didn’t mean to link that exact post lol, but he said his was fixed with the 450 driver a bit later. Mine is actually connected via PCIE 3.0 x1 because I can’t fit a 1650 Super into my 2U chassis lol, but even that should be fine at around 1GB/sec.