NVIDIA hardware acceleration inconsistently working with web streaming

Plex Media Server Desktops & Laptops
,
1

Server Version#: 1.30.2.6563 in plexinc/pms-docker on OpenMediaVault 6 using nvidia runtime
NVIDIA Tesla P4, driver 525.60.13, CUDA 12…0
Using web client.

This is my first attempt to use hardware acceleration, and it’s almost working. Starting a stream (as in, pressing “Play”) usually works, but changing quality mid-stream is a 50-50 chance of success. Here’s a relevant snippet of debug logs, with the quality change at 21:02:37 (line 1593).

If it doesn’t work, there’s no indication in the web client other than no buffering taking place (no spinning wheel is present). The server dashboard continues to erroneously report playback progress for some time. No amount of waiting will make the playback start.

nvidia-smi displays the transcoder process as expected when working, however watching it at a 1s interval reveals a 1-2s new process consuming up to 200MB memory, then dropping to ~16, then gone when it fails. No process ever takes its place when the transcode fails, however it balloons to ~700MB when working as expected.

It doesn’t appear to be related to the selected quality - 1080p 8Mbps sometimes works, sometimes doesn’t. Requesting a new quality can sometimes cause playback to start at the new requested quality. When not working, there is no entry in the “Plex Media Server/Cache/Transcode/Sessions” directory.

Fiddling with streams too much can make it enter something like a “broken state” where starting a new stream simply doesn’t work anymore. Restarting the docker container and resetting the GPU with “nvidia-smi -r” helped once, but not always.

However, if I disable hardware accelerated encoding and instead rely on CPU, streaming works as expected. The GPU does hardware-accelerated decoding just fine. So there seems to be an issue with the encoder specifically.

What’s really odd, and I noticed just now while writing this post, is refreshing the tab in my browser seems to “restore” a failed stream, at the original quality (not the newly requested one). The transcoder process appears in nvidia-smi, and playback begins.

How do I go about debugging this further? The lack of reproducibility is frustrating. I see no relevant entries in dmesg or journalctl, and going as far as enabling verbose logging hasn’t given me any hints.

2

There is a known problem with CUDA SDK 12.0

We already have a correction to PMS.

Do you need DEB or RPM ?

If, after you complete testing, please let us know if the issue is solved.

The code I have is “Development Status”

3

I had discounted that as a possibility because it sometimes works - if this was a CUDA 12 incompatibility, I expected it would fail every time. I’ve grabbed that build from another thread on here but failed to get it to install in my docker container. Will the server version number display 1.31.1.6617 if it installs correctly?

4

Actually – NO, it would work sometimes depending on the video file.

Docker should have installed cleanly.

Which container mechanism are you using and on which host OS? (I might need upgrade the Debian installer)

5

The behavior I’m seeing is the same file will sometimes work, sometimes fail. Is that expected?

In terms of installing, I’m trying to build a new docker image with the updated version. I should have been more clear - the installer returns fine, but the started server based on this new image displays version number 1.30.2.6563. I’m doing this via Portainer running on OpenMediaVault 6 (Debian 11-based). This is likely a problem with my lack of knowledge of docker. My Dockerfile looks like:

FROM plexinc/pms-docker
WORKDIR /tmp
COPY plexmediaserver_1.31.1.6617-cbcbc6f56_amd64.deb ./
RUN dpkg -i --force-confold --force-architecture plexmediaserver_1.31.1.6617-cbcbc6f56_amd64.deb && rm -f plexmediaserver_1.31.1.6617-cbcbc6f56_amd64.deb

The output says:

Preparing to unpack plexmediaserver_1.31.1.6617-cbcbc6f56_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Custom environment detected.  Skipping preinstallation validation.
Unpacking plexmediaserver (1.31.1.6617-cbcbc6f56) over (1.30.2.6563-3d4dc0cce) ...
Setting up plexmediaserver (1.31.1.6617-cbcbc6f56) ...
PlexMediaServer install: Custom environment detected.  Skipping postinstallation tasks. Continuing.

That’s the same install command I see in the plexinc/pms-docker git repo. It sure looks like it installs, but the container I start up says the wrong version. If you happen to know what I’m doing wrong that would be greatly appreciated, otherwise I’ll keep googling.

6

I managed to build an image by modifying the plexinc/pms-docker Dockerfile, and now have a container running 1.31.1.6617. Unfortunately it’s still failing to transcode as before.

7

I’m working with the engineer right now.
We’re finishing out a build which allows control of the buffers (the problem)

Can you please show me the XML <media> portion of a file which fails to transcode?

8

This is one of the files I’ve been testing with:

<Media id="773638" duration="2619104" bitrate="24877" width="3840" height="1606" aspectRatio="2.35" audioChannels="6" audioCodec="eac3" videoCodec="hevc" videoResolution="4k" container="mkv" videoFrameRate="24p" videoProfile="main 10">
    <Part accessible="1" exists="1" id="791886" key="/library/parts/791886/1666575746/file.mkv" duration="2619104" file="/mnt/TV/Severance/Severance.S01.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW/Severance.S01E05.The.Grim.Barbarity.Of.Optics.and.Design.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW.mkv" size="8145807369" container="mkv" deepAnalysisVersion="6" requiredBandwidths="58555,39864,28988,27748,27203,26658,25684,25684" videoProfile="main 10">
        <Stream id="1948803" streamType="1" default="1" codec="hevc" index="0" bitrate="24106" DOVIBLCompatID="1" DOVIBLPresent="1" DOVIELPresent="0" DOVILevel="6" DOVIPresent="1" DOVIProfile="8" DOVIRPUPresent="1" DOVIVersion="1.0" bitDepth="10" chromaLocation="topleft" chromaSubsampling="4:2:0" codedHeight="1608" codedWidth="3840" colorPrimaries="bt2020" colorRange="tv" colorSpace="bt2020nc" colorTrc="smpte2084" frameRate="23.976" height="1606" level="150" profile="main 10" refFrames="1" requiredBandwidths="57775,39097,28217,26976,26431,25886,24915,24915" width="3840" displayTitle="4K DoVi/HDR10 (HEVC Main 10)" extendedDisplayTitle="4K DoVi/HDR10 (HEVC Main 10)">
        </Stream>
        <Stream id="1948804" streamType="2" selected="1" default="1" codec="eac3" index="1" channels="6" bitrate="768" language="English" languageTag="en" languageCode="eng" audioChannelLayout="5.1(side)" requiredBandwidths="767,767,767,767,767,767,767,767" samplingRate="48000" displayTitle="English (EAC3 5.1)" extendedDisplayTitle="English (EAC3 5.1)">
        </Stream>
        <Stream id="1948805" streamType="3" codec="srt" index="2" bitrate="0" language="English" languageTag="en" languageCode="eng" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="English (SRT)" extendedDisplayTitle="English (SRT)">
        </Stream>
        <Stream id="1948806" streamType="3" codec="srt" index="3" bitrate="0" language="English" languageTag="en" languageCode="eng" requiredBandwidths="1,1,1,1,1,1,1,1" title="CC" displayTitle="English (SRT)" extendedDisplayTitle="CC (English SRT)">
        </Stream>
        <Stream id="1948807" streamType="3" codec="srt" index="4" bitrate="0" language="Arabic" languageTag="ar" languageCode="ara" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Arabic (SRT)" extendedDisplayTitle="Arabic (SRT)">
        </Stream>
        <Stream id="1948808" streamType="3" codec="srt" index="5" bitrate="0" language="Bulgarian" languageTag="bg" languageCode="bul" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Bulgarian (SRT)" extendedDisplayTitle="Bulgarian (SRT)">
        </Stream>
        <Stream id="1948809" streamType="3" codec="srt" index="6" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" requiredBandwidths="1,1,1,1,1,1,1,1" title="Mandarin, Simplified" displayTitle="Chinese (SRT)" extendedDisplayTitle="Mandarin, Simplified (Chinese SRT)">
        </Stream>
        <Stream id="1948810" streamType="3" codec="srt" index="7" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" requiredBandwidths="1,1,1,1,1,1,1,1" title="Mandarin, Traditional" displayTitle="Chinese (SRT)" extendedDisplayTitle="Mandarin, Traditional (Chinese SRT)">
        </Stream>
        <Stream id="1948811" streamType="3" codec="srt" index="8" bitrate="0" language="Czech" languageTag="cs" languageCode="ces" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Czech (SRT)" extendedDisplayTitle="Czech (SRT)">
        </Stream>
        <Stream id="1948812" streamType="3" codec="srt" index="9" bitrate="0" language="Danish" languageTag="da" languageCode="dan" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Danish (SRT)" extendedDisplayTitle="Danish (SRT)">
        </Stream>
        <Stream id="1948813" streamType="3" codec="srt" index="10" bitrate="0" language="German" languageTag="de" languageCode="deu" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="German (SRT)" extendedDisplayTitle="German (SRT)">
        </Stream>
        <Stream id="1948814" streamType="3" codec="srt" index="11" bitrate="0" language="Greek" languageTag="el" languageCode="ell" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Greek (SRT)" extendedDisplayTitle="Greek (SRT)">
        </Stream>
        <Stream id="1948815" streamType="3" codec="srt" index="12" bitrate="0" language="Spanish" languageTag="es" languageCode="spa" requiredBandwidths="1,1,1,1,1,1,1,1" title="Latin American" displayTitle="Spanish (SRT)" extendedDisplayTitle="Latin American (Spanish SRT)">
        </Stream>
        <Stream id="1948816" streamType="3" codec="srt" index="13" bitrate="0" language="Spanish" languageTag="es" languageCode="spa" requiredBandwidths="1,1,1,1,1,1,1,1" title="European" displayTitle="Spanish (SRT)" extendedDisplayTitle="European (Spanish SRT)">
        </Stream>
        <Stream id="1948817" streamType="3" codec="srt" index="14" bitrate="0" language="Estonian" languageTag="et" languageCode="est" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Estonian (SRT)" extendedDisplayTitle="Estonian (SRT)">
        </Stream>
        <Stream id="1948818" streamType="3" codec="srt" index="15" bitrate="0" language="Finnish" languageTag="fi" languageCode="fin" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Finnish (SRT)" extendedDisplayTitle="Finnish (SRT)">
        </Stream>
        <Stream id="1948819" streamType="3" codec="srt" index="16" bitrate="0" language="French" languageTag="fr" languageCode="fra" requiredBandwidths="1,1,1,1,1,1,1,1" title="Canadian" displayTitle="French (SRT)" extendedDisplayTitle="Canadian (French SRT)">
        </Stream>
        <Stream id="1948820" streamType="3" codec="srt" index="17" bitrate="0" language="French" languageTag="fr" languageCode="fra" requiredBandwidths="1,1,1,1,1,1,1,1" title="European" displayTitle="French (SRT)" extendedDisplayTitle="European (French SRT)">
        </Stream>
        <Stream id="1948821" streamType="3" codec="srt" index="18" bitrate="0" language="Hebrew" languageTag="he" languageCode="heb" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Hebrew (SRT)" extendedDisplayTitle="Hebrew (SRT)">
        </Stream>
        <Stream id="1948822" streamType="3" codec="srt" index="19" bitrate="0" language="Hindi" languageTag="hi" languageCode="hin" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Hindi (SRT)" extendedDisplayTitle="Hindi (SRT)">
        </Stream>
        <Stream id="1948823" streamType="3" codec="srt" index="20" bitrate="0" language="Hungarian" languageTag="hu" languageCode="hun" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Hungarian (SRT)" extendedDisplayTitle="Hungarian (SRT)">
        </Stream>
        <Stream id="1948824" streamType="3" codec="srt" index="21" bitrate="0" language="Indonesian" languageTag="id" languageCode="ind" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Indonesian (SRT)" extendedDisplayTitle="Indonesian (SRT)">
        </Stream>
        <Stream id="1948825" streamType="3" codec="srt" index="22" bitrate="0" language="Italian" languageTag="it" languageCode="ita" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Italian (SRT)" extendedDisplayTitle="Italian (SRT)">
        </Stream>
        <Stream id="1948826" streamType="3" codec="srt" index="23" bitrate="0" language="Japanese" languageTag="ja" languageCode="jpn" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Japanese (SRT)" extendedDisplayTitle="Japanese (SRT)">
        </Stream>
        <Stream id="1948827" streamType="3" codec="srt" index="24" bitrate="0" language="Korean" languageTag="ko" languageCode="kor" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Korean (SRT)" extendedDisplayTitle="Korean (SRT)">
        </Stream>
        <Stream id="1948828" streamType="3" codec="srt" index="25" bitrate="0" language="Lithuanian" languageTag="lt" languageCode="lit" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Lithuanian (SRT)" extendedDisplayTitle="Lithuanian (SRT)">
        </Stream>
        <Stream id="1948829" streamType="3" codec="srt" index="26" bitrate="0" language="Latvian" languageTag="lv" languageCode="lav" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Latvian (SRT)" extendedDisplayTitle="Latvian (SRT)">
        </Stream>
        <Stream id="1948830" streamType="3" codec="srt" index="27" bitrate="0" language="Malay" languageTag="ms" languageCode="msa" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Malay (SRT)" extendedDisplayTitle="Malay (SRT)">
        </Stream>
        <Stream id="1948831" streamType="3" codec="srt" index="28" bitrate="0" language="Dutch" languageTag="nl" languageCode="nld" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Dutch (SRT)" extendedDisplayTitle="Dutch (SRT)">
        </Stream>
        <Stream id="1948832" streamType="3" codec="srt" index="29" bitrate="0" language="Norwegian" languageTag="no" languageCode="nor" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Norwegian (SRT)" extendedDisplayTitle="Norwegian (SRT)">
        </Stream>
        <Stream id="1948833" streamType="3" codec="srt" index="30" bitrate="0" language="Polish" languageTag="pl" languageCode="pol" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Polish (SRT)" extendedDisplayTitle="Polish (SRT)">
        </Stream>
        <Stream id="1948834" streamType="3" codec="srt" index="31" bitrate="0" language="Portuguese" languageTag="pt" languageCode="por" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Portuguese (SRT)" extendedDisplayTitle="Portuguese (SRT)">
        </Stream>
        <Stream id="1948835" streamType="3" codec="srt" index="32" bitrate="0" language="Portuguese" languageTag="pt" languageCode="por" requiredBandwidths="1,1,1,1,1,1,1,1" title="Brazilian" displayTitle="Portuguese (SRT)" extendedDisplayTitle="Brazilian (Portuguese SRT)">
        </Stream>
        <Stream id="1948836" streamType="3" codec="srt" index="33" bitrate="0" language="Russian" languageTag="ru" languageCode="rus" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Russian (SRT)" extendedDisplayTitle="Russian (SRT)">
        </Stream>
        <Stream id="1948837" streamType="3" codec="srt" index="34" bitrate="0" language="Slovak" languageTag="sk" languageCode="slk" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Slovak (SRT)" extendedDisplayTitle="Slovak (SRT)">
        </Stream>
        <Stream id="1948838" streamType="3" codec="srt" index="35" bitrate="0" language="Slovenian" languageTag="sl" languageCode="slv" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Slovenian (SRT)" extendedDisplayTitle="Slovenian (SRT)">
        </Stream>
        <Stream id="1948839" streamType="3" codec="srt" index="36" bitrate="0" language="Swedish" languageTag="sv" languageCode="swe" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Swedish (SRT)" extendedDisplayTitle="Swedish (SRT)">
        </Stream>
        <Stream id="1948840" streamType="3" codec="srt" index="37" bitrate="0" language="Tamil" languageTag="ta" languageCode="tam" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Tamil (SRT)" extendedDisplayTitle="Tamil (SRT)">
        </Stream>
        <Stream id="1948841" streamType="3" codec="srt" index="38" bitrate="0" language="Telugu" languageTag="te" languageCode="tel" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Telugu (SRT)" extendedDisplayTitle="Telugu (SRT)">
        </Stream>
        <Stream id="1948842" streamType="3" codec="srt" index="39" bitrate="0" language="Thai" languageTag="th" languageCode="tha" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Thai (SRT)" extendedDisplayTitle="Thai (SRT)">
        </Stream>
        <Stream id="1948843" streamType="3" codec="srt" index="40" bitrate="0" language="Turkish" languageTag="tr" languageCode="tur" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Turkish (SRT)" extendedDisplayTitle="Turkish (SRT)">
        </Stream>
        <Stream id="1948844" streamType="3" codec="srt" index="41" bitrate="0" language="Ukrainian" languageTag="uk" languageCode="ukr" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Ukrainian (SRT)" extendedDisplayTitle="Ukrainian (SRT)">
        </Stream>
        <Stream id="1948845" streamType="3" codec="srt" index="42" bitrate="0" language="Vietnamese" languageTag="vi" languageCode="vie" requiredBandwidths="1,1,1,1,1,1,1,1" displayTitle="Vietnamese (SRT)" extendedDisplayTitle="Vietnamese (SRT)">
        </Stream>
        <Stream id="1948846" streamType="3" codec="srt" index="43" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" requiredBandwidths="1,1,1,1,1,1,1,1" title="Cantonese, Traditional" displayTitle="Chinese (SRT)" extendedDisplayTitle="Cantonese, Traditional (Chinese SRT)">
        </Stream>
    </Part>
</Media>
9

Well this just got weirder. I think I’ve been chasing the wrong issue.

I saw your post regarding a new build, 6638, and tried it out, but unfortunately I’m still experiencing the problem.

However, I spent 10 minutes playing with my NVIDIA Shield (2017), and flipping quality up and down worked flawlessly. I confirmed by the web dashboard that the requested quality was being transcoded, and confirmed a transcode process running in nvidia-smi. This, plus having to refresh the web browser to fix broken streams, makes me think the issue is somehow in the web client.

I’ve captured some logs from my browser’s console when the problem occurs. I normally use Firefox, but tried in Chrome and had the same problem. I notice lots of CORS errors - maybe related? All but one show up after I request the new quality. Is there another source of logs I should be capturing?

I was also able to reproduce a “fix” of sorts - disabling hardware-accelerated video encoding in the server settings allows for changing quality with no problems (even with hardware-accelerated decoding taking place). This doesn’t make sense to me, but consistently works.

10

@Adarnof

The browser is not the issue. (just to be clear)

The issue we’re working on is twofold.

  1. Some 1080p H.264 will not transcode - fixed in 6617 build
  2. Nvidia drivers SDK 12.0 (used in Linux-based servers) – fixed in 6638 build.

I can pull the Nvidia Shield server for you but doubt this will address your issue
(Your server runs ON the Nvidia box itself with Android)

If you can do the following, I will look and see what’s happening

  1. Settings - Server - General - Show Advanced
    – Enable DEBUG logging if not
    – Disable VERBOSE logging if enabled
    – SAVE if changes made

  2. Now recreate the failure

  3. Download the Server logs from Settings - Server - Troubleshooting - Download Logs

  4. Attach the ZIP file here

11

When I said the NVIDIA Shield was working, I meant using it as a client to play from my server running in the docker container on OpenMediaVault. The android app that runs on it seems to be able to request formats just fine from my server.

I’ve attached two sets of logs - one where I requested a quality change in the web client which failed on my first attempt, and a second set where I requested a series of quality changes through the app on my NVIDIA Shield without issue. This behavior is why I think the problem I’m experiencing may be related to the client, and is not actually the CUDA 12 incompatibility you’ve been chasing down.

Plex Media Server Logs android client working.zip (163.4 KB)
Plex Media Server Logs web client not working.zip (65.4 KB)

12

Nvidia Shield, especially the Pro 2019, can DirectPlay anything from PMS

My question to you: Are you talking about a server problem or a player problem?

That’s where I"m confused because this thread is about CUDA 12.0 with discrete Nvidia GPU cards in Linux boxes.

13

I thought it was a server problem when I started this thread. I had just added a GPU and made the change to use hardware acceleration, and that’s when I noticed the problem. I have not seen the same symptoms being reported in other threads about NVIDIA hardware acceleration problems - the error regarding surfaces never appears, and transcoding the same source file sometimes works, sometimes fails. However, the issue is only present when using NVIDIA hardware-accelerated video encoding (NOT just hardware-accelerated video decoding - it behaves fine when just using hardware-accelerated decoding), so I think it is playing a role here, either in the act of transcoding or in how the clients request transcoding.

The shield can indeed direct play everything I’ve thrown at it, but I tested changing playback quality on it anyway, and have noticed the problem with transcoding on my server does not appear when the shield app requests a quality which requires transcoding. This strikes me as quite odd, considering how I’ve been reliably reproducing this issue when testing in the web client (it’s more than 75% of the time, but oddly not every time). I’ve checked several times now, and have never been able to reproduce the problem when, in the plex app on the shield, I change playback quality from “Play Original Quality” to something like 1080p 8Mbps. You’ll see in the logs I cycle through several qualities, and after buffering for a few moments, the plex app starts playback normally, and I see transcoder activity in nvidia-smi on my server. In contrast, when I change quality on the web client, it usually just sits at a black screen without any indication of buffering or playback, and no transcoder activity can be seen in nvidia-smi on my server.

14

If I go to the start of the actual transcode and go to the end of the logfile, there are no errors with 6638 installed in your container.

Jan 27, 2023 19:19:35.666 [0x7f570d47fb38] DEBUG - [Req#172/Transcode] [Universal] Using local file path instead of URL: /data/TV/Severance/Severance.S01.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW/Severance.S01E05.The.Grim.Barbarity.Of.Optics.and.Design.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW.mkv
Jan 27, 2023 19:19:35.666 [0x7f570d47fb38] DEBUG - [Req#172/Transcode] TPU: hardware transcoding: final decoder: nvdec, final encoder: nvenc
Jan 27, 2023 19:19:35.667 [0x7f570d47fb38] DEBUG - [Req#172/Transcode/JobRunner] Job running: CUDA_CACHE_PATH="/config/Library/Application Support/Plex Media Server/Cache/Shaders/CUDA" EAE_ROOT=/tmp/pms-01a5acbe-8a8d-4f7c-a835-7e70f323b029/EasyAudioEncoder FFMPEG_EXTERNAL_LIBS='/config/Library/Application\ Support/Plex\ Media\ Server/Codecs/89482f6-4464-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4f85-bae5-1f03d99c2834 "/usr/lib/plexmediaserver/Plex Transcoder" -codec:0 hevc -hwaccel:0 nvdec -hwaccel_fallback_threshold:0 10 -threads:0 1 -extra_hw_frames:0 1 -hwaccel_output_format:0 cuda -hwaccel_device:0 cuda -codec:1 eac3_eae -eae_prefix:1 dmcle5ujfn74hpdkpbm6mpnl_ -analyzeduration 20000000 -probesize 20000000 -i /data/TV/Severance/Severance.S01.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW/Severance.S01E05.The.Grim.Barbarity.Of.Optics.and.Design.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW.mkv -filter_complex "[0:0]hwupload[0];[0]scale_cuda=w=3060:h=1280:format=p010[1];[1]tonemap_cuda=mobius:nv12[2]" -map "[2]" -codec:0 h264_nvenc -b:0 20000k -forced-idr:0 1 -r:0 23.975999999999999 -force_key_frames:0 "expr:gte(t,n_forced*1)" -filter_complex "[0:1] aresample=async=1:ochl='stereo':rematrix_maxval=0.000000dB:osr=48000[3]" -map "[3]" -metadata:s:1 language=eng -codec:1 aac -b:1 256k -f dash -seg_duration 1 -dash_segment_type mp4 -init_seg_name 'init-stream$RepresentationID$.m4s' -media_seg_name 'chunk-stream$RepresentationID$-$Number%05d$.m4s' -window_size 5 -delete_removed false -skip_to_segment 1 -time_delta 0.0625 -manifest_name "http://127.0.0.1:32400/video/:/transcode/session/dmcle5ujfn74hpdkpbm6mpnl/b57beb13-ab17-4ac3-9941-92d9199a1cc6/manifest?X-Plex-Http-Pipeline=infinite" -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -vsync cfr -init_hw_device cuda=cuda: -filter_hw_device cuda -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/dmcle5ujfn74hpdkpbm6mpnl/b57beb13-ab17-4ac3-9941-92d9199a1cc6/progress
Jan 27, 2023 19:19:35.667 [0x7f570d47fb38] DEBUG - [Req#172/Transcode/JobRunner] In directory: "/transcode/Transcode/Sessions/plex-transcode-dmcle5ujfn74hpdkpbm6mpnl-b57beb13-ab17-4ac3-9941-92d9199a1cc6"
Jan 27, 2023 19:19:35.667 [0x7f570d47fb38] DEBUG - [Req#172/Transcode/JobRunner] Jobs: Starting child process with pid 629
Jan 27, 2023 19:19:35.672 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Codecs: testing hevc (decoder) with hwdevice vaapi
Jan 27, 2023 19:19:35.672 [0x7f570bd2eb38] DEBUG - Request: [127.0.0.1:49196 (Loopback)] PUT /video/:/transcode/session/dmcle5ujfn74hpdkpbm6mpnl/b57beb13-ab17-4ac3-9941-92d9199a1cc6/progress?status=startup (12 live) #1cf Signed-in Token (Adarnof) (range: bytes=0-) 
Jan 27, 2023 19:19:35.672 [0x7f570a9f9b38] DEBUG - [Req#172/Transcode] Whacked session 7qqtoi4kusmidz86k0ekc577, 1 remaining.
Jan 27, 2023 19:19:35.672 [0x7f5711058b38] DEBUG - Completed: [127.0.0.1:49196] 204 PUT /video/:/transcode/session/dmcle5ujfn74hpdkpbm6mpnl/b57beb13-ab17-4ac3-9941-92d9199a1cc6/progress?status=startup (12 live) #1cf 0ms 203 bytes (pipelined: 1) (range: bytes=0-) 
Jan 27, 2023 19:19:35.672 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Codecs: hardware transcoding: testing API vaapi
Jan 27, 2023 19:19:35.674 [0x7f570e47db38] ERROR - [Req#1b6/Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Jan 27, 2023 19:19:35.674 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error
Jan 27, 2023 19:19:35.674 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Could not create hardware context for hevc
Jan 27, 2023 19:19:35.674 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Codecs: testing hevc (decoder) with hwdevice nvdec
Jan 27, 2023 19:19:35.675 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] Codecs: hardware transcoding: testing API nvdec
Jan 27, 2023 19:19:35.675 [0x7f570e47db38] DEBUG - [Req#1b6/Transcode] [FFMPEG] - Loaded lib: libcuda.so.1

If you look carefully, PMS fails.

The only reason this could fail is the video itself.

The GP104 can transcode 4:2:0 HEVC but it cannot transcode 4:4:4 (which is common with Dolby Vision files.

May I see the XML for the file ?

15

Here’s the full XML of the file used in the logs I posted. It says 4:2:0 for the video stream, which is consistent with what MediaInfo tells me. I have been able to reproduce this on non-Dolby Vision content as well.

<?xml version="1.0" encoding="UTF-8"?>
<MediaContainer size="1" allowSync="1" identifier="com.plexapp.plugins.library" librarySectionID="1" librarySectionTitle="TV Shows" librarySectionUUID="0a1718f6-268c-48be-9afb-a29ef56837da" mediaTagPrefix="/system/bundle/media/flags/" mediaTagVersion="1674778349">
<Video ratingKey="7" key="/library/metadata/7" parentRatingKey="2" grandparentRatingKey="1" guid="plex://episode/61bca2111ebce73e8819233f" parentGuid="plex://season/605a0a344d799b002dc10d05" grandparentGuid="plex://show/5e1622d34f2292003e91314a" type="episode" title="The Grim Barbarity of Optics and Design" titleSort="Grim Barbarity of Optics and Design" grandparentKey="/library/metadata/1" parentKey="/library/metadata/2" librarySectionTitle="TV Shows" librarySectionID="1" librarySectionKey="/library/sections/1" grandparentTitle="Severance" parentTitle="Season 1" contentRating="TV-MA" summary="Irving and Dylan confront Burt about his lies. Mark and Helly discover a strange new department." index="5" parentIndex="1" audienceRating="8.0" viewOffset="135501" viewCount="1" lastViewedAt="1674867884" year="2022" thumb="/library/metadata/7/thumb/1674795336" art="/library/metadata/1/art/1674795335" parentThumb="/library/metadata/2/thumb/1674795335" grandparentThumb="/library/metadata/1/thumb/1674795335" grandparentArt="/library/metadata/1/art/1674795335" grandparentTheme="/library/metadata/1/theme/1674795335" duration="2619104" originallyAvailableAt="2022-03-10" addedAt="1666575746" updatedAt="1674795336" audienceRatingImage="themoviedb://image.rating">
<Media id="5" duration="2619104" bitrate="24881" width="3840" height="1606" aspectRatio="2.35" audioChannels="6" audioCodec="eac3" videoCodec="hevc" videoResolution="4k" container="mkv" videoFrameRate="24p" videoProfile="main 10">
<Part accessible="1" exists="1" id="5" key="/library/parts/5/1666575746/file.mkv" duration="2619104" file="/data/TV/Severance/Severance.S01.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW/Severance.S01E05.The.Grim.Barbarity.Of.Optics.and.Design.2160p.ATVP.WEB-DL.DDP5.1.Atmos.DV.HDR10+.H.265-CRFW.mkv" size="8145807369" container="mkv" videoProfile="main 10">
<Stream id="177" streamType="1" default="1" codec="hevc" index="0" bitrate="24881" DOVIBLCompatID="1" DOVIBLPresent="1" DOVIELPresent="0" DOVILevel="6" DOVIPresent="1" DOVIProfile="8" DOVIRPUPresent="1" DOVIVersion="1.0" bitDepth="10" chromaLocation="topleft" chromaSubsampling="4:2:0" codedHeight="1608" codedWidth="3840" colorPrimaries="bt2020" colorRange="tv" colorSpace="bt2020nc" colorTrc="smpte2084" frameRate="23.976" height="1606" level="150" profile="main 10" refFrames="1" width="3840" displayTitle="4K DoVi/HDR10 (HEVC Main 10)" extendedDisplayTitle="4K DoVi/HDR10 (HEVC Main 10)">
</Stream>
<Stream id="178" streamType="2" selected="1" default="1" codec="eac3" index="1" channels="6" bitrate="768" language="English" languageTag="en" languageCode="eng" audioChannelLayout="5.1(side)" samplingRate="48000" displayTitle="English (EAC3 5.1)" extendedDisplayTitle="English (EAC3 5.1)">
</Stream>
<Stream id="179" streamType="3" codec="srt" index="2" bitrate="0" language="English" languageTag="en" languageCode="eng" displayTitle="English (SRT)" extendedDisplayTitle="English (SRT)">
</Stream>
<Stream id="180" streamType="3" codec="srt" index="3" bitrate="0" language="English" languageTag="en" languageCode="eng" title="CC" displayTitle="English (SRT)" extendedDisplayTitle="CC (English SRT)">
</Stream>
<Stream id="181" streamType="3" codec="srt" index="4" bitrate="0" language="Arabic" languageTag="ar" languageCode="ara" displayTitle="Arabic (SRT)" extendedDisplayTitle="Arabic (SRT)">
</Stream>
<Stream id="182" streamType="3" codec="srt" index="5" bitrate="0" language="Bulgarian" languageTag="bg" languageCode="bul" displayTitle="Bulgarian (SRT)" extendedDisplayTitle="Bulgarian (SRT)">
</Stream>
<Stream id="183" streamType="3" codec="srt" index="6" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" title="Mandarin, Simplified" displayTitle="Chinese (SRT)" extendedDisplayTitle="Mandarin, Simplified (Chinese SRT)">
</Stream>
<Stream id="184" streamType="3" codec="srt" index="7" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" title="Mandarin, Traditional" displayTitle="Chinese (SRT)" extendedDisplayTitle="Mandarin, Traditional (Chinese SRT)">
</Stream>
<Stream id="185" streamType="3" codec="srt" index="8" bitrate="0" language="Czech" languageTag="cs" languageCode="ces" displayTitle="Czech (SRT)" extendedDisplayTitle="Czech (SRT)">
</Stream>
<Stream id="186" streamType="3" codec="srt" index="9" bitrate="0" language="Danish" languageTag="da" languageCode="dan" displayTitle="Danish (SRT)" extendedDisplayTitle="Danish (SRT)">
</Stream>
<Stream id="187" streamType="3" codec="srt" index="10" bitrate="0" language="German" languageTag="de" languageCode="deu" displayTitle="German (SRT)" extendedDisplayTitle="German (SRT)">
</Stream>
<Stream id="188" streamType="3" codec="srt" index="11" bitrate="0" language="Greek" languageTag="el" languageCode="ell" displayTitle="Greek (SRT)" extendedDisplayTitle="Greek (SRT)">
</Stream>
<Stream id="189" streamType="3" codec="srt" index="12" bitrate="0" language="Spanish" languageTag="es" languageCode="spa" title="Latin American" displayTitle="Spanish (SRT)" extendedDisplayTitle="Latin American (Spanish SRT)">
</Stream>
<Stream id="190" streamType="3" codec="srt" index="13" bitrate="0" language="Spanish" languageTag="es" languageCode="spa" title="European" displayTitle="Spanish (SRT)" extendedDisplayTitle="European (Spanish SRT)">
</Stream>
<Stream id="191" streamType="3" codec="srt" index="14" bitrate="0" language="Estonian" languageTag="et" languageCode="est" displayTitle="Estonian (SRT)" extendedDisplayTitle="Estonian (SRT)">
</Stream>
<Stream id="192" streamType="3" codec="srt" index="15" bitrate="0" language="Finnish" languageTag="fi" languageCode="fin" displayTitle="Finnish (SRT)" extendedDisplayTitle="Finnish (SRT)">
</Stream>
<Stream id="193" streamType="3" codec="srt" index="16" bitrate="0" language="French" languageTag="fr" languageCode="fra" title="Canadian" displayTitle="French (SRT)" extendedDisplayTitle="Canadian (French SRT)">
</Stream>
<Stream id="194" streamType="3" codec="srt" index="17" bitrate="0" language="French" languageTag="fr" languageCode="fra" title="European" displayTitle="French (SRT)" extendedDisplayTitle="European (French SRT)">
</Stream>
<Stream id="195" streamType="3" codec="srt" index="18" bitrate="0" language="Hebrew" languageTag="he" languageCode="heb" displayTitle="Hebrew (SRT)" extendedDisplayTitle="Hebrew (SRT)">
</Stream>
<Stream id="196" streamType="3" codec="srt" index="19" bitrate="0" language="Hindi" languageTag="hi" languageCode="hin" displayTitle="Hindi (SRT)" extendedDisplayTitle="Hindi (SRT)">
</Stream>
<Stream id="197" streamType="3" codec="srt" index="20" bitrate="0" language="Hungarian" languageTag="hu" languageCode="hun" displayTitle="Hungarian (SRT)" extendedDisplayTitle="Hungarian (SRT)">
</Stream>
<Stream id="198" streamType="3" codec="srt" index="21" bitrate="0" language="Indonesian" languageTag="id" languageCode="ind" displayTitle="Indonesian (SRT)" extendedDisplayTitle="Indonesian (SRT)">
</Stream>
<Stream id="199" streamType="3" codec="srt" index="22" bitrate="0" language="Italian" languageTag="it" languageCode="ita" displayTitle="Italian (SRT)" extendedDisplayTitle="Italian (SRT)">
</Stream>
<Stream id="200" streamType="3" codec="srt" index="23" bitrate="0" language="Japanese" languageTag="ja" languageCode="jpn" displayTitle="Japanese (SRT)" extendedDisplayTitle="Japanese (SRT)">
</Stream>
<Stream id="201" streamType="3" codec="srt" index="24" bitrate="0" language="Korean" languageTag="ko" languageCode="kor" displayTitle="Korean (SRT)" extendedDisplayTitle="Korean (SRT)">
</Stream>
<Stream id="202" streamType="3" codec="srt" index="25" bitrate="0" language="Lithuanian" languageTag="lt" languageCode="lit" displayTitle="Lithuanian (SRT)" extendedDisplayTitle="Lithuanian (SRT)">
</Stream>
<Stream id="203" streamType="3" codec="srt" index="26" bitrate="0" language="Latvian" languageTag="lv" languageCode="lav" displayTitle="Latvian (SRT)" extendedDisplayTitle="Latvian (SRT)">
</Stream>
<Stream id="204" streamType="3" codec="srt" index="27" bitrate="0" language="Malay" languageTag="ms" languageCode="msa" displayTitle="Malay (SRT)" extendedDisplayTitle="Malay (SRT)">
</Stream>
<Stream id="205" streamType="3" codec="srt" index="28" bitrate="0" language="Dutch" languageTag="nl" languageCode="nld" displayTitle="Dutch (SRT)" extendedDisplayTitle="Dutch (SRT)">
</Stream>
<Stream id="206" streamType="3" codec="srt" index="29" bitrate="0" language="Norwegian" languageTag="no" languageCode="nor" displayTitle="Norwegian (SRT)" extendedDisplayTitle="Norwegian (SRT)">
</Stream>
<Stream id="207" streamType="3" codec="srt" index="30" bitrate="0" language="Polish" languageTag="pl" languageCode="pol" displayTitle="Polish (SRT)" extendedDisplayTitle="Polish (SRT)">
</Stream>
<Stream id="208" streamType="3" codec="srt" index="31" bitrate="0" language="Portuguese" languageTag="pt" languageCode="por" displayTitle="Portuguese (SRT)" extendedDisplayTitle="Portuguese (SRT)">
</Stream>
<Stream id="209" streamType="3" codec="srt" index="32" bitrate="0" language="Portuguese" languageTag="pt" languageCode="por" title="Brazilian" displayTitle="Portuguese (SRT)" extendedDisplayTitle="Brazilian (Portuguese SRT)">
</Stream>
<Stream id="210" streamType="3" codec="srt" index="33" bitrate="0" language="Russian" languageTag="ru" languageCode="rus" displayTitle="Russian (SRT)" extendedDisplayTitle="Russian (SRT)">
</Stream>
<Stream id="211" streamType="3" codec="srt" index="34" bitrate="0" language="Slovak" languageTag="sk" languageCode="slk" displayTitle="Slovak (SRT)" extendedDisplayTitle="Slovak (SRT)">
</Stream>
<Stream id="212" streamType="3" codec="srt" index="35" bitrate="0" language="Slovenian" languageTag="sl" languageCode="slv" displayTitle="Slovenian (SRT)" extendedDisplayTitle="Slovenian (SRT)">
</Stream>
<Stream id="213" streamType="3" codec="srt" index="36" bitrate="0" language="Swedish" languageTag="sv" languageCode="swe" displayTitle="Swedish (SRT)" extendedDisplayTitle="Swedish (SRT)">
</Stream>
<Stream id="214" streamType="3" codec="srt" index="37" bitrate="0" language="Tamil" languageTag="ta" languageCode="tam" displayTitle="Tamil (SRT)" extendedDisplayTitle="Tamil (SRT)">
</Stream>
<Stream id="215" streamType="3" codec="srt" index="38" bitrate="0" language="Telugu" languageTag="te" languageCode="tel" displayTitle="Telugu (SRT)" extendedDisplayTitle="Telugu (SRT)">
</Stream>
<Stream id="216" streamType="3" codec="srt" index="39" bitrate="0" language="Thai" languageTag="th" languageCode="tha" displayTitle="Thai (SRT)" extendedDisplayTitle="Thai (SRT)">
</Stream>
<Stream id="217" streamType="3" codec="srt" index="40" bitrate="0" language="Turkish" languageTag="tr" languageCode="tur" displayTitle="Turkish (SRT)" extendedDisplayTitle="Turkish (SRT)">
</Stream>
<Stream id="218" streamType="3" codec="srt" index="41" bitrate="0" language="Ukrainian" languageTag="uk" languageCode="ukr" displayTitle="Ukrainian (SRT)" extendedDisplayTitle="Ukrainian (SRT)">
</Stream>
<Stream id="219" streamType="3" codec="srt" index="42" bitrate="0" language="Vietnamese" languageTag="vi" languageCode="vie" displayTitle="Vietnamese (SRT)" extendedDisplayTitle="Vietnamese (SRT)">
</Stream>
<Stream id="220" streamType="3" codec="srt" index="43" bitrate="0" language="Chinese" languageTag="zh" languageCode="zho" title="Cantonese, Traditional" displayTitle="Chinese (SRT)" extendedDisplayTitle="Cantonese, Traditional (Chinese SRT)">
</Stream>
</Part>
</Media>
<Director id="90" filter="director=90" tag="Aoife McArdle" />
<Writer id="96" filter="writer=96" tag="Anna Ouyang Moench" />
<Producer id="92" filter="producer=92" tag="Natasha Smith" />
<Producer id="97" filter="producer=97" tag="Josephine Sorena" />
<Guid id="imdb://tt13424090" />
<Guid id="tmdb://3396432" />
<Guid id="tvdb://8891225" />
<Rating image="themoviedb://image.rating" value="8.0" type="audience" />
<Role id="32" filter="actor=32" tag="Anthoula Katsimatides" tagKey="5d77685285719b001f3a8b49" role="Florence" thumb="https://metadata-static.plex.tv/a/people/a2ab9e5f1d28b8bfcc217aaac8e04017.jpg" />
<Role id="46" filter="actor=46" tag="Brian Rock" tagKey="6081ae19f9e7080035ee461f" role="Goat Wrangler" />
<Role id="22" filter="actor=22" tag="Nikki M. James" tagKey="5d77691bad5437001f75838d" role="Alexa" thumb="https://metadata-static.plex.tv/1/people/19675cc1d405a09167f7769733c6f9e0.jpg" />
<Role id="20" filter="actor=20" tag="Michael Cumpsty" tagKey="5d77682a3c3c2a001fbcbf02" role="Doug Graner" thumb="https://metadata-static.plex.tv/9/people/98bf578c8b09a936cc0dde58733603d1.jpg" />
<Role id="25" filter="actor=25" tag="Rachel Addington" tagKey="5f403d3ac63b480040e0e462" role="Elizabeth" />
<Role id="30" filter="actor=30" tag="Nora Dale" tagKey="61af7430127b052b99b64bb9" role="Gabriela &#39;Gabby&#39; Arteta" />
<Role id="21" filter="actor=21" tag="Claudia Robinson" tagKey="5d77686854c0f0001f307f8a" role="Felicia" />
<Role id="26" filter="actor=26" tag="Mark Kenneth Smaltz" tagKey="5d7768334de0ee001fccaebe" role="Judd" thumb="https://metadata-static.plex.tv/people/5d7768334de0ee001fccaebe.jpg" />
<Marker id="116" type="intro" startTimeOffset="165520" endTimeOffset="244347">
<Attributes id="116" />
</Marker>
<Extras size="0">
</Extras>
<Related>
</Related>
</Video>
</MediaContainer>
16

And just like that, the “No Decoder Surfaces Left” error has appeared while testing 1080p transcoding with build 6638.

Plex Media Server Logs_2023-01-27_20-41-55.zip (485.8 KB)

18

@Adarnof

Would you be kind enough to ‘dd’ cut off a 50 MB sample of the file so we can test with it?

dd if=filename.ext of=testfile-for-chuck.ext bs=1M count=50

then post a link to where I can download it?

19

Here’s the file that produced the “No Decoder Surfaces Left” message. Just noticed it’s not proper 1080p, as the width is 2048 (hence the “2K” in the file name).

20

Thank you…

Would you try setting a hidden preference in Preferences.xml, then start PMS and retest ?

Specific hidden pref to add is nvdecExtraFrames="2" (PMS stopped)

Also, do ANY 1080p / 2160p files play normally?

21

I’ve added that to the xml element as in <Preferences [...] HardwareAcceleratedCodecs="1" logDebug="1" nvdecExtraFrames="2"/> while PMS was stopped, but upon starting it’s still encountering that error and falling back to CPU transcoding when transcoding to a 1080p quality.

But only when transcoding works at all, most of the time nothing happens like I’ve reported for the 4k transcodes earlier.

EDIT: just saw the question at the end of your post, and no I haven’t found a file that plays back normally.