PMS 1.32.6+ HW transcoding issues and corrections

Network setup (creates br0 using ovs-switch)

Edit as appropriate to use nmcli :slight_smile:

[chuck@lizum plexqa.2003]$ cat Network-setup 
#!/bin/bash


# If I am not root, exit
if [ "$(whoami)" != "root" ]; then

  echo ERROR:  MUST run as root user
  exit
fi

# Confirm required Openvswitch-switch installed
if [ -z "$(dpkg -l | grep openvswitch-switch)" ]; then
  echo INSTALLING openvswitch-switch	
  apt -y install openvswitch-switch
fi

# Get hostname
Hostname=$(hostname)

# Determine which adapter to use 
AdapterName="$(ip link show | grep UP | grep -v UNKNOWN | awk '{print $2}' | tr -d ':' | head -1 )"

if [ "$AdapterName" == "" ]; then
  echo ERROR:  No usable adapters found.   Please verify basic network configuration
  exit 1
fi

# Inform:
echo Creating network configuration for:  $AdapterName on $Hostname

# Archive all YAML files
cd /etc/netplan
if [ ! -z "$(find . -name \*.yaml -print)" ]; then
  tar cf /etc/netplan/netplan-archive."$(date +%Y-%m-%d-%T)".tar *.yaml
  rm -f *.yaml
fi


# Setup network.
# will use 'br0' as bridge for LXD/LXC
# Write basic template then EDIT based on $(ip link) results above.

cat > /etc/netplan/$Hostname-config.yaml <<EOF
# This is the network config based on 'subiquity'
# Customized for HOSTNAME LXD/LXC use
network:
  ethernets:

    # Adapter will come up without IP
    ADAPTERNAME: {}

    # br0 will get and have IP
    br0:
      dhcp4: true
      dhcp6: true  
  version: 2
EOF

# Edit the host and adapter name to use what we found , writing into /etc/netplan
sed -i s/HOSTNAME/$Hostname/ /etc/netplan/$(hostname)-config.yaml
sed -i s/ADAPTERNAME/$AdapterName/ /etc/netplan/$(hostname)-config.yaml


netplan generate
netplan apply

#
echo Network configuration generated and applied
echo Creating bridge and adding port

# Now,  if there is an existing ovs bridge br0, clean up and apply fresh
if [ ! -z "$(ovs-vsctl list-br)" ]; then

  # remove bridge (free up adapter)
  echo Removing existing bridge: br0
  ovs-vsctl del-br br0
fi


# Add br0 to ovs-vsctl and connect to physical adapter
echo Creating br0 and adding $AdapterName
ovs-vsctl add-br br0 -- add-port br0 $AdapterName

# Confirm
Bridges="$(ovs-vsctl list-br)"
if [ -z "$Bridges" ]; then
  echo ERROR:  Unable to create bridge 'br0'.  Setup manually and add '$AdapterName' as a port on the bridge.
  exit 1
fi

# Did adapter get attached?
Adapters="$(ovs-vsctl list-ports br0 | grep -v veth )"
if [ "$Adapters" != "$AdapterName" ]; then
  echo ERROR:  Bridge 'br0' does not have $AdapterName attached.   It has: $Adapters.
  echo "        Fix manually"
  exit 1
fi


# Now restart the host and proceed to step 2
echo Now reboot the host.
echo CONFIRM:  'br0' is the default adapter and has IP address previously assigned this adapter


[chuck@lizum plexqa.2004]$
1 Like

Do you need LXD-setup ?

LXD-setup.
– Setup is type ‘dir’ on local file system (move with lxd stopped as required
– script is looking for br0 as predefined bridge adapter to use.

[chuck@lizum plexqa.2005]$ cat LXD-setup
#!/bin/bash

if [ "$(whoami)" != "root" ]; then
  echo ERROR: Must be 'root' user
  exit
fi

sudo snap remove --purge lxd
sudo snap install lxd


# LXD init
lxd init <<EOF
no
yes
default
dir
no
no
yes
br0
yes
all
8443
yes
yes
EOF

# Now install LXDware
lxc launch images:ubuntu/22.04 lxd-dashboard
lxc exec lxd-dashboard /bin/bash <<EOF
apt update && apt install wget nginx php-fpm php-curl sqlite3 php-sqlite3 -y 
wget https://github.com/lxdware/lxd-dashboard/archive/v3.7.0.tar.gz
tar -xzf v3.7.0.tar.gz
cp -a lxd-dashboard-3.7.0/default /etc/nginx/sites-available/
cp -a lxd-dashboard-3.7.0/lxd-dashboard /var/www/html/

# Update from Ubuntu 20.04 -> 22.04
sed -i 's|fastcgi_pass unix:/run/php/php7.4-fpm.sock;|#fastcgi_pass unix:/run/php/php7.4-fpm.sock;|'  /etc/nginx/sites-available/default
sed -i 's|#fastcgi_pass unix:/run/php/php8.1-fpm.sock;|fastcgi_pass unix:/run/php/php8.1-fpm.sock;|'  /etc/nginx/sites-available/default

# Create directories
mkdir -p /var/lxdware/data/sqlite
mkdir -p /var/lxdware/data/lxd
mkdir -p /var/lxdware/backups

# Ownership
chown -R www-data:www-data /var/lxdware/
chown -R www-data:www-data /var/www/html

# Start web server
systemctl restart nginx

# ALL DONE
exit

EOF

# Set defaults
echo 'Setting default network profile for LXD network (bridge)'
lxc profile device set default eth0 nictype bridged
lxc profile device set default eth0 parent br0

#
#
#
echo  CONFIRM:  lxd-dashboard has an IP address and open it in the browser
echo  Warning:  Initial username will be the admin.  Recommend 'plexqa' / OurDefaultPassword

[chuck@lizum plexqa.2006]$
1 Like

I probably will. Thanks a ton for all of this

steps:

  1. Network setup
  2. LXD setup
  3. lxc launch ubuntu:22.04 ubuntu-test
  4. lxc stop ubuntu-test
  5. Add the GPU
  6. Add the disk devices for media.

SetUser (UID/GID idmap)

[chuck@lizum plexqa.2009]$ cat SetUser 
#!/bin/bash
# set uid and gid for container

# specified a container name  ($1)
[ -z "$1" ] && echo "Usage: $0 ContainerName Username " && exit 1
[ -z "$2" ] && echo "Error: Username missing" && exit 2

# Confirm container
if [ "$(lxc list | grep "$1")" == "" ]; then
  echo "Error: Unknown container name '$1'"
  exit 2
fi

# make sure user exists
if [ "$(id "$2")" = "" ]; then

  echo \"$1\" is not a valid username
  exit 1

fi

Uid=$(id -u "$2")
Gid=$(id -g "$2")

# Now set for container ($1)
if [ "$(lxc list | grep "$1")" = "" ]; then
  echo \"$1\" is not a valid container name
  exit 2
fi

echo Setting  "uid $Uid 0\ngid $Gid 0" in container $2

printf "uid $Uid 0\ngid $Gid 0" | lxc config set "$1" raw.idmap -

echo Restarting container
lxc restart $1


[chuck@lizum plexqa.2010]
1 Like

Update: Did a complete re-install, tried running plex in a docker container on bare-metal OMV. Works flawlessly after updating to the 6.2 proxmox kernel and installing the latest intel gpu drivers from bullseye-backports. I still get that weird error about nodevicesfound but there is no openCL error, and I was able to do plenty of simultaneous 4K HDR transcodes on 1.32.6 and 1.32.7. Thanks for all the help :slight_smile:

Datapoint: SR-IOV does not seem to work with HDR. i5-13600k works with the latest plex revisions on all kinds of transcodes, provided you are simply running in a docker container on otherwise bare-metal.

Didn’t test on a supported bare-metal “regular OS” like Ubuntu server ?

That’s the datapoint I’m suggesting. It’s also where you’ll find LXD/LXC shines

SR-IOV is the “Mediation” for multiple kernels.

When you only have one kernel , there is no need for mediation outside the native kernel handlers.

so does this mean you have the rest of the geminilake issue sorted and it will be in some new release in the coming weeks? I noticed a recent release said it fixed some linux HW transcoding issues and pointed back to a thread about geminilake.

just checking where we are in the status. Thanks.

@djfriday13

Yes, we know how to resolve this. (GeminiLake)

There have been other fixes in the general transcoder (subtitles, etc)

The work we need to complete now is to change both the transcoder and PMS so both are in sync with PMS in control.

2 Likes

I did try it but I then gave docker containers a shot on bare metal and all worked well

thanks! i have been tracking but seeing some movement on geminilake but it was my understanding it wasn’t complete. no reason i need to update so i was just tracking to find my next stable release.

don’t like staying too far behind because i don’t like jumping up several major changes at once.

thanks again for all the help you provide. always appreciated.

I’m running Truecharts/TrueNAS scale on an 8700k, and held off on upgrading due to this issue. Is it resolved now, and safe to update?

@pirateguybrush_gmail_com

The Intel “Core” CPUs KabyLake (-7xxx) and above do not have the problem.
You have a CoffeeLake (-8xxx)

1 Like

Finally, a true and good update on what all is left for GeminiLake. Thanks for the work! Hopefully the patching won’t be too long from now. And good to see subtitles also getting a fix.

The subtitle work is “new development”.

  1. Image based subtitles, when I finish working out bullet-proof command line args will be easiest - but still not ‘cookie-cutter’ easy

  2. It’s more involved to
    – Render Text-based subtitles as images (for burning)
    – Fold that image stream back into the pipeline in real time & in sync

So does this mean that PGS burning in will actually work and not buffer like crazy on most devices? I would love to get PGS to work since they are the cleanest integration into any video.

It’s funny how really the only type of subtitles that come out on Blu-rays still doesn’t have much support outside of Blu-ray players lol.

Hi, I’m just checking in on the progress of this issue. I’m a little late but it seemed from this thread that my setup should be working but its not. Seems to be failing on HW accel falling back to CPU, this setup worked in the past.

EDIT: Update - I noticed that my client I was downloading (iPad) was set to download original quality video/audio. Changing it to 1080p High seems to invoke the GPU Transcoding. So why is CPU Transcoding occuring with the original quality.

The file format source details:

<Media id="27349" duration="7028104" bitrate="40979" width="3840" height="1608" aspectRatio="2.35" audioChannels="6" audioCodec="dca-ma" videoCodec="hevc" videoResolution="4k" container="mkv" videoFrameRate="24p" audioProfile="ma" videoProfile="main 10">
<Part accessible="1" exists="1" id="27628" key="/library/parts/27628/1626427928/file.mkv" duration="7028104" file="/media/movies/Split (2016)/Split (2016) [imdb-tt4972582][Bluray-2160p][HDR10][DTS-HD MA 5.1][x265]-W4NK3R.mkv" size="35919151248" audioProfile="ma" container="mkv" deepAnalysisVersion="6" indexes="sd" requiredBandwidths="76952,74094,71620,68266,65345,62548,51871,48069" videoProfile="main 10">
<Stream id="80422" streamType="1" default="1" codec="hevc" index="0" bitrate="37431" language="English" languageTag="en" languageCode="eng" bitDepth="10" chromaLocation="topleft" chromaSubsampling="4:2:0" codedHeight="1608" codedWidth="3840" colorPrimaries="bt2020" colorRange="tv" colorSpace="bt2020nc" colorTrc="smpte2084" frameRate="23.976" height="1608" level="153" original="1" profile="main 10" refFrames="1" requiredBandwidths="73290,70451,68093,64747,61824,59027,48363,44227" title="Split.2016.2160p.UHD.BluRay.DTS-HD.MA.5.1.HDR.x265-W4NK3R" width="3840" displayTitle="4K HDR10 (HEVC Main 10)" extendedDisplayTitle="Split.2016.2160p.UHD.BluRay.DTS-HD.MA.5.1.HDR.x265-W4NK3R (4K HDR10 HEVC Main 10)"> </Stream>
<Stream id="80423" streamType="2" selected="1" default="1" codec="dca" index="1" channels="6" bitrate="3408" language="English" languageTag="en" languageCode="eng" audioChannelLayout="5.1(side)" bitDepth="24" original="1" profile="ma" requiredBandwidths="4059,3897,3739,3645,3558,3547,3547,3547" samplingRate="48000" displayTitle="English (DTS-HD MA 5.1)" extendedDisplayTitle="English (DTS-HD MA 5.1)"> </Stream>
<Stream id="80424" streamType="3" codec="pgs" index="2" bitrate="31" language="English" languageTag="en" languageCode="eng" headerCompression="1" original="1" requiredBandwidths="72,72,72,72,72,72,72,72" displayTitle="English (PGS)" extendedDisplayTitle="English (PGS)"> </Stream>
<Stream id="80425" streamType="3" codec="pgs" index="3" bitrate="28" language="German" languageTag="de" languageCode="deu" headerCompression="1" requiredBandwidths="83,83,83,83,83,83,83,83" displayTitle="German (PGS)" extendedDisplayTitle="German (PGS)"> </Stream>
<Stream id="80426" streamType="3" codec="pgs" index="4" bitrate="26" language="Italian" languageTag="it" languageCode="ita" headerCompression="1" requiredBandwidths="56,56,56,56,56,56,56,56" displayTitle="Italian (PGS)" extendedDisplayTitle="Italian (PGS)"> </Stream>
<Stream id="80427" streamType="3" codec="pgs" index="5" bitrate="27" language="Spanish" languageTag="es" languageCode="spa" headerCompression="1" requiredBandwidths="64,64,64,64,64,64,64,64" displayTitle="Spanish (PGS)" extendedDisplayTitle="Spanish (PGS)"> </Stream>
<Stream id="80428" streamType="3" codec="pgs" index="6" bitrate="28" language="Turkish" languageTag="tr" languageCode="tur" headerCompression="1" requiredBandwidths="73,73,73,73,73,73,73,73" displayTitle="Turkish (PGS)" extendedDisplayTitle="Turkish (PGS)"> </Stream>
<Stream id="80549" key="/library/streams/80549" streamType="3" codec="srt" language="English" languageTag="en" languageCode="eng" format="srt" displayTitle="English (SRT External)" extendedDisplayTitle="English (SRT External)"> </Stream>
</Part>
</Media>

Unraid 6.12.4
i5-12500 Alder Lake
Linux 6.1.49
Plex LSIO latest beta 1.32.7.7571

Debug logs attached, let me know if I can do anything to troubleshoot.

Plex Media Server Logs_2023-10-16_13-41-26.zip (531.3 KB)

Thank you for including the XML with your post.

PGS subtitles (any image-based subs) will block use of HW transcoding.
(There is currently no means to burn subtitles in hardware)

That said, did you pass all of /dev/dri into the container?

Oct 16, 2023 13:40:29.916 [22903591152440] DEBUG - [Req#142/Transcode] TPU: hardware transcoding: enabled, but no hardware decode accelerator found
Oct 16, 2023 13:40:29.916 [22903591152440] DEBUG - [Req#142/Transcode] [Universal] Using local file path instead of URL: /media/movies/Unbreakable (2000)/Unbreakable (2000) [imdb-tt0217869][Bluray-2160p][HDR10][DTS-HD MA 5.1][x265]-NOKITKAT.mkv
Oct 16, 2023 13:40:29.916 [22903591152440] DEBUG - [Req#142/Transcode/HCl#4d] HTTP requesting GET http://127.0.0.1:32400/library/streams/80421?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx
Oct 16, 2023 13:40:29.917 [22903596055352] DEBUG - Request: [127.0.0.1:47122 (Loopback)] GET /library/streams/80421 (18 live) #144 GZIP Signed-in Token (timpotter) (iPad)
Oct 16, 2023 13:40:29.918 [22903596055352] DEBUG - [Req#144] Calculated media file path for path [file:///media/movies/Unbreakable%20(2000)/Unbreakable%20(2000)%20[imdb-tt0217869][Bluray-2160p][HDR10][DTS-HD%20MA%205.1][x265]-NOKITKAT.en.srt]: ["/media/movies/Unbreakable (2000)/Unbreakable (2000) [imdb-tt0217869][Bluray-2160p][HDR10][DTS-HD MA 5.1][x265]-NOKITKAT.en.srt"]
Oct 16, 2023 13:40:29.918 [22903596055352] DEBUG - Content-Length of /media/movies/Unbreakable (2000)/Unbreakable (2000) [imdb-tt0217869][Bluray-2160p][HDR10][DTS-HD MA 5.1][x265]-NOKITKAT.en.srt is 64086 (of total: 64086).
Oct 16, 2023 13:40:29.922 [22903593261880] DEBUG - Request: [10.1.50.194:64788 (WAN)] GET /library/metadata/10656 (18 live) #13a TLS GZIP Signed-in Token (timpotter) (iPad)
Oct 16, 2023 13:40:29.924 [22903593261880] DEBUG - [Req#13a] It took 0.000000 ms to retrieve 113 items.
Oct 16, 2023 13:40:29.925 [22903593261880] DEBUG - We're going to try to auto-select an audio stream for account 1.
Oct 16, 2023 13:40:29.925 [22903593261880] DEBUG - Selecting best audio stream for part ID 27628 (language: )
Oct 16, 2023 13:40:29.925 [22903593261880] DEBUG - Audio Stream: 80423, Subtitle Stream: -1
Oct 16, 2023 13:40:29.926 [22903707863864] DEBUG - Completed: [10.1.50.194:64788] 200 GET /library/metadata/10656 (18 live) #13a TLS GZIP 3ms 4856 bytes (pipelined: 1)
Oct 16, 2023 13:40:29.933 [22903707863864] DEBUG - Completed: [127.0.0.1:47122] 200 GET /library/streams/80421 (18 live) #144 GZIP 16ms 64086 bytes (pipelined: 1)
Oct 16, 2023 13:40:29.933 [22903676824376] DEBUG - [HttpClient/HCl#4d] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:32400/library/streams/80421?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx
Oct 16, 2023 13:40:29.934 [22903591152440] DEBUG - [Req#142/Transcode] Downloaded stream from [http://127.0.0.1:32400/library/streams/80421?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx] (codec: srt) to temporary file [/transcode/Transcode/Sessions/plex-transcode-94bb22b303281c6d379b794f2bac4b102e0630b9-4fd88e31-e007-4d45-a55c-c9f7cd78fb3f/temp-0.srt]
Oct 16, 2023 13:40:29.934 [22903591152440] DEBUG - [Req#142/Transcode] TPU: hardware transcoding: final decoder: , final encoder: 

Thank you, yes /dev/dri is mapped.

Is there a way to prefer hw transcoding over burning in pgs subtitles besides stripping them from the sources?

Worked for me on a QNAP TS-453D. Thanks! The VaapiDriver="i965" attribute was already present on the Preferences.xml file, so I had to remove the duplicate I had just added.

ALL:

Updating you on our progress.

  1. We are opening a Forum Preview of the updated PMS / Transcoder changes for Apollo Lake, Gemini Lake, and older CPUs (Sky Lake and below)

  2. We would greatly appreciate one final test from as many as have time to confirm we’re ready to ship.

Our Preview thread is here.

1 Like