Linux Packaging - Developer's Sandbox - HDR tonemapping packaging support

Please understand what my goal here is.

  1. This is only the packaging. It does not provide any information to PMS.

  2. I have written it this way to help you (the admin) be aware of:
    a. Any local configuration issues (the base Debian package)
    b. HDR Tone mapping dependencies for sub-9xxx CPUs which need them (Beignet)
    c. HDR Tone mapping dependencies for 9xxx and above.

  3. Given:
    a. Processors -7xxx (KabyLake) and -8xxx (CoffeeLake) rely on Beignet for tone mapping.
    b. Processors -9xxx and above rely on Intel Compute Runtime for their tonemapping.

  4. My goal with this packaging update is to take as much “guesswork” as possible out of installing extra packages to support HDR tone mapping for those who don’t necessarily understand the intricacies of why one is needed over another.

  5. When PMS starts, specifically the transcoder, knowing the video it has to transcode, it checks to see which runtime libraries are present. If the right ones are installed, HDR tone mapping will occur. ( at least that’s the plan :wink: )

Regarding the reports of Tonemapping failing when enabled but working when disabled, I am working with the transcoding team to get that all sorted out. I relay the information to them.

What will be of most help is creating discrete DEBUG (not VERBOSE) log file ZIP sets.

  1. HDR disabled but working
    -followed by-
  2. HDR enabled, transcoding the same file, but failing.

This applies to all processors , KabyLake and above.

If we can do that, while I finish this packaging (which is about to be put into a formal build for testing here), then we’ll be ready to go by addressing both facets concurrently (packaging and runtime)

Any form of pass/fail matrix we can create would be outstanding.
It’ll paint the landscape and hopefully make focusing in on the fault easier.

1 Like

@ChuckPa

Apr 06, 2021 11:16:49.028 [0x7faddeffd700] ERROR - [Transcoder] [AVHWDeviceContext @ 0xbe7140] No matching devices found.
Apr 06, 2021 11:16:49.028 [0x7faddeffd700] ERROR - [Transcoder] [Parsed_hwmap_2 @ 0x14fa2c0] Failed to created derived device context: -19.
Apr 06, 2021 11:16:49.028 [0x7faddeffd700] ERROR - [Transcoder] [Parsed_hwmap_2 @ 0x14fa2c0] Failed to configure output pad on Parsed_hwmap_2
Apr 06, 2021 11:16:49.029 [0x7faddeffd700] ERROR - [Transcoder] Error reinitializing filters!
Apr 06, 2021 11:16:49.029 [0x7faddeffd700] ERROR - [Transcoder] Failed to inject frame into filter network: No such device
Apr 06, 2021 11:16:49.029 [0x7faddeffd700] ERROR - [Transcoder] Error while processing the decoded data for stream #0:0
Apr 06, 2021 11:16:49.042 [0x7fadf7153700] DEBUG - Jobs: '/usr/lib/plexmediaserver/Plex Transcoder' exit code for process 52937 is 1 (failure)
Apr 06, 2021 11:16:49.042 [0x7fadcbfff700] DEBUG - Streaming Resource: Changing client to use software decoding

Can I PM you the logs? I have with HDR tone mapping off, HW transcoding works, turned on HDR tone mapping, HW transcoding fails.

@emb531

You can PM me the logs but Engineering is already aware.

A note for those who just want HW transcoding of HDR content working… I’ve tried a few different builds, including the latest GA (general availability, public build), as well as the one linked in this thread. And neither actually gives me HW transcoding of HDR content. However, there is a build that does work. It’s this one:
plexmediaserver_1.21.1.3830-6c22540d5_amd64.deb

You can find it at this link: plexmediaserver_1.21.1.3830-6c22540d5_amd64.deb - Google Drive

Note that the build that actually works, does NOT mention the newer Intel libs during install… Instead, it mentions the older beignet one, and the opencl library. But if you install the latest Intel libs from Releases · intel/compute-runtime · GitHub , then the above plex media server build may work for you.

I’m using the intel i5-10400 cpu, with no graphics card, and HW transcoding of 4k HDR content totally works, but ONLY using the build linked in this post.

1 Like

That said, I’m happy to try other builds.

@ChuckPa , the build you linked above, seemed to do quite well regarding identifying which libraries I need, and telling me whether they were already installed.

1 Like

Gang,

Been a bit busy sorting things out.

I’m about ready to spin a new one.
There shouldn’t be any changes other than those inherent to PMS

If nothing fails from the packaging perspective – meaning I identify everything correctly – then I’ll send this over for final QA and release to everyone.

There is a release of PMS making its way through testing now which fixes one significant problem where HW transcoding cuts out / is inconsistent. I’ll know more when it comes to me to rattle

1 Like

I’m running the same chip (on a z490 board) and Version 1.22.0.4163 works perfectly for me. After that is where the drama begins for me. Sounds like things are happening though so hopefully not too much longer now. I only recently got the 10400 as I was giving it plenty of time to be confirmed working, which it was. Damn new chips really toppled things over…

2 Likes

ALL:

With the recent developments surrounding

  1. HW transcoding,
  2. HDR tonemapping,
  3. Beignet installation requirement regardless of ICR status
  4. “Flip-Flop” of HW transcoding when HDR enabled

I’d like to finalize packaging and make certain this is production ready

I cannot address functional issues with CometLake or RocketLake CPUs.

I’m only verifying the installer is capable of properly detecting the CPU and capabilities.

It should still detect:

  1. Native, Docker, LXC, or VM
  2. systemd, init, or custom
  3. Intel i915 visibility (required for va-api)
  4. Nvidia GPU visibility
  5. All local site customizations from /etc/default/plexmediaserver or override.conf
  6. All previous packaging capability / reporting should be as always.
  7. Should be able to update from within a container.

I would appreciate help one last time with this to make sure it is where it needs to be.
I only have a 9th gen (Hades Canyon NUC8) to work with here at this time.

Prior to install, I did a sudo apt purge plexmediaserver

sudo dpkg -i plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb
Selecting previously unselected package plexmediaserver.
(Reading database ... 172451 files and directories currently installed.)
Preparing to unpack plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.22.0.4163-d8c4875dd) ...
Setting up plexmediaserver (1.22.0.4163-d8c4875dd) ...
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         render
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /dev/shm  (set in Preferences.xml)
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Processor:           Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
logger: journald entry could not be written
dpkg: error processing package plexmediaserver (--install):
 installed plexmediaserver package post-installation script subprocess returned error exit status 1
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for mime-support (3.64ubuntu1) ...
Errors were encountered while processing:
 plexmediaserver

Post install, the Plex service was not running, I had to start manually via systemctl

cat plexinstaller.log
# Plex Media Server installation configuration info:  Thu 08 Apr 2021 11:54:52 AM EDT
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/dev/shm"
LangEncoding="en_US.UTF-8"
ExistingVersion=0
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
CPUSKU=10600
NeedBeignet=0
NeedIntelCompute=1
HaveIntelGmmlib=1
HaveIntelIGCCore=1
HaveIntelIGCOpenCL=1
HaveIntelOpenCL=1
HaveIntelOCLoc=1
HaveIntelZeroGPU=1
HaveIntelCompute=1
HaveBeignetIcd=0
HaveOpenCL=1
HaveBeignet=0
Platform="System Product Name"
Processor="Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux Saturn 5.11.12-051112-generic #202104071432 SMP Wed Apr 7 14:50:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="31934 1342 19695"

Let me check.

I didn’t get anywhere near that code.

EDIT


  # Output to log
  if [ $Systemd -eq 1 ]; then
    echo MESSAGE="$Message" | logger --journald
  else
    echo "$Message" | logger
  fi
}

Nothing touched here.

Added:



  Output "  Plex User:           $PlexUser"
  Output "  Plex Group:          $PlexGroup"
  Output "  Video Group:         $VideoGroup"
  Output "  Metadata Dir:        $AppSuppDir"
  Output "  Temp Directory:      $PlexTempDir " "$HaveTranscoderPref"
  Output "  Lang Encoding:       $LangEncoding"
  Output "  Processor:           $Processor"

Digging.

Yes, thanks, the one you mentioned works for me.

Same issue as before:

Preparing to unpack plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.22.0.4163-d8c4875dd) over (1.22.0.4163-d8c4875dd) ...
Setting up plexmediaserver (1.22.0.4163-d8c4875dd) ...
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation starting.
PlexMediaServer install: 
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         render
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /tmp 
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Processor:           Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
logger: journald entry could not be written
dpkg: error processing package plexmediaserver (--install):
 installed plexmediaserver package post-installation script subprocess returned error exit status 1
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...
Processing triggers for desktop-file-utils (0.24-1ubuntu3) ...
Processing triggers for mime-support (3.64ubuntu1) ...
Errors were encountered while processing:
 plexmediaserver

I just fixed that. I know I did.

I double checked it. I even purged plexmediaserver, same was what emb531 did. I even downloaded it twice as part of my double-checking. :slight_smile: Downloaded twice, installed twice. Same issue both times.

Hmmm… I hope I’m not going crazy.

At least you’ll have company :crazy_face:

:rofl:

Something stupid isn’t right. I need to slow down and walk through it.

1 Like

Processor removed.

chuck@plexdev:~/git/output-all$ sudo dpkg -i plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb
(Reading database ... 187339 files and directories currently installed.)
Preparing to unpack plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Warning:  Multiple override files found. PMS may not function as expected.
PlexMediaServer install:           Files found are:  Another.conf override.conf
PlexMediaServer install:           Selected: "override.conf".
PlexMediaServer install: Warning:  "/var/lib/plexmediaserver/Library/Application Support" isn't owned by "plex", UID: 998.  Found "UNKNOWN", UID: 12345 instead. Continuing.
PlexMediaServer install: Pre-installation Validation complete.  Errors: 0, Warnings: 2
Unpacking plexmediaserver (1.22.0.4163-d8c4875dd) over (1.22.0.4163-d8c4875dd) ...
Setting up plexmediaserver (1.22.0.4163-d8c4875dd) ...
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation starting.
PlexMediaServer install: 
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         video
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /tmp 
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Config file used:    /etc/systemd/system/plexmediaserver.service.d/override.conf
PlexMediaServer install:   Intel QSV Hardware:  Not found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:  
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation successful.  Errors: 0, Warnings: 2
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
chuck@plexdev:~/git/output-all$ 

Let’s continue:

I will release this to QA / production – barring any more gremlins.

1 Like

This one worked for me!

sudo dpkg -i plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb
Selecting previously unselected package plexmediaserver.
(Reading database ... 172451 files and directories currently installed.)
Preparing to unpack plexmediaserver_1.22.0.4163-d8c4875dd_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.22.0.4163-d8c4875dd) ...
Setting up plexmediaserver (1.22.0.4163-d8c4875dd) ...
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         video
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /dev/shm  (set in Preferences.xml)
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Intel QSV Hardware:  Not found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: Starting Plex Media Server.
PlexMediaServer install: PlexMediaServer-1.22.0.4163-d8c4875dd - Installation successful.  Errors: 0, Warnings: 0
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for mime-support (3.64ubuntu1) ...

cat /tmp/plexinstaller.log
# Plex Media Server installation configuration info:  Thu 08 Apr 2021 02:59:46 PM EDT
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="video"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/dev/shm"
LangEncoding="en_US.UTF-8"
ExistingVersion=0
HaveHardware=0
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=0
HaveIntelCore=0
HaveIntelXeon=0
Havei915=0
HaveNvidia=0
Processor=i5-10600K
CPUSKU=0
NeedBeignet=0
NeedIntelCompute=0
HaveIntelGmmlib=0
HaveIntelIGCCore=0
HaveIntelIGCOpenCL=0
HaveIntelOpenCL=0
HaveIntelOCLoc=0
HaveIntelZeroGPU=0
HaveIntelCompute=0
HaveBeignetIcd=0
HaveOpenCL=0
HaveBeignet=0
Platform="System Product Name"
Processor="i5-10600K"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux Saturn 5.11.12-051112-generic #202104071432 SMP Wed Apr 7 14:50:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="31934 1503 19381"

I have no idea why passing the CPU “string” doesn’t work.
There is no logical reason for it when the same exact text can be pasted to the shell command line (sh or bash) and it works flawlessly.

So be it.

Sorry, I tried.

Is this a VM >?

This isn’t detecting the i915

HaveIntelXeon=0
Havei915=0
HaveNvidia=0
Processor=i5-10600K

No it is bare metal. Perhaps it is the kernel? I updated it to the newest available mainline kernel (5.11.12-051112-generic)

sudo vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD