Linux Packaging - Developer's Sandbox - HDR tonemapping packaging support

ChuckPa · March 10, 2021, 4:01pm

ALL:

This is a little play space to work on the new packaging updates.

I’m going to use the Debian preinst & postinst model for now.
I can easily transfer these into the %spec for RPMs when done.

The goal is to:

Determine what can be seen in the hardware
Ensure the detection is correct for all environments (Bare, Container, and VM)

Below, please find preinst.

Written to work on Debian / Ubuntu – for now
Should parse the CPU type correctly
Should get a clean -xxxxx parsed
Should then correctly determine if Beignet/OpenCL -or- Intel Compute Runtime
All results written to /tmp/plexinstaller.log

It looks like this:

[chuck@lizum ~/icr.243]$ cat /tmp/plexinstaller.log 
# Plex Media Server installation configuration info:  Wed Mar 10 10:49:25 AM EST 2021
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=1
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=0
HaveHardware=1
HaveTranscoderPref=0
NeedUser=1
NeedGroup=1
NeedVideo=1
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=8809
NeedBeignet=0
NeedIntelCompute=1
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveLibigdmm11=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform="NUC8i7HVK"
Processor="Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz"
Distro="Fedora 33 (Workstation Edition)"
Kernel="Linux lizum 5.10.21-200.fc33.x86_64 #1 SMP Mon Mar 8 00:24:40 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="32024 3959 15569"
[chuck@lizum ~/icr.244]$

The script (runs as root for accurate reporting) :

Please read further down for the latest version.

Updates:

Changed inventory logic
Keep Beignet for everything below CometLake (-10xxx)
Corrected Intel Compute Runtime package names

Please share how well this parses your host

Thanks.

emb531 · March 10, 2021, 4:49pm

So it seems to be incorrectly detecting my system needs Intel Compute (currently I have i7-7700K with Beignet/OpenCL installed

# Plex Media Server installation configuration info:  Wed 10 Mar 2021 11:43:29 AM EST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/dev/shm"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=7700
NeedBeignet=0
NeedIntelCompute=1
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveLibigdmm11=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform="C7Z170-OCE"
Processor="Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz"
Distro="Debian GNU/Linux 10 (buster)"
Kernel="Linux Godzilla 5.10.0-4-amd64 #1 SMP Debian 5.10.19-1 (2021-03-02) x86_64 GNU/Linux"
Memory="32029 2501 22788"

dpkg -l | grep opencl
ii  beignet-opencl-icd:amd64         1.3.2-6+b10                       amd64        OpenCL library for Intel GPUs
ii  ocl-icd-libopencl1:amd64         2.2.12-2                          amd64        Generic OpenCL ICD Loader

ChuckPa · March 10, 2021, 5:17pm

Actually, that’s a correct Need but the Have detection seems off.

In the new Intel-Compute-Runtime, KabyLake (being HEVC/HDR capable) will be ICR-based. Beignet/OpenCL won’t be used.

I do need look at why it didn’t detect already installed though.

I’m going to change to allow Beignet for anything under CometLake unless Engineering says otherwise. That will be compatible with what exists.

ChuckPa · March 10, 2021, 5:38pm

@emb531

Updated the logic, care to give that a spin?

It should now detect your existing Beignet as well as define as NeedBeignet.

emb531 · March 10, 2021, 6:03pm

Looks good now!

# Plex Media Server installation configuration info:  Wed 10 Mar 2021 01:02:37 PM EST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/dev/shm"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=7700
NeedBeignet=1
NeedIntelCompute=0
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveLibigdmm11=0
HaveZeroIntelGPU=0
HaveBeignet=1
HaveOpenCL=1
HaveDependencies=1
HaveIntelCompute=0
Platform="C7Z170-OCE"
Processor="Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz"
Distro="Debian GNU/Linux 10 (buster)"
Kernel="Linux Godzilla 5.10.0-4-amd64 #1 SMP Debian 5.10.19-1 (2021-03-02) x86_64 GNU/Linux"

millercentral · March 10, 2021, 6:09pm

Here is mine, seems correct (AMD Threadripper + Nvidia GPU, native install):

# Plex Media Server installation configuration info:  Wed 10 Mar 2021 10:05:57 AM PST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="video"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=0
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=0
HaveIntelCore=0
HaveIntelXeon=0
Havei915=0
HaveNvidia=1
QSVClass=
NeedBeignet=0
NeedIntelCompute=0
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveLibigdmm11=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform="MS-7C60"
Processor="AMD Ryzen Threadripper 3960X 24-Core Processor"
Distro="Ubuntu 20.04.1 LTS"
Kernel="Linux skynet 5.8.0-40-generic #45~20.04.1-Ubuntu SMP Fri Jan 15 11:35:04 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="64257 4527 56943"

(I don’t know what Verbose=1 means, but my by server logging is not set to verbose…)

CaptainRedHat · March 10, 2021, 6:32pm

Here’s what I have:

# Plex Media Server installation configuration info:  Wed 10 Mar 2021 06:31:02 PM UTC
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=10700
NeedBeignet=0
NeedIntelCompute=1
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveLibigdmm11=0
HaveZeroIntelGPU=0
HaveBeignet=1
HaveOpenCL=1
HaveDependencies=1
HaveIntelCompute=0
Platform="MS-7C83"
Processor="Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux umadbro 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="15877 877 13761"

I noticed it says HaveIntelOpenCLICD=0,HaveLibIGC=0,HaveLibigdmm11=0,HaveZeroIntelGPU=0, and HaveIntelCompute=0, but I do believe I have those packages installed:

ii  intel-gpu-tools                      1.25-2                                amd64        tools for debugging the Intel graphics driver
ii  intel-igc-core                       1.0.6087                              amd64        Intel(R) Graphics Compiler for OpenCL(TM)
ii  intel-igc-opencl                     1.0.6087                              amd64        Intel(R) Graphics Compiler for OpenCL(TM)
ii  intel-level-zero-gpu                 1.0.19096                             amd64        Intel(R) Graphics Compute Runtime
ii  intel-ocloc                          21.08.19096                           amd64        Intel(R) Graphics Compute Runtime
ii  intel-opencl                         21.08.19096                           amd64        Intel(R) Graphics Compute Runtime

ChuckPa · March 10, 2021, 10:07pm

Thanks everyone!

Verbose=1 is for package output. If zero, it would be terse (aka: dpkg style)
Intel packages detection corrected.

Newer updated script below

Please verify distinction betwen IntelCompute-OpenCL and Beignet-OpenCL-ICD

I have confirmed Beignet will be used for all -9xxx and lower processors. Neo (Intel Compute) will be used for -10xxx and above.

pshanew · March 11, 2021, 5:25am

This is the output from a system running Ubuntu 20.10. I run Plex Media Server in a Docker container on this system, but it has beignet-opencl-icd and intel-opencl-icd installed. Something interesting to note is that this is an Intel Compute Stick with an Atom x5-Z8300 processor; the script is detecting it as an 8300 “QSVClass.” While it does support QSV and works with hardware accelerated transcoding, its codec support is not that of a Coffee Lake processor.

# Plex Media Server installation configuration info:  Thu Mar 11 12:15:33 AM EST 2021
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=1
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=0
HaveHardware=1
HaveTranscoderPref=0
NeedUser=1
NeedGroup=1
NeedVideo=1
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=0
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=8300
NeedBeignet=0
NeedIntelCompute=0
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveOcloc=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform="STK1AW32SC"
Processor="Intel(R) Atom(TM) x5-Z8300  CPU @ 1.44GHz"
Distro="Ubuntu 20.10"
Kernel="Linux computestick 5.8.0-44-generic #50-Ubuntu SMP Tue Feb 9 06:29:41 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="1896 895 174"

TeknoJunky · March 11, 2021, 5:48am

for reference

cat /tmp/plexinstaller.log
# Plex Media Server installation configuration info:  Wed 10 Mar 2021 11:47:54 PM CST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=1
OverrideFile="/etc/systemd/system/plexmediaserver.service.d/override.conf"
PlexUser="plex"
PlexGroup="dvrapps"
VideoGroup="render"
AppSuppDir="/dataplex/pms/Library/Application Support"
PlexTempDir="/transcode"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=0
HaveIntelXeon=1
Havei915=0
HaveNvidia=1
QSVClass=
NeedBeignet=0
NeedIntelCompute=0
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveOcloc=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform="VB1416"
Processor="Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz"
Distro="Debian GNU/Linux bullseye/sid"
Kernel="Linux proximo 5.10.0-3-amd64 #1 SMP Debian 5.10.13-1 (2021-02-06) x86_64 GNU/Linux"
Memory="64371 14546 7935"

ChuckPa · March 11, 2021, 7:21am

@TeknoJunky

Does that E5 have QSV capability ? I didn’t see it but I easily could have missed it given how scrambled Xeon names are.

ChuckPa · March 11, 2021, 7:22am

Thank you @pshanew

Looks like I have some more work to do on inventory .

I was not thinking of how it looks in Docker.

Is the inventory correct? Atom CPUs don’t have QSV
It does have a GPU which can be used by Beignet .

TeknoJunky · March 11, 2021, 1:02pm

mine does not

pshanew · March 11, 2021, 5:31pm

I’m sorry, I should have been more clear here (translated: I really butchered that opening sentence). While I run Plex Media Server in a Docker container on this system, the output I posted was collected from running the preinst script on the system natively. When run from within the container the script detects it and exits properly.

Yes, the inventory is correct. This family of Atom processors (Cherry Trail) does indeed have Quick Sync Video features (though I don’t believe it was marketed as such). You have to really dig in the technical product specs to find it. It’s surprisingly capable for such a modest CPU. Here’s the output of vainfo:

vainfo: VA-API version: 1.8 (libva 2.8.0)
vainfo: Driver version: Intel i965 driver for Intel(R) CherryView - 2.4.0
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264MultiviewHigh      :	VAEntrypointVLD
      VAProfileH264MultiviewHigh      :	VAEntrypointEncSlice
      VAProfileH264StereoHigh         :	VAEntrypointVLD
      VAProfileH264StereoHigh         :	VAEntrypointEncSlice
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD

At any rate, I have /dev/dri passed through to my container and it performs hardware accelerated transcoding when asked to.

All that to say, the script correctly inventoried this processor’s video capabilities.

riiviin · March 11, 2021, 6:24pm

I am a Linux novice, but I think I added the preinst script correctly:

# Plex Media Server installation configuration info:  Thu 11 Mar 2021 12:19:53 PM CST
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="users"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=0
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
QSVClass=10710
NeedBeignet=0
NeedIntelCompute=1
HaveIntelOpenCLICD=1
HaveLibIGC=1
HaveOcloc=1
HaveZeroIntelGPU=1
HaveBeignet=1
HaveOpenCL=1
HaveDependencies=1
HaveIntelCompute=1
Platform="NUC10i7FNH"
Processor="Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux nuc 5.8.0-44-generic #50~20.04.1-Ubuntu SMP Wed Feb 10 21:07:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"

ChuckPa · March 11, 2021, 6:58pm

@riiviin

This isn’t something you add.

It’s a developer’s scribblings.

Put it in a directory
Run it as root
See what the results are
Compare with what your system actually has.

Being a novice, this is probably above your mastery level but by all means follow along and learn

JKalchik · March 11, 2021, 8:05pm

From a quick glance, this all looks reasonably accurate.

jkalchik@ceres:/tmp> sudo ./preinst
[sudo] password for root:
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
jkalchik@ceres:/tmp> cat /tmp/plexinstaller.log

Plex Media Server installation configuration info: Thu Mar 11 14:03:22 CST 2021

Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser=“plex”
PlexGroup=“plex”
VideoGroup=“video”
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding=“en_US.UTF-8”
ExistingVersion=0
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=0
HaveNvidia=1
QSVClass=
NeedBeignet=0
NeedIntelCompute=0
HaveIntelOpenCLICD=0
HaveLibIGC=0
HaveOcloc=0
HaveZeroIntelGPU=0
HaveBeignet=0
HaveOpenCL=0
HaveDependencies=0
HaveIntelCompute=0
Platform=“System Product Name”
Processor=“Intel® Core™ i9-10900F CPU @ 2.80GHz”
Distro=“openSUSE Leap 15.2”
Kernel=“Linux ceres 5.3.18-lp152.66-default #1 SMP Tue Mar 2 13:18:19 UTC 2021 (73933a3) x86_64 x86_64 x86_64 GNU/Linux”
Memory=“31923 1078 283”

riiviin · March 11, 2021, 8:06pm

Hilarious, that is certainly easier than what I did, but I am pretty sure it still worked. The log output matches the system.

ChuckPa · March 11, 2021, 8:17pm

@JKalchik

It failed to detect the “QSVClass” (which is really the CPU SKU )

I am almost done updating the scripting

ChuckPa · March 11, 2021, 8:26pm

~~Let’s run this again.~~

~~I’ve enabled the full verbosity at the end of preinst. This is normally only printed if failures.~~

~~Package names updated~~
~~Transfer variables named properly.~~

~~[ChuckPa-preinst.tar (3rd time’s the charm)|attachment]~~
~~(upload://fQuBJgtoCSKvJpMIVfUfxZKI00S.tar) (40 KB)~~

Topic		Replies	Views
PMS Packaging 07-2022 update - Forum Preview & Testing Desktops & Laptops server-linux	134	3998	January 18, 2023
Ubuntu 22.04 Packaging - development & preview Plex Media Server server-linux	171	3181	July 7, 2022
No hardware transcoding in Ubuntu server Plex Media Server server-linux	43	810	June 30, 2021
HW Transcoding artifacts Plex Media Server server-linux	49	653	September 1, 2021
HW acceleration of intel igp broken after latest upgrade Desktops & Laptops server-linux	19	413	August 21, 2022

Linux Packaging - Developer's Sandbox - HDR tonemapping packaging support

ALL:

The script (runs as root for accurate reporting) :

Please share how well this parses your host

Plex Media Server installation configuration info: Thu Mar 11 14:03:22 CST 2021

Related topics