Linux Packaging - Developer's Sandbox - HDR tonemapping packaging support

@timstephens24

Thank you for checking that.
It so unfortunate we couldn’t get lucky here.

I knew it was experimental but … DRATS :frowning:

At least we can confirm the packaging is ready

At first chance I get, I will snag a working binary

1 Like

Progress is still progress :slight_smile:

If they want the crash file let me know, I have it saved off.

Thanks but I think we’re good. This is a pretty obvious failure.

If I can recreate it then it must be bad :rofl:

1 Like

Did this actually get rolled out? Just trying to get my intel 10400 an an up-to-date installation again but can’t see anything that was released that brings it back to 4163 working function again so far.

# Plex Media Server installation configuration info:  Sun Apr 25 05:27:05 AM EDT 2021plexmediaserver_1.22.3.4392-d7c624def_amd64.deb
LangEncoding="en_US.UTF-8"
ExistingVersion=12203
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=0
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=Intel(R)
CPUSKU=0
NeedBeignet=0
NeedIntelCompute=0
HaveIntelGmmlib=0
HaveIntelIGCCore=0
HaveIntelIGCOpenCL=0
HaveIntelOpenCL=0
HaveIntelOCLoc=0
HaveIntelZeroGPU=0
HaveIntelCompute=0
HaveBeignet=0
HaveOpenCL=0
Platform="Z590 UD"
Processor="Intel(R)"
Distro="Ubuntu 21.04"
Kernel="Linux mooncake 5.11.13-051113-generic #202104101231 SMP Sat Apr 10 12:34:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="15801 6798 156"

Doesn’t look like any of the flag detection parts are working on my system as I have all that stuff installed:

dpkg -l | grep "intel"
ii  intel-gmmlib                                20.4.1                                                               amd64        Intel(R) Graphics Memory Management Library Package
ii  intel-igc-core                              1.0.6812                                                             amd64        Intel(R) Graphics Compiler for OpenCL(TM)
ii  intel-igc-opencl                            1.0.6812                                                             amd64        Intel(R) Graphics Compiler for OpenCL(TM)
ii  intel-level-zero-gpu                        1.1.19533+i538~u20.04                                                amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.
ii  intel-media-va-driver-non-free:amd64        21.1.3+i538~u20.04                                                   amd64        VAAPI driver for the Intel GEN8+ Graphics family
ii  intel-microcode                             3.20201110.0ubuntu1                                                  amd64        Processor microcode firmware for Intel CPUs
ii  intel-ocloc                                 21.15.19533                                                          amd64        Intel(R) Graphics Compute Runtime
ii  intel-opencl                                21.15.19533                                                          amd64        Intel(R) Graphics Compute Runtime
rc  intel-opencl-icd                            20.13.16352-1                                                        amd64        Intel graphics compute runtime for OpenCL
ii  libdrm-intel1:amd64                         2.4.104+i526~u20.04                                                  amd64        Userspace interface to intel-specific kernel DRM services -- runtime
ii  xserver-xorg-video-intel                    2:2.99.917+git20200714-1ubuntu1                                      amd64        X.Org X server -- Intel i8xx, i9xx display driver

I looked inside the script and it appears the script is incorrectly assuming the processor number is always the third item in the list returned by cpuinfo, my cpuinfo reads: (Only showing core 0)

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 167
model name	: 11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz
stepping	: 1
microcode	: 0x3c
cpu MHz		: 3900.000
cache size	: 12288 KB
physical id	: 0
siblings	: 12
core id		: 0
cpu cores	: 6
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 27
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap avx512ifma clflushopt intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm md_clear flush_l1d arch_capabilities
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_mode_based_exec tsc_scaling
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs
bogomips	: 7824.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

Here @ChuckPa , I’ve fixed the script to work on Gen 11 processors.

JerryD-preinst.zip (8.8 KB)

# Plex Media Server installation configuration info:  Sun Apr 25 09:37:18 AM EDT 2021plexmediaserver_1.22.3.4392-d7c624def_amd64.deb
LangEncoding="en_US.UTF-8"
ExistingVersion=12203
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=0
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=i5-11600K
CPUSKU=11600
NeedBeignet=0
NeedIntelCompute=1
HaveIntelGmmlib=1
HaveIntelIGCCore=1
HaveIntelIGCOpenCL=1
HaveIntelOpenCL=1
HaveIntelOCLoc=1
HaveIntelZeroGPU=1
HaveIntelCompute=1
HaveBeignet=0
HaveOpenCL=1
Platform="Z590 UD"
Processor="i5-11600K"
Distro="Ubuntu 21.04"
Kernel="Linux mooncake 5.11.13-051113-generic #202104101231 SMP Sat Apr 10 12:34:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="15801 7045 224"

@RoadRacer12

thanks for those mods. since that first script, I’ve done some rework.

I like your idea and will incorporate into how I do inventory.

Working/failing for 11K CPUs isn’t , AFAIK, based on what I do in the package.
The actual problem is upstream with the i915 driver itself.

Yes, I can and will make certain the packaging is appropriate but from Intel’s documentation:

  1. ocl-icd-opencl1 is needed for either solution
  2. Beignet is used for all -2xxx through -8xxx CPUs at this time.
  3. IntelComputeRuntime (ICR) is used for -9xxx and above.

Do you concur?

Yes, I believe that is correct.

Thanks.

I’ve got the changes.

I’ve since chatted with the transcoder team about corrections made to the transcoder itself. When those changes are ready (pass QA), I will spin us a package for use here.

1 Like

The package is named ocl-icd-libopencl1 - but otherwise agreed.

@emb531

Yes… Typo on my part here.

Already have

    if [ $NeedOpenCL -gt 0 ] && [ $HaveOpenCL -eq 0 ]; then
      Warnings=$((Warnings + 1))
      Output ""
      Output "WARNING: The OpenCL library, required for hardware transcoding of HDR content, is missing."
      Output "         Please install package:  'ocl-icd-libopencl1'"
    fi

from the earlier Beignet-only support.

HDR tonemapping now works again on the most recent PlexPass release. Thanks very much, @ChuckPa!

Same here! Glad to be up to date now.

You all ready to give the updated packaging a test drive?

  1. Using the current 1.23.0.4438 binaries
  2. Updated packaging

Changes:

  1. Inventory independent of need
  2. CPU / Processor identification timing error corrected.
  3. Instructions for installing Intel Compute Runtime added.

https://drive.google.com/file/d/1ItlIAG-Ky-q-JU6aBzQf9Y_pHn1eEkbr/view?usp=sharing

If there are no issues with this, I’ll submit it to QA for testing and production.

Looks good!

(Reading database ... 174698 files and directories currently installed.)
Preparing to unpack ChuckPa_plexmediaserver_1.23.0.4438-5eef0fd80_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.23.0.4438-5eef0fd80) over (1.23.0.4438-5eef0fd80) ...
Setting up plexmediaserver (1.23.0.4438-5eef0fd80) ...
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         video
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /dev/shm  (set in Preferences.xml)
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Intel i915 Hardware: Found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:
PlexMediaServer install:   OpenCL:              Installed
PlexMediaServer install:   Intel Gmmlib:        Installed
PlexMediaServer install:   Intel IGC Core:      Installed
PlexMediaServer install:   Intel IGC OpenCL:    Installed
PlexMediaServer install:   Intel OpenCL:        Installed
PlexMediaServer install:   Intel OCLoc:         Installed
PlexMediaServer install:   Intel Zero GPU:      Installed
PlexMediaServer install:
PlexMediaServer install: INFO:    dpkg will automatically install beignet-opencl-icd and ocl-icd-libopencl1 automatically
PlexMediaServer install:          if available for your host upon successful installation of Plex Media Server.
PlexMediaServer install:
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation successful.  Errors: 0, Warnings: 0
Processing triggers for libc-bin (2.31-0ubuntu9.3) ...
Processing triggers for mime-support (3.64ubuntu1) ...

# Plex Media Server installation configuration info:  Tue 27 Apr 2021 12:07:27 PM EDT
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="video"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/dev/shm"
LangEncoding="en_US.UTF-8"
ExistingVersion=12300
HaveHardware=1
HaveTranscoderPref=1
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=0
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=i5-10600K
CPUSKU=10600
NeedOpenCL=1
NeedBeignet=0
NeedIntelCompute=1
HaveIntelGmmlib=1
HaveIntelIGCCore=1
HaveIntelIGCOpenCL=1
HaveIntelOpenCL=1
HaveIntelOCLoc=1
HaveIntelZeroGPU=1
HaveIntelCompute=1
HaveBeignet=0
HaveOpenCL=1
Platform="System Product Name"
Processor="i5-10600K"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux Saturn 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="31942 1929 5020"

Thank you for that. I see ONE tweak needed.

  • Printing the Beignet installation INFO even though it’s not needed.

I’d like to go with this layout if there are no objections ?
My intent is to both warn and notify that certain Debian installations will auto install – I won’t know what’s in the distro’s repo.

root@plexdev:~/git/plex-postprocessor/targets/debian/files/debian# ./*.postinst
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation starting.
PlexMediaServer install: 
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         video
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /tmp 
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Config file used:    /etc/systemd/system/plexmediaserver.service.d/override.conf
PlexMediaServer install:   Intel i915 Hardware: Found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:  
PlexMediaServer install:   OpenCL:              Not Installed
PlexMediaServer install:   Beignet:             Not Installed
PlexMediaServer install: 
PlexMediaServer install: WARNING: The OpenCL library, required for hardware transcoding of HDR content, is missing.
PlexMediaServer install:          Please install package:  'ocl-icd-libopencl1'
PlexMediaServer install:  
PlexMediaServer install: INFO:    dpkg will install package 'ocl-icd-libopencl1' automatically
PlexMediaServer install:          if available for your host upon successful installation of Plex Media Server.
PlexMediaServer install: 
PlexMediaServer install: WARNING: The Beignet OpenCL library, required for hardware transcoding of HDR content, is missing.
PlexMediaServer install:          Please install package:  'beignet-opencl-icd'
PlexMediaServer install:  
PlexMediaServer install: INFO:    dpkg will install package 'beignet-opencl-icd' automatically
PlexMediaServer install:          if available for your host upon successful installation of Plex Media Server.
PlexMediaServer install:  
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation successful.  Errors: 0, Warnings: 4
root@plexdev:~/git/plex-postprocessor/targets/debian/files/debian# 

4K HDR transcoding works for me without the standard OpenCL package. I only have the Intel Compute Runtime packages installed.

(Reading database ... 378652 files and directories currently installed.)
Preparing to unpack ChuckPa_plexmediaserver_1.23.0.4438-5eef0fd80_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.23.0.4438-5eef0fd80) over (1.23.0.4438-5eef0fd80) ...
Setting up plexmediaserver (1.23.0.4438-5eef0fd80) ...
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         render
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /tmp
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Intel i915 Hardware: Found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:
PlexMediaServer install:   OpenCL:              Not Installed
PlexMediaServer install:   Intel Gmmlib:        Installed
PlexMediaServer install:   Intel IGC Core:      Installed
PlexMediaServer install:   Intel IGC OpenCL:    Installed
PlexMediaServer install:   Intel OpenCL:        Installed
PlexMediaServer install:   Intel OCLoc:         Installed
PlexMediaServer install:   Intel Zero GPU:      Installed
PlexMediaServer install:
PlexMediaServer install: WARNING: The OpenCL library, required for hardware transcoding of HDR content, is missing.
PlexMediaServer install:          Please install package:  'ocl-icd-libopencl1'
PlexMediaServer install:
PlexMediaServer install: INFO:    dpkg will automatically install beignet-opencl-icd and ocl-icd-libopencl1 automatically
PlexMediaServer install:          if available for your host upon successful installation of Plex Media Server.
PlexMediaServer install:
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: PlexMediaServer-1.23.0.4438-5eef0fd80 - Installation successful.  Errors: 0, Warnings: 1
Processing triggers for libc-bin (2.31-0ubuntu9.3) ...
Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...
Processing triggers for desktop-file-utils (0.24-1ubuntu3) ...
Processing triggers for mime-support (3.64ubuntu1) ...
# Plex Media Server installation configuration info:  Tue 27 Apr 2021 03:42:09 PM CDT
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=0
OverrideFile=""
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12300
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=0
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=i7-10710U
CPUSKU=10710
NeedOpenCL=1
NeedBeignet=0
NeedIntelCompute=1
HaveIntelGmmlib=1
HaveIntelIGCCore=1
HaveIntelIGCOpenCL=1
HaveIntelOpenCL=1
HaveIntelOCLoc=1
HaveIntelZeroGPU=1
HaveIntelCompute=1
HaveBeignet=0
HaveOpenCL=0
Platform="NUC10i7FNH"
Processor="i7-10710U"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux nuc 5.11.16-051116-generic #202104211235 SMP Wed Apr 21 12:39:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="31820 1171 25361"

@riiviin

Thank you for reporting that.

What you have makes more sense to me. Intel has a version of OpenCL which is tailored to Compute Runtime. It makes sense to work with it over generic OpenCL.

If someone else can confirm this finding, I’ll change (drop) the requirement

@ChuckPa

Actually, you should leave it in. The dpkg flags for ocl-icd-libopencl1 were ri on my system, so I believe Plex was still using it while preinst considered it not installed. Hardware 4K HDR transcoding did still work once it was gone, but it was less efficient. When I reinstalled ocl-icd-libopencl1, it returned to previous performance levels.