HW Transcoding with HDR Tone mapping not working (hw works without tone mapping enabled)

[andreab809]

Server Version#: 1.32.6.7468-07e0d4a7e I did tried to progressively downgrade untill to 1.32.1.6999-91e1e2e2c, still same problem, not tried older versions
Player Version#: chrome web

I’m experiencing exactly this bug, HW transcoding is not working if tone mapping is enabled, it works if tonemapping is disabled

Media info:

> * Codec HEVC
> * Bitrate 20794 kbps
> * Lingua Inglese
> * Language Tag en
> * Bit Depth 10
> * Chroma Location topleft
> * Chroma Subsampling 4:2:0
> * Coded Height 2160
> * Coded Width 3840
> * Color Primaries bt2020
> * Color Range tv
> * Color Space bt2020nc
> * Color Trc smpte2084
> * Frame Rate 23.976 fps
> * Height 2160
> * Livello 5.1
> * Profilo main 10
> * Ref Frames 1
> * Width 3840
> * Display Title 4K HDR10 (HEVC Main 10)
> * Extended Display Title (4K HDR10 HEVC Main 10)
> 
> * Codec TRUEHD
> * Canali 8
> * Bitrate 3925 kbps
> * Lingua Italiano
> * Language Tag it
> * Audio Channel Layout 7.1
> * Bit Depth 24
> * Sampling Rate 48000 Hz
> * Titolo TrueHD Atmos 7.1
> * Display Title Italiano (TRUEHD 7.1)
> * Extended Display Title TrueHD Atmos 7.1 (Italiano)
> 
> * Codec AC3
> * Canali 6
> * Bitrate 640 kbps
> * Lingua Italiano
> * Language Tag it
> * Audio Channel Layout 5.1(side)
> * Sampling Rate 48000 Hz
> * Titolo AC3 5.1
> * Display Title Italiano (AC3 5.1)
> * Extended Display Title AC3 5.1 (Italiano)
> 
> * Codec TRUEHD
> * Canali 8
> * Bitrate 3579 kbps
> * Lingua Inglese
> * Language Tag en
> * Audio Channel Layout 7.1
> * Bit Depth 24
> * Sampling Rate 48000 Hz
> * Titolo TrueHD Atmos 7.1
> * Display Title Inglese (TRUEHD 7.1)
> * Extended Display Title TrueHD Atmos 7.1 (Inglese)
> 
> * Codec AC3
> * Canali 6
> * Bitrate 640 kbps
> * Lingua Inglese
> * Language Tag en
> * Audio Channel Layout 5.1(side)
> * Sampling Rate 48000 Hz
> * Titolo AC3 5.1
> * Display Title Inglese (AC3 5.1)
> * Extended Display Title AC3 5.1 (Inglese)
> 
> * Imposti true
> * Codec SRT
> * Lingua Italiano
> * Language Tag it
> * Titolo Forced
> * Display Title Italiano Forzato (SRT)
> * Extended Display Title Forced (Italiano SRT)
> 
> * Codec SRT
> * Lingua Italiano
> * Language Tag it
> * Titolo SDH
> * Display Title Italiano (SRT)
> * Extended Display Title SDH (Italiano SRT)
> 
> * Codec SRT
> * Lingua Inglese
> * Language Tag en
> * Titolo SDH
> * Display Title Inglese (SRT)
> * Extended Display Title SDH (Inglese SRT)

GPU info:

> root@server:~# lspci -v -s $(lspci | grep VGA | cut -d" " -f 1)
> 00:02.0 VGA compatible controller: Intel Corporation GeminiLake [UHD Graphics 605] (rev 06) (prog-if 00 [VGA controller])
> 	Subsystem: IP3 Tech (HK) Limited GeminiLake [UHD Graphics 605]
> 	Flags: bus master, fast devsel, latency 0, IRQ 130, IOMMU group 0
> 	Memory at c0000000 (64-bit, non-prefetchable) [size=16M]
> 	Memory at b0000000 (64-bit, prefetchable) [size=256M]
> 	I/O ports at f000 [size=64]
> 	Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
> 	Capabilities: [40] Vendor Specific Information: Len=0c <?>
> 	Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
> 	Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable- 64bit-
> 	Capabilities: [d0] Power Management version 2
> 	Capabilities: [100] Process Address Space ID (PASID)
> 	Capabilities: [200] Address Translation Service (ATS)
> 	Capabilities: [300] Page Request Interface (PRI)
> 	Kernel driver in use: i915
> 	Kernel modules: i915

> root@server:~# vainfo 
> error: XDG_RUNTIME_DIR is invalid or not set in the environment.
> error: can't connect to X server!
> libva info: VA-API version 1.17.0
> libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
> libva info: Found init function __vaDriverInit_1_17
> libva info: va_openDriver() returns 0
> vainfo: VA-API version: 1.17 (libva 2.12.0)
> vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 23.1.1 ()
> vainfo: Supported profile and entrypoints
>       VAProfileNone                   :	VAEntrypointVideoProc
>       VAProfileNone                   :	VAEntrypointStats
>       VAProfileMPEG2Simple            :	VAEntrypointVLD
>       VAProfileMPEG2Main              :	VAEntrypointVLD
>       VAProfileH264Main               :	VAEntrypointVLD
>       VAProfileH264Main               :	VAEntrypointEncSlice
>       VAProfileH264Main               :	VAEntrypointFEI
>       VAProfileH264Main               :	VAEntrypointEncSliceLP
>       VAProfileH264High               :	VAEntrypointVLD
>       VAProfileH264High               :	VAEntrypointEncSlice
>       VAProfileH264High               :	VAEntrypointFEI
>       VAProfileH264High               :	VAEntrypointEncSliceLP
>       VAProfileVC1Simple              :	VAEntrypointVLD
>       VAProfileVC1Main                :	VAEntrypointVLD
>       VAProfileVC1Advanced            :	VAEntrypointVLD
>       VAProfileJPEGBaseline           :	VAEntrypointVLD
>       VAProfileJPEGBaseline           :	VAEntrypointEncPicture
>       VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
>       VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
>       VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
>       VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
>       VAProfileVP8Version0_3          :	VAEntrypointVLD
>       VAProfileVP8Version0_3          :	VAEntrypointEncSlice
>       VAProfileHEVCMain               :	VAEntrypointVLD
>       VAProfileHEVCMain               :	VAEntrypointEncSlice
>       VAProfileHEVCMain               :	VAEntrypointFEI
>       VAProfileHEVCMain10             :	VAEntrypointVLD
>       VAProfileHEVCMain10             :	VAEntrypointEncSlice
>       VAProfileVP9Profile0            :	VAEntrypointVLD
>       VAProfileVP9Profile2            :	VAEntrypointVLD

kernel:

> root@server:~# uname -ra
> Linux server 6.1.0-11-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-4 (2023-08-08) x86_64 GNU/Linux

[FordGuy61]

@andreab809

GeminiLake

Hardware accelerated transcoding is broken on Gemini Lake CPUs.

Workaround is to fall back to 1.31.3.6868. 1.32.1.6999 might also work.

Synology - DSM 7.0 Release - Plex Installation (Not Operational) Issues

Sticking a pin in it – As of build 1.32.6.7371-b6a09ad81., Gemini Lake CPU HW transcoding is still broken. We’re still recouping from the staffing changes.

HDR Tone Mapping with HW Transcoding not working for 12900K CPU on PMS Version 1.32.5.7210

Let me recap recent history for you: 1.31.3.6868 - worked for (I think?) everyone. 1.32.0.xxxx - Intel Media Driver upgraded to current release – Major breakage across all the platforms 1.32.5.7210 - Fixed ApolloLake processors , Added AMD GPU support (SDR) 1.32.5.7349 - Hotfix for something else in PMS – works on our RaptorLake – work in flight for the next PMS release – – Staffing changes – Now we’re at the point of figuring out, by looking at the code changes, what was being done…

[andreab809]

In my case even older version like 1.31.3.6868 and 1.32.1.6999 doesn’t work untill I disable HDR tone mapping.

[90152]

HW transcoding not work on 1.32.6.7468 with HDR tone mapping turned on.
When I downgrade the PSM (1.32.6.7468 to 1.32.5.7349), the HW transcoding works fine even with HDR tone mapping turned on.

CPU:intel i5-13500
Unraid:6.12.4
Linux kernel:6.1.49

[ChuckPa]

@90152

Need check that again / See your DEBUG logs.

May I see the DEBUG logs which capture this?

What I am finding is

HDR on = SW transcode
HDR off = HW transcode + tone map

[SkynetMainframe]

in my case none of the old versions i’ve tried so far work, is hw transcoding with hdr tonemapping supposed to work with an 11900K?

[ChuckPa]

HDR tonemapping does work; but I’m using 1.31.3.6868 on Unraid.

The problem we’re seeing, which we’re trying to track down…

1. Fix subtitles
2. HDR breaks

root@BabyNUC:~# uname -a
Linux BabyNUC 6.1.49-Unraid #1 SMP PREEMPT_DYNAMIC Wed Aug 30 09:42:35 PDT 2023 x86_64 Intel(R) Core(TM) i3-7100U CPU @ 2.40GHz GenuineIntel GNU/Linux
root@BabyNUC:~# 

Have you deleted the codecs (plex stopped) and let them download again?

[SkynetMainframe]

@ChuckPa

1.31.3.6868

i just tried, and i’m using 1.31.3.6868 on Ubuntu 22.04.3

edit: i just tried on my debian server and it works with 1.31.3.6868 so something with Ubuntu maybe? or just my install?

edit2: definitely something wrong with my install on Ubuntu, i renamed plexmediaserver directory to .old and installed fresh 1.31.3.6868 and hw acceleration works. sooo what do I do? I really don’t want to have to rebuild my server.

[ChuckPa]

Stop Plex.

Go into /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Codecs

Delete everything you see except the .device_id file (your license key).

Start Plex.

It will download fresh codecs again with each first-playback of that codec (codecs-on-demand)

[SkynetMainframe]

i did that, unfortunately i mistakenly corrupted my database somehow now while trying to do a overwrite of the fresh install thinking that would fix my issues…it did not.

[ChuckPa]

The databases are in Plex Media Server/Plug-in Support/Databases and not related.

You had an errant rm command ?

If you’ve ‘blasted’ it, you have no real choice but to start over.

Overwriting things at random is dangerous.

[SkynetMainframe]

i had renamed plexmediaserver to plexmediaserver.old and installed a fresh plex to see if i could get hw transcoding working, i did. so i then tried to copy plexmediaserver.old back over to just plexmediaserver but forgot to delete the original folder first, and now? my database is corrupted.

Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - SQLITE3:0x80000001, 11, database corruption at line 68825 of [a29f994989]
Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - SQLITE3:0x80000001, 11, database corruption at line 68975 of [a29f994989]
Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - SQLITE3:0x80000001, 11, statement aborts at 10: [] database disk image is malformed
Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - SQLITE3:0x80000001, 11, database disk image is malformed in "PRAGMA cache_size=2048"
Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - Database corruption: sqlite3_statement_backend::prepare: database disk image is malformed for SQL: PRAGMA cache_size=2048
Sep 06, 2023 03:17:50.106 [0x7fa379ed3aa8] ERROR - Error: Unable to set up server: sqlite3_statement_backend::prepare: database disk image is malformed for SQL: PRAGMA cache_size=2048 (N4soci10soci_errorE)

[ChuckPa]

Copying on top of is not fatal – UNLESS PMS is running.

Given the record numbers, you can try my tool to recover from a DB Backup (which is colocated) but there are no guarantees at this point.

GitHub - ChuckPa/PlexDBRepair: Database repair utility for Plex Media Server...

The tool will allow you to pick from the list of available valid database files.

PMS must be stopped (it supports stopping from inside the tool) .

RUN AS ROOT user

[SkynetMainframe]

thank you, i shall give it a go.
edit: thank you for the attempt, it did fail with every command. I guess it was time to start over anyway it’s only been 11 years of watch history

[ChuckPa]

@SkynetMainframe

If you have cloud sync enabled, plex.tv will have your watch history.
It will all come back as soon as you rebuild.

[damse]

I have the same issue with the same CPU - intel i5-13500 on Unraid.
Is the only workaround downgrading the PMS version?

[ChuckPa]

as I posted elsewhere, the biggest impacted systems seems to be Unraid.
(either most impacted — or most vocal ) lol

I upgraded my system to the 6.1.49 kernel.

So far, I have found:

1. 1080p HW is unimpacted
2. 2160p HW HDR → SDR regressed.

Is this what everyone else is finding?

I am asking because:
The common impact I’ve seen with others is:
– /etc/modprobe.d/i915.conf
– options i915 enable_guc=2

I don’t know how to change & test this on Unraid

Strange part – With HDR tone mapping turned off, I still get tonemapping functionality

Ref:

• Offloading some media decoding functionality from the CPU to the HEVC/H.265 micro (µ) Controller (HuC). Only applicable if using intel-media-driver for hardware video acceleration. Introduced with Gen9.
• Using the GuC for scheduling, context submission, and power management. Introduced with Alder Lake-P (Mobile), within Gen12.

To use this functionality, the GuC firmware must be loaded. With regards to HuC support, some video features (e.g. CBR rate control on SKL low-power encoding mode) require loading the HuC firmware as well [6]. The GuC and HuC firmware files are both provided by linux-firmware.

GuC functionality is controlled by the i915.enable_guc kernel parameter. Its usage is as follows:

By default, the Intel Media Driver is used on Intel CPUs.
For all CPUs, CometLake and higher, Intel Media Driver is required.

[pshanew]

@ChuckPa

So far, I have found:

1. 1080p HW is unimpacted
2. 2160p HW HDR → SDR regressed.

Rather than repeat what I posted, I’ll link it here.

Intel N100 HW HDR Tonemapping not working

In my case, hardware accelerated transcoding hardware/format detection succeeds. The final analysis finds VAAPI available for both decoding and encoding: Sep 08, 2023 16:13:53.778 [140133279042360] INFO - [Req#d6/Transcode] Preparing driver imd for GPU JasperLake [UHD Graphics] Sep 08, 2023 16:13:53.778 [140133279042360] DEBUG - [Req#d6/Transcode/DriverDL/imd] Skipping download; already exists Sep 08, 2023 16:13:53.778 [140133279042360] DEBUG - [Req#d6/Transcode] TPU: hardware transcoding: fin…

For completeness’ sake, I reiterate here. This is on an Intel Celeron N5015 (Jasper Lake-based CPU); I’m on a kernel and distribution which are almost certainly unsupported (6.4.6/Pop!_OS 22.04 LTS), but performed hardware-accelerated transcoding with HDR tone mapping enabled on the previous stable server release.

Without HDR tone mapping enabled, HW accelerated transcoding works as expected. With HDR tone mapping enabled, transcoding falls back to software. Needless to say, on this CPU, this does not go well .

There is no virtual machine, container, or other abstraction involved here.

GuC/HuC are loaded successfully, if that’s in question:

sudo dmesg | grep -i i915
[   16.412738] i915 0000:00:02.0: vgaarb: deactivate vga console
[   16.412812] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[   16.413972] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[   16.414711] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/icl_dmc_ver1_09.bin (v1.9)
[   17.268613] i915 0000:00:02.0: [drm] [ENCODER:238:DDI A/PHY A] failed to retrieve link info, disabling eDP
[   17.289020] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/ehl_guc_70.1.1.bin version 70.1.1
[   17.289030] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/ehl_huc_9.0.0.bin version 9.0.0
[   17.306395] i915 0000:00:02.0: [drm] GT0: HuC: authenticated!
[   17.306822] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[   17.306825] i915 0000:00:02.0: [drm] GT0: GUC: SLPC disabled
[   17.330199] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
[   17.353267] fbcon: i915drmfb (fb0) is primary device
[   17.353274] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
[   21.562394] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[   21.656869] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-■■■2-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_ops [i915])

[ChuckPa]

@pshanew

I’m on a kernel and distribution which are almost certainly unsupported (6.4.6/Pop!_OS 22.04 LTS)

got that right

6.4 kernel was release on June 25th. Don’t know how “fresh” 6.4.6 is but with updates that quick, I’d not put it in production just yet

@pshanew

Without HDR tone mapping enabled, HW accelerated transcoding works as expected. With HDR tone mapping enabled, transcoding falls back to software. Needless to say, on this CPU, this does not go well .

I think this is what we’re all seeing?

1. HDR ON = NO HW enabled
2. HDR OFF = HW transcode + HDR tonemap

Here’s the $100 question:

Are there subtitles in the file ?

Asking because:

1. Previous version had subtitle problem (premature end of playback with subtitles)
2. Bug fixed
3. Did this fix break something else ?

[pshanew]

@ChuckPa

got that right

Lol, no worries there.

@ChuckPa

I’d not put it in production just yet

Kernel releases aren’t what they used to be. Minor releases (in the major.minor.patch scheme) represent small, incremental changes these days. As they should do. Heck, even the bump from 5.19 to 6.0 was just because Linus wanted to. It’s almost as though he’s capricious or something.

@ChuckPa

I think this is what we’re all seeing?

1. HDR ON = NO HW enabled
2. HDR OFF = HW transcode + HDR tonemap

For 2 (HDR off), I can only confirm that the hardware accelerated transcode is functioning. If there is HDR tone mapping occurring in this case, the client (Safari browser on macOS) is performing it.

@ChuckPa

Are there subtitles in the file ?

No, no subtitles at all. This is the Life Untouched 4K Demo. I’ve forgotten where I got it at this point (LG? Sony?). If you have a particular sample you’d like me to test, let me know. I have the “LG Colors of Journey HDR UHD 4K” demo as well, if that would help.

All indications are right now that it’s having issues enumerating OpenCL platform availability.