HDR Tonemapping issue on Proxmox 8.1 LXC with Intel Alder Lake vGPU

[Traveler885]

Server Version#: 1.32.7.7621-871adbd44
Player Version#: iOS 8.28 (cellular)

I’m following up on a post earlier this year that went unresolved. I have two Proxmox 8.1 hosts with an Intel i5-1240P and i3-N305. I have a Plex LXC, and the Proxmox host is configured with a vGPU using the DKMS kernel extension. Linux kernel 6.5.11-4-pve. A vGPU VF (virtual function) is assigned to the Plex LXC. Hardware transcoding works just fine, but enabling HDR tonemapping results in a corrupted video stream.

In the Plex LXC I see:

root@plex:~# groups plex
plex : plex video ssl-cert

Nov 25, 2023 19:22:39.326 [140460131576632] INFO - Preemptively preparing driver imd for GPU Alder Lake-N [UHD Graphics]
Nov 25, 2023 19:22:39.327 [140460131576632] INFO - Preemptively preparing driver icr for GPU Alder Lake-N [UHD Graphics]
Nov 25, 2023 19:22:43.156 [140460055829304] DEBUG - [Req#30/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.177 [140460055829304] DEBUG - [Req#30/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.183 [140460055829304] DEBUG - [Req#30/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.188 [140460055829304] DEBUG - [Req#30/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.194 [140460055829304] DEBUG - [Req#30/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.346 [140460055829304] DEBUG - [Req#42/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.354 [140460055829304] DEBUG - [Req#42/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.359 [140460055829304] DEBUG - [Req#42/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:43.365 [140460055829304] DEBUG - [Req#42/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:44.404 [140460106414904] INFO - [Req#59/Transcode/5B43855B-979C-4722-A7C8-460DAD6E2F1D] Preparing driver icr for GPU Alder Lake-N [UHD Graphics]
Nov 25, 2023 19:22:44.405 [140460106414904] DEBUG - [Req#59/Transcode/5B43855B-979C-4722-A7C8-460DAD6E2F1D] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (Alder Lake-N [UHD Graphics])
Nov 25, 2023 19:22:44.414 [140460106414904] INFO - [Req#59/Transcode/5B43855B-979C-4722-A7C8-460DAD6E2F1D] Preparing driver imd for GPU Alder Lake-N [UHD Graphics]

root@proxmox3:~# lspci -v -s 00:02.0
00:02.0 VGA compatible controller: Intel Corporation Alder Lake-N [UHD Graphics] (prog-if 00 [VGA controller])
        DeviceName: Onboard - Video
        Subsystem: Intel Corporation Alder Lake-N [UHD Graphics]
        Flags: bus master, fast devsel, latency 0, IRQ 157, IOMMU group 0
        Memory at 6000000000 (64-bit, non-prefetchable) [size=16M]
        Memory at 4000000000 (64-bit, prefetchable) [size=256M]
        I/O ports at 3000 [size=64]
        Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit-
        Capabilities: [d0] Power Management version 2
        Capabilities: [100] Process Address Space ID (PASID)
        Capabilities: [200] Address Translation Service (ATS)
        Capabilities: [300] Page Request Interface (PRI)
        Capabilities: [320] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: i915
        Kernel modules: i915

Any ideas on where to go from here to troubleshoot? From the locked thread I learned the HDR tonemapping uses the GPU, whereas the transcoding uses different APIs (Intel Xe graphics engine).

[ChuckPa]

@Traveler885

and the Proxmox host is configured with a vGPU using the DKMS kernel extension.

Capabilities: [320] Single Root I/O Virtualization (SR-IOV)

Don’t use the vGPU. Use the real instance. Those extensions screw up the Intel Media Driver every time.

Plex uses the real GPU through the Intel Media Driver. The IMD knows how to navigate and use the Xe graphics.

Pass the real /dev/dri (the PCIE device).

[Traveler885]

Well if one wants to share the GPU with another VM, like Windows, that’s impossible without vGPU.

[ChuckPa]

The best I find about this (which I’ve been following) is this:

vGPU (SR-IOV) with Intel 12th Gen iGPU [Updated Nov 2023]

The documentation states it’s experimental.

The problem is combination of i915 and kernel 6.2 requirement.

It also tells you you’ll still end up using renderD128/card0

Have you used this procedure to setup the SR-IOV and then, in Plex, told it which device (card0/renderD128) to use ? (Transcoder settings - Show Advanced)

[ChuckPa]

Just also found this.

Even if it does, say, when Proxmox get Linux Kernel 6.2, Intel has confirmed that Alchemist WILL NOT SUPPORT SRIOV.

Now is likely a good time to review the plan and retool as needed

Multiple OS/Environments on one “appliance” can actually overload the hardware.

[Traveler885]

Proxmox has supported Linux Kernel 6.2 for almost six months. Proxmox 8.1 came out last week and ships with Kernel 6.5, so that’s not a problem. I’ll check more into the Plex settings.

[Traveler885]

Intel fully supports SR-IOV on 12th generation CPUs, which is what I’m using. vGPU with DKMS is using SR-IOV for the VFs. Yes, Plex advanced detects the Alder-Lake GPU. The card0/renderD128 are the LXC devices that Proxmox maps to the specified VF in the LXC config file.

The Github DKMS repo states: Linux i915 driver with SR-IOV support (dkms module) for linux 6.1 ~ linux 6.5. GitHub - strongtz/i915-sriov-dkms: dkms module of Linux i915 driver with SR-IOV support

[ChuckPa]

Not to be a jerk but I have to pass this up to Engineering because:

1. Proxmox – not official support
2. SR-IOV by WHOSE github ? A third party?
3. We don’t support Gentoo at all.

I don’t see Intel official on this anywhere. Sorry.

While it should be / probably is supported in the CPU, I need more definitive info / guidance.

If you could point me to official Intel release info / instructions, that would make it easier for me.

At this point, I need discuss with them and then ask to see about planning to support / if they are already looking at it.

If PMS does SR-IOV then it has to work on ALL the platforms and not just Proxmox (which isn’t officially supported). We can’t go forcibly installing kernel modules on customer’s computers without their consent at the kernel level. That’s not cool.

I’m not being a stick in the mud but unless they have figured this out already ---- it’s going to be a tall ask.

[ChuckPa]

@Traveler885 and all who find this:

I’ve talked with the transcoder engineer at length about this.

We’ve come to this:

1. We agree it’s not for Plex to be mucking in anyone’s kernel configs.
   We won’t setup SR-IOV capabilities as PMS installs.
   This will be the user’s responsibility

2. We will look to support SR-IOV if it’s available on the host.
   This requires we change how PMS detects and evaluates potentially viable GPUs for transcoding use.

3. I’ve completed writing the request for engineering.
   We will triage & schedule the work with the next triage team meeting.

[Traveler885]

@ChuckPa Thank you! 100% agree Plex shouldn’t muck with kernel configs and that it’s up to the user to do any needed SR-IOV setup.

SUPER happy to hear Plex will look into supporting SR-IOV. Awesome news!