Loss of HW transcoding

Late to the party, but I’ve also recently discovered that my Plex instance stopped doing hardware transcoding. Here’s the outputs from running it here.

This is an Intel NUC NUC6i5SYH with i5-6260U running VMWare ESXi 6.7 and doing PCI passthrough on the iGPU to the VM. This VM configuration has definitely worked with hardware transcoding previously, so I suspect I’m also hitting this issue. I’ve included a bunch of outputs below, including the output of the v7 preinst script and the generated installer log.

Hope this is vaguely helpful and not just redundant information at this time :slight_smile:

root@jmvpms1:~/tmp# dpkg -l | grep plexmediaserver
ii  plexmediaserver                        1.22.0.4163-d8c4875dd             amd64        Plex organizes all of your personal media so you can easily access and enjoy it.
root@jmvpms1:~/tmp# cat /proc/cpuinfo | grep "model name"
model name	: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
model name	: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
model name	: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
model name	: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
root@jmvpms1:~/tmp# ls -l /dev/dri
total 0
drwxr-xr-x 2 root root       120 Mar 13 17:30 by-path
crw-rw---- 1 root video 226,   0 Mar 13 17:30 card0
crw-rw---- 1 root video 226,   1 Mar 13 17:30 card1
crw-rw---- 1 root video 226, 128 Mar 13 17:30 renderD128
crw-rw---- 1 root video 226, 129 Mar 13 17:30 renderD129
root@jmvpms1:~/tmp# id plex
uid=107(plex) gid=113(plex) groups=113(plex),44(video)
root@jmvpms1:~/tmp# ./preinst 
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: 
PlexMediaServer install:   DEVELOPMENT  V7 draft
PlexMediaServer install: Configuration information discovered:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         video
PlexMediaServer install:   Metadata Dir:        /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install:   Temp Directory:      /tmp
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Config file used:   /etc/systemd/system/plexmediaserver.service.d/override.conf
PlexMediaServer install:   Intel i915 Hardware: Found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:  
PlexMediaServer install:   OpenCL ICD:          Installed
PlexMediaServer install:   Beignet:             Installed
PlexMediaServer install:  
PlexMediaServer install: Pre-installation Validation complete.
root@jmvpms1:~/tmp# cat /tmp/plexinstaller.log 
# Plex Media Server installation configuration info:  Mon 15 Mar 09:47:43 WET 2021
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=1
OverrideFile="/etc/systemd/system/plexmediaserver.service.d/override.conf"
PlexUser="plex"
PlexGroup="plex"
VideoGroup="video"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12200
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
CPUSKU=6260
NeedBeignet=1
NeedIntelCompute=0
HaveIntelGmmlib=0
HaveIntelIGCCore=0
HaveIntelIGCOpenCL=0
HaveIntelOpenCL=0
HaveIntelOCLoc=0
HaveIntelZeroGPU=0
HaveIntelCompute=0
HaveBeignetIcd=1
HaveOpenCL=1
HaveBeignet=1
Platform="VMware Virtual Platform"
Processor="Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz"
Distro="Ubuntu 20.04.2 LTS"
Kernel="Linux jmvpms1 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"
Memory="16011 7766 3940"

@deepseth

You had replied to the wrong thread.

That thread was new development for CometLake CPUs (-10xxx SKU).
You have an i5-Skylake.

Your normal Plex logs (DEBUG ON, VERBOSE OFF), will be needed.

  1. verify logging
  2. Start playback
  3. Let play for 20 seconds
  4. Stop playback
  5. Let logs flush to disk (20 seconds)
  6. Download Logs ZIP file
  7. Attach ZIP file for review.

Whoops. Apologies for the mix up.

I’ve done a fresh install of Debian 10 and installed the latest version of plex from the repository

root@jmvpms2:/zpool/media# uname -a
Linux jmvpms2 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux
root@jmvpms2:/zpool/media# cat /etc/debian_version 
10.9
root@jmvpms2:/zpool/media# cat /etc/apt/sources.list.d/plexmediaserver.list
deb https://downloads.plex.tv/repo/deb/ public main
root@jmvpms2:/zpool/media# dpkg -l | grep plexmedia
ii  plexmediaserver                1.22.1.4228-724c56e62        amd64        Plex organizes all of your personal media so you can easily access and enjoy it.

I’ve removed the iGPU from the previous VM and attached it to this one instead and validated that it’s not a permissions issue:

root@jmvpms2:/zpool/media# lspci | grep VGA
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
13:00.0 VGA compatible controller: Intel Corporation Iris Graphics 540 (rev 0a)
root@jmvpms2:/zpool/media# ls -l /dev/dri
total 0
drwxr-xr-x 2 root root        120 Mar 30 14:11 by-path
crw-rw---- 1 root render 226,   0 Mar 30 14:13 card0
crw-rw---- 1 root render 226,   1 Mar 30 14:13 card1
crw-rw---- 1 root render 226, 128 Mar 30 14:13 renderD128
crw-rw---- 1 root render 226, 129 Mar 30 14:13 renderD129
root@jmvpms2:/zpool/media# id plex
uid=998(plex) gid=998(plex) groups=998(plex),44(video),107(render)

I’ve then tried to play WeAreGoingOnBullrun.mp4 a few times, which should be visible in the logs. The first few times, HW transcoding wasn’t actually enabled on the server. Prior to the final play, it was.

Logs attached.
Thanks!

Plex Media Server Logs_2021-03-30_14-30-46.zip (257.2 KB)

I’ve scanned the Plex Media Server.log myself, and I’m seeing the same errors in this fresh install as I see in my old Ubuntu based install:

Mar 30, 2021 14:28:33.152 [0x7f328affd700] DEBUG - [Transcode] Codecs: testing h264_vaapi (encoder)
Mar 30, 2021 14:28:33.152 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Mar 30, 2021 14:28:33.152 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Mar 30, 2021 14:28:33.152 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Mar 30, 2021 14:28:33.152 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Mar 30, 2021 14:28:33.152 [0x7f328affd700] DEBUG - [Transcode] Codecs: testing h264_nvenc (encoder)
Mar 30, 2021 14:28:33.152 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvenc
Mar 30, 2021 14:28:33.153 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Mar 30, 2021 14:28:33.153 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Mar 30, 2021 14:28:33.153 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred
Mar 30, 2021 14:28:33.153 [0x7f328affd700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice vaapi
Mar 30, 2021 14:28:33.154 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Mar 30, 2021 14:28:33.154 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Mar 30, 2021 14:28:33.154 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Mar 30, 2021 14:28:33.154 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Input/output error
Mar 30, 2021 14:28:33.154 [0x7f328affd700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice nvdec
Mar 30, 2021 14:28:33.154 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvdec
Mar 30, 2021 14:28:33.154 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Mar 30, 2021 14:28:33.154 [0x7f328affd700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Mar 30, 2021 14:28:33.154 [0x7f328affd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred

@deepseth

what is the hypervisor in use ?

VMWare ESXi 6.7.0 Update 3 (Build 17499825)

Done a bit more digging and it looks like it’s related to the in-built virtual SVGA adaptor being enabled as well. Disabling that so that only a single video card is passed through to the system now results in hw transcoding working. I’m unsure if I ever disabled that before or not when I had it working before.

Unfortunately it also kills the virtual console for the VM, and results in output only going via HDMI, which is not ideal given that it’s a headless remote host, but not a deal breaker as I have a decent backup strategy, so could always just rebuild the VM if it failed to boot for any reason.

@deepseth

Glad you found it.

Not allowing ESXi to use the internal GPU does allow it to pass through to the VM.
(I have that same issue here with that same loss of HDMI. Not sure how I want to proceed with that)

Apologies for not being clear, as I think you may have misunderstood.

Enabling iGPU passthrough results in ESXi no longer outputting to local screen by HDMI. This is expected and fine.

When creating a VM, a virtual VMWare SVGA adaptor is added to each VM by default. When connecting to the virtual console of the VM via the ESXi web interface, this is the graphics adaptor that’s used to display the virtual screen.

If that built in vitual SVGA adaptor is disabled, the VM no longer outputs anything to the console, and instead now only outputs to HDMI. This isn’t helpful, as there’s nothing actually attached to the HDMI port, and I’ve now not got the ability to access the console of the VM, but it’s the only way I’ve so far found to make HW transcoding work.

It’s not a huge problem, but this still feels like a (small, at least) bug where Plex is unable to correctly identify the correct /dev/dri/renderD* device to use for HW transcoding when more than one exists.

Ironically, if I was using containers (inside my VM) this would probably not be a problem.

I’m sorry, I probably agreed with you by saying it backwards.

As I understand it, ESXi 7.0 update 1d :

I must disable the SVGA output to allow passing through the iGPU to the guest.

I understand why it has to be this way. The i915 is tied to the i965 / iHD which are used for hardware transcoding.

When we enable the HDMI output, the i915 is allocated to the HDMI.

If you check /sys/module/i915, you’ll find it doesn’t resolve to (available) hardware.

The kernel sees enough to populate /dev/dri but that’s not enough for hardware transcoding. I used to check for /dev/dri/render* as validation of HW transcoding but I found that wasn’t sufficient. That’s why I switched to detecting using the i915 method (what the transcoder also uses)

I’ve since reinstalled ESXi on USB. My goal is, once this new Hades Canyon is stable, is to disable the HDMI in the USB’s config. That will make HW transcoding possible. It will also make for easy recovery to use the HDMI when I need to. My next step is to learn how to properly disable HDMI in 7.0 U1

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.