Plex Hardware Transcoding on VMware ESXi 6.7/Ubuntu/Quadro P5000

Plex Media Server Desktops & Laptops
1

Hi all-

I’m following in the footsteps of many others who have shared their knowledge in getting this configuration up and running, and I’m still hitting a snag, I continue to see the following in my logs when I attempt transcoding:

Apr 30, 2021 03:20:21.465 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:21.465 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:21.936 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:21.937 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:22.418 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:22.418 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:22.891 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:22.891 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:23.339 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:23.339 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:23.769 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:23.769 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:24.223 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
Apr 30, 2021 03:20:24.223 [0x7ff009ffb700] ERROR - [Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Apr 30, 2021 03:20:24.453 [0x7ff0313fb700] WARN - [Transcode/136lwljqby4fik1yojmjygpe] Transcode runner appears to have died.
Apr 30, 2021 03:20:24.505 [0x7fef98ffd700] WARN - [Transcode/136lwljqby4fik1yojmjygpe] Transcode runner appears to have died.

What I have done so far:

  • Passed through the GPU to the VM successfully (nvidia-smi works fine)
  • Set svga.present to FALSE (I only see one device, /dev/dri/renderD128)
  • Added “plex” to both the “video” and “render” groups
  • Enabled hardware transcoding in Plex

I’m beginning to wonder if perhaps my CPUs are too old (Released in 2012) and don’t support Quick Sync Video, mostly because it’s not in this list. I have two Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz, but I was under the impression that Plex would use the NVIDIA GPU if the Intel GPU wasn’t present.

Any folks out there have any insight they can share here?

Thank you,
Darren

2

You are correct that your CPU is too old for Quick Sync Video.
Only select later Xeons have this capability.

I am using ESXi 7.0.1d with QSV. VGA is disabled (turns off HDMI). The i915 is now available. Passing the UHD630 through and locking VM memory into RAM (won’t swap) allows QSV to work.

I use a NUC8i7-HVK for this. Can’t speak to the Nvidia, sorry

3

Thanks, Chuck. Is that a certainty that this will never work, or should it still work with my NVIDIA GPU with the proper configuration?

4

your only means to obtain HW transcoding, using that CPU, is the Nvidia GPU card passthrough.

5

Thanks, Chuck, that’s what I was hoping to hear.

I guess the outstanding question now is “what’s configured incorrectly here?” I’m not sure exactly how to troubleshoot. vainfo tells me I need X installed - is that required here?

> vainfo
error: can't connect to X server!
error: failed to initialize display
6

@darrendavid

  1. You’re running in ESXi – there won’t be a display head
  2. No display head – No X server
  3. No X server – No vainfo

You need the Nvidia drivers installed.
PMS will use nvdec (decoder) and nvenc (encoder) – headless.

I am using Ubuntu server in mine. Only thing passed through is the UHD630
I don’t need i915 drivers because the kernel includes them for me.

Screenshot from 2021-04-30 20-48-32
Screenshot from 2021-04-30 20-48-32767×652 37 KB

The video card here is the one provided by ESXi for the console.

7

Good news! This appears to all be working now. I followed the instructions in this post to get nvenc and nvdec properly built against the CUDA libraries and now I’m getting hardware transcoding in my ESXi VM with GPU hardware passthru.

Darren

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.