VMWare ESXI 7.0 NVidia GPU Passthrough

[EikeW]

Plex Server
Gastbetriebssystem
Ubuntu Linux (64 Bit)
Kompatibilität
Virtuelle ESXi 7.0-Maschine
VMware Tools
Ja
CPUs
4
Arbeitsspeicher
32 GB
Player Version#: Latest

plex-4-k-streaming

Hi Guys,
as you can see on the picture i have a NAS and and esxi server running. in short words i want to passthrough the nvidia p400

Screenshot-2022-03-11-185531

the card is working well in the vm and it is shown up in the plex server:

lspci -k | grep -A 2 -i “VGA”
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
Subsystem: VMware SVGA II Adapter
Kernel driver in use: vmwgfx
Kernel modules: vmwgfx

0b:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1)
DeviceName: pciPassthru0
Subsystem: NVIDIA Corporation GP107GL [Quadro P400]

now comes the question. how can i tell plex to use this card ? if i start a movie and it will decode it always uses the vmware adapter card even if it say:

prime-select query
nvidia

do you have any ides what i can do ?

i can’t disable the “main” card. if i do so the vm will not boot.
i have NOT installed any driver on the linux system.

thanks guys

[ChuckPa]

examine the contents of /dev/dri.

You will most likely see the additional node /dev/dri/renderD129.

With PMS stopped, add preference HardwareDevicePath="/dev/dri/renderD129"

Save and start PMS.

[EikeW]

…sorry that i have to ask but where exacly do i have to add the preference?

thanks

[ChuckPa]

Add the preference to Preferences.xml in the Plex Media Server directory (/var/lib/plexmediaserver/Library/Application Support/Plex Media Server)

Be very careful to not damage others. PMS will erase it all and start over if it can’t recover from errors

[EikeW]

ddddddd

[EikeW]

it look like it has no efect still the same

[ChuckPa]

1. Add the device and confirm it’s added
   
   

2. Per ESXi rules, preallocate/reserve all the memory.
   

   

3. If your Display (the ESXi console) console is active (the VGA) - turn it off. ESXi will not allow a VM to access the GPU if it’s using it for console display purposes.
   ( You will only be able to administer the device via web console & SSH after this)

I have the i7-8809G CPU. The Radeon is the renderD128 node. QSV is renderD129.

Check your logs to see what is reported.

[EikeW]

aha i maybe i have to turn off the console…can you tell me where do i have to do it ?

[EikeW]

here ?

[ChuckPa]

Is that the host or the VM?

The ESXi host must have it disabled.

I use an external USB drive for my ESXI host. I have edited the svga.present value in that config.

I know you can turn it off from the ESXi HDMI console. ONCE

[EikeW]

Naahhh this is the vm Maschine itself i have to Turn Off the whole esxi Host? So i can,t Access any Maschine via the Webpage again?

I only Pass the Card to the Plex Server No one Else ist using it

[ChuckPa]

You will always be able to access the ESXi host with the web page.
This is how I use it.

When you turn off the VGA adapter in the ESXi host config, you will not have the HDMI console (the black and yellow screens you log into with a USB keyboard)

With the VGA disabled, the UHD 630, which is attached to it, can then be allocated to any VM you wish just as I do.

I have been looking for the command (ESXi command) to turn it off but haven’t found it yet via Google)

[ChuckPa]

To show you my configuration:

[chuck@lizum 5274-9249.2004]$ ls
b.b00         esx_ui.v00   jumpstrt.gz   lsuv2_nv.v00  nmlx4_en.v00          qcnic.v00     state.tgz        vmkfcoe.v00
bnxtnet.v00   esxupdt.v00  k.b00         lsuv2_oe.v00  nmlx4_rd.v00          qedentv.v00   s.v00            vmkusb_n.v00
bnxtroce.v00  features.gz  loadesx.v00   lsuv2_oe.v01  nmlx5_co.v00          qedrntv.v00   tpmesxup.v00     vmkusb.v00
boot.cfg      gc.v00       lpfc.v00      lsuv2_oe.v02  nmlx5_rd.v00          qfle3f.v00    tpm.v00          vmw_ahci.v00
brcmfcoe.v00  i40en.v00    lpnic.v00     lsuv2_sm.v00  ntg3.v00              qfle3i.v00    uc_amd.b00       vmware_e.v00
brcmnvme.v00  i40iwn.v00   lsi_mr3.v00   mtip32xx.v00  nvme_pci.v00          qfle3.v00     uc_hygon.b00     vmx.v00
btldr.v00     iavmd.v00    lsi_msgp.v00  native_m.v00  nvmerdma.v00          qflge.v00     uc_intel.b00     vsanheal.v00
crx.v00       icen.v00     lsi_msgp.v01  ne1000.v00    nvmxnet3.v00          qlnative.v00  useropts.gz      vsanmgmt.v00
elx_esx_.v00  igbn.v00     lsi_msgp.v02  nenic.v00     nvmxnet3.v01          rste.v00      useropts-VGA.gz  vsan.v00
elxiscsi.v00  imgdb.tgz    lsuv2_hp.v00  nfnic.v00     preupgrade-state.tgz  sb.v00        vdfs.v00         weaselin.v00
elxnet.v00    iser.v00     lsuv2_in.v00  nhpsa.v00     procfs.b00            sfvmk.v00     vim.v00          xorg.v00
esx_dvfi.v00  ixgben.v00   lsuv2_ls.v00  nmlx4_co.v00  pvscsi.v00            smartpqi.v00  vmkata.v00
[chuck@lizum 5274-9249.2005]$ zcat useropts.gz 
vga=FALSE
[chuck@lizum 5274-9249.2006]$ 

This disables the console VGA adapter & HDMI output from ESXi console.

[ChuckPa]

Supplemental:

I can’t remember the exact command but this looks right / at least close for doing it from a running ESXi on an Intel NUC.

Passthrough of Integrated GPU (iGPU) for standard Intel NUC

esxcli system settings kernel set -s vga -v FALSE

[EikeW]

Hi Chuck,
i made this in the Host but the Result is still the same :

[ChuckPa]

I don’t know what isn’t happening.

When I pass through the iGPU,

1. Edit the VM settings
2. at the popup window: “Add Other Device”
3. select the device (Nvidia or iGPU)
4. SAVE

Have you also considered:

Navigator → Host → Manage → Hardware → PCI devices →

1. Find the Nvidia
2. Check the box (Select) the Nvidia in the list.
3. Toggle (enable) passthrough

[EikeW]

everything looks good for me …i have no idea why it will not happen

[EikeW]

HardwareDevicePath="/dev/dri/renderD129"

this is still true or do i have to say HardwareDevicePath="/dev/dri/-renderD129"

[ChuckPa]

You must look in /dev/dri in the VM to see how it is passed through.

If the CPU already has QSV – The CPU will present renderD128 (first found).

You can always try each of the render names found in /dev/dri.

My CPU is odd. (8809G)

1. renderD128 = AMD
2. renderD129 = Intel

[EikeW]

you can see it in the screenshot ddddd up in the post