Quadro P2000 - PCI-E Passthrough in vmware,

Hi ,

I was hoping someone here would have and share their experience using Plex running on a Virtual machine with a Quadro P2000 dedicated to a single VM via PCI-E passthrough.

My setup:

A physical server running VMware ESXi, 6.7.0:

HPe Proliant ML350 Gen9
2x Intel® Xeon® CPU E5-2620 v4 @ 2.10GHz
160 GB RAM
2x 100GB SSD’s RAID 0
2x 1.6TB SSD’s RAID 0

It’s running a few VM’s primarily test servers with a low load but using most of the local SSD storage.
For media files i have a QNAP TS-459 PRO II NAS, with 2x 8TB Ironwolf HDD’s in RAID 0 for a total of 16TB.

The problem is that we have a mix of 4K and 1080P TV’s and devices and PC’s, and using transcoding quite a bit, which is now becoming a problem with those 4K HEVC x265 media files since the CPU’s are Broadwell based and therefore dont support intel Quicksync. So I was thinking about adding a Nvidia Quadro P2000 card to offload my CPU’s that dont seem to be able to handle even one 4K transcoded stream.

The Plex server, is a virtual machine running Windows Server 2019 and runs off local SSD storage, and the VM is configurable to whatever amount of CPU cores and RAM amount needed. The media shares are running on that QNAP NAS with gigabit connection, to the server, (i might team the 2 NICs on it to allow two high bandwith streams to run simultaneously, if those two ironwolf’s can even handle that)
The plan is to add a Quadro P2000 card or maybe test first with a cheap GTX 1050 (only 2 streams though) and pass it through to the Virtual machine.

Does anyone here have experience with something similar to what i’m trying to achieve here in a virtutual environment, I would realy appreciate som insight into this.

thanks - Kasper

That’s quite a beefy system, and while I can’t give any insight into numbers, I can confirm that VMWare handles PCIe passthrough quite well. Or more correctly, Intel CPUs do. The features you need for that to work are VT-x and VT-d, which is supported a few generations before those CPUs and will run fine assuming you enable them in BIOS/UEFI.

One thing to note is broadwell’s DO support quicksync (but not h.265), and goes as far back as sandy bridge, but it’s for consumer-grade CPUs, and isn’t supported outside of entry-level, single socket Xeons, even in the newest chips. It’s the price we pay for running enterprise hardware with plex.

As far as 4k streams, I’ve yet to see anything above about 85Mbps avg, although things will burst up into the 100Mbps range, 2-3 streams are easily handled even on a decent HDD with gigabit. I can’t imagine a pair of ironwolfs would struggle until you get into pretty high stream counts with bonded gigabit.

This guy did some benchmarks of a P2000 with much older CPUs (and no quicksync since they are still Xeons), and managed 13 concurrent streams for 4k h265 down to 1080p h264: https://www.youtube.com/watch?v=FmlrfQZ2ExA&feature=youtu.be&t=444 so I can’t imagine you will throw much at it that would overwhelm it.

Hope that helps, best of luck.

Thanks a lot jcampbell7 :slight_smile:

I’ve had good experiences with passthrough of PCI-E devices, just havent tried with GPU’s yet.
I would expect to have some sort of issue with displaydrivers /vmware tools / no actual screen connected etc etc.
The P2000 might be a bit overkill, maybe a P400 would be sufficient, any thoughts on that? Two simultaneous streams would probably be sufficient, or maybe adding 2 P400’s which would still be 2/3’s of the price of the P2000.
I’m also considering putting those Ironwolf’s in a RAID USB enclosure Link instead of in the NAS to maximize bandwith.

completely aside from your p2k and vmware questions;

if you keep your 4k content in a separate library, then you can avoid transcoding by not playing 4k content on non-4k devices.

also, even if you get transcoding working through the VM its not going to do the color mapping conversion from HDR to SDR so that colors will look messed up on SDR devices.

I have a p400 in an old dell r610 and it runs 30-60% utilization, so I expect it could run 2, maybe 3, 4k streams.

note bare metal debian not virtualized

image

Yeah they look pretty messed up on SDR devices, so it’s definetly a good idea not to transcode them in most cases, and I will be seperating them in a different library. But it’s not even possible for me to remote play 4K from my server with my 70mbit upload right now so I would like to transcode them a little bit if possible.

transcoding the video of any kind will remove the HDR, and it will convert to 264 (plex does not encode to 265) so you will still be losing a ton.

if you cannot direct play 4k, you should not bother with it, at all.

Nice!! I think I’ll give that a shot then…they are pretty inexpensive in comparison :slight_smile:

do note, the p400 is limited to 2 enc streams, the rest will fail over to the cpu as needed.

Woot! I thought these issues was due to the x265 codec being so hard on the system.
Shouldnt my CPU’s be able to handle x265 with their brute force and core counts? I can run 10+ 1080p 264 transcoded streams to varoius devices at different resolutions, but not a single 4K x265 to 1080p x264 transcode. Super confusing stuff this, sorry :slight_smile:

265 is extremely hard, and those cpu while good in there day are still 2 generations behind the cpus that are still having problems decoding 265.

264 1080 is like a grain of sand on the beach of 265 4k

Ahh, ok would a GTX 1050 be the better choice of the two then? Better performance and the same two stream limit ?

Thanks a lot for the valuable info TeknoJunky, i really appreciate it:)

no check the link, you can see that only the p2000 quadro and up have the unlimited encode sessions

really, encoding is not the problem (because it is 264 1080 or 720)

its the DECODING of 4k x265 that is the hardest part

for DECODING on my desktop I use a gtx 960 (gm206) which supports 4k 10bit DEcoding

yeah seems so :astonished:
Would a 9600K - up to 9900K be able to transcode multiple streams of x265 , or only 1 at the time…I mean the on die GPU’s are pretty weak.

the intel 600 series igpu are not weak for transcoding, and they are recommended and have the best plex HA transcoding support.

ie any intel gpu with HD630 or similar gpu.

as for the CPU only transcoding, I couldn’t say, but compared to your xeons it would have double the cpumark

https://www.cpubenchmark.net/compare/Intel-Xeon-E5-2620-v4-vs-Intel-i9-9900K/2766vs3334

But if the choice is between a P400 and a GTX 1050.
The 1050 could be used for light gaming in another system, whereas the P400 is pretty much useless for anything but workloads, if i end up not using it.
Is there any benefit of the P400 over a 1050 ?

You have me doubting that i even want to add a GPU for transcoding, but I am curious :blush:

well they are completely different cards too.

the p400 is a small single slot half height card so fits in much smaller systems and doesn’t require a separate power cable.

the 10 series are all dual slots that I found

and well compared @ https://www.videocardbenchmark.net/compare/Quadro-P400-vs-GeForce-GTX-1050-Ti/3807vs3595

says the 1050 is over 3x the gpu power than p400

My gaming PC is a 9900K with a 2080 TI, I should probably try it out and see if it cvan handle several transcodes…though there is no way im getting another one for just Plex :slight_smile:

if its a server, just get a cheap p400 and be done

or a used gtx 960 (make sure its gm206) if your server will fit full size cards

or if you are building new, get a cpu with an intel HD 630 igpu and you won’t even need a separate gpu.

Yeah but they are priced about the same, and I can fit a dual slot card in the server so i think I’ll try it out. Dont want to mess with high power requirements and a “GPU enablement kit” for the server so power via the PCI-E slot is a clear bennefit…but even the P2000 can run off PCI-E slot power alone, AND is single slot…I really like those…it’s just a tad expensive for just messing around :slight_smile:

Strange that Plex doesnt support x265 encoding…is there a reason for this limitation…I wonder?

I have a 980 and a 980TI as well, bit they require massive power to run, and I also read that the 960 llike yours should be better (because it’s running a never process die or something)