I started a fun little project that isn’t going as well as I had hoped.
I decided to run FreeNAS in a VM. That has actually gone fairly well. I followed some threads from the FreeNAS forum and have a pretty nice 11.2U7 instance running on ESXi 6.7U3.
I was also able to install PMS in a jail relatively seamlessly and have been watching movies on my TV without a problem so long as they are DirectPlay.
The problem comes with transcoding. I am not using HW transcoding so just relying on my new Ryzen 5 3600 which has a Passmark of almost 20000 and a single thread score of around 2800. Plenty of horespower.
When I try to watch a movie on my computer or remotely, all I get is a spinning circle. If I try to force my TV (Plex on Amazon Firetvstick) to transcode (for video quality to 720p 2Mbps) the movie never starts.
As a troubleshooting step, I decided to create a virtual disk to be used as a mount for my Transcode temp directory (running on SSD). When I try to watch a video from my pc, the Transcode directory fills very quickly with hundred and hundreds (actually now over 1000) chunk-stream-nnnnn.m4s files. Within 5 minutes, Plex consumed 28GB of space in my temp directory, but the movie never played.
So it seems like the transcoder is doing something. My CPU gets pretty crushed as expected, but it seems to have stopped after about 5 minutes and 28GB so it feels like it’s probably completely transcoded.
This is also the case when trying to watch movies remotely which tries to transcode down to 720p 2Mbps. I’ve let the movie try to start for 20 minutes and CPUs pegged at 100% but never starts.
Does anyone have any thoughts or ideas? Thanks in advance!
In order to get VNET to work when FreeNAS is virtualized, you need to enable promiscuous mode on the vSwitch or Port Group.
I had originally gone down this path, but I had 80% packet loss and no connectivity to Plex so I turned VNET off. Once VNET was off, all of my networking issues went away, but then playback of transcoded videos didn’t work.
I’ve since turned promiscuous mode back on my ESXi port group (and Allow Forged Packets) and I am actually getting transcoded videos to play. But I still have 60-80% packet loss. Even from my FreeNAS VM to my Jail. Both inbound and outbound.
So I guess I’ll move back to the FreeNAS side of the house and see if I can get the packet loss issue sorted out. Either that, of hope that the Plex team can update (if even possible) so the FreeNAS jail doesn’t require VNET. Not holding my breath there.
I hope this helps anyone that tries to go down this path.
And if anyone has any thoughts on why the packet loss on the VNET NIC, I’m open to ideas.
Seems like a issue with FreeNAS or FreeBSDs vnet or maybe some miss-configuration, but none is related to plex.
Plex like other softwares need a full network stack to work properly, old style jails don’t have this, they “share” the connection with the host and don’t even have loopback interfaces which has some limitations. VNET/VIMAGE solves this in FreeBSD but there’s still a few bugs arround it, and FreeNAS uses an older FreeBSD base (11.X) perhaps some of those issues are improved in FreeBSD 12? but that’s all me speculating… Regardless seems like you need to sort out the VNET issue inside ESXI (maybe try the FreeNAS forums first, i.e: https://www.ixsystems.com/community/threads/freenas-under-vmware-having-iocage-networking-problems.68379/) and hopefully that will be it, or maybe its not even fully supported.
Thanks @mikec_pt! Completely agree that it isn’t a Plex issue at this point. Other than Plex needing VNET, which I don’t know enough about how VNET works in order to understand why Plex requires it. It just seemed interesting to me that DirectPlay worked fine without VNET. My library is completely accessible without VNET. It even looks like transcoding works without VNET. There is just something with getting the client to stream the transcoded movie that is the issue without VNET.
Regardless…I’ve also got a thread going in the FreeNAS forum looking for thoughts/help. I knew I needed to attack this from both angles.
The most basic reason is that like I said without VNET you don’t have a loopback interface in the Jail, other things will fail like getting metadata, possibly codecs (unsure).
You can always choose to have plex on the host rather than a jail, after all your already using virtualization, so I’d say jails over virtualization are a bit overkill unless your testing but you’ll still likely run into similar issues.
Vnet is a form of virtualization (network virtualization), maybe thats even behind the problem, but again that’s best asked in FreeNAS/BSD forums or vmware.
A lot of the functionality of plex still works with none-vnet jails and more so if you completely disable security according to some threads here, but I would never advise that ofc.
Honestly from a brief Google search your best option is running plex in a FreeBSD VM directly (not jailed) if you want to keep ESXi has the hypervisor.
FWIW FreeBSD can also be an hypervisor via bhyve or even has a XenSource Dom0.
In the end it really depends on what you want from the hypervisor, ESXi probably offers a better UI and as all you need. But if the idea is just to run jails and on or another VM maybe bhyve is enough!?
One other thing is that Hardware Transcoding is not supported under Virtualization, but it is under a jail if the host is real hardware, so if your cpu supports QuickSync you might want to take that into consideration.
