Server Build Recommendation for 4K Transcoding with Tone Mapping (AMD Ryzen vs. Intel Comet/Rocket Lake)

Hi all, first off please do not reply to this thread saying “do not transcode 4k”, or “store a duplicate copy of 1080p and 4k files”, its 2021 and that advice is outdated and irrelevant considering the computer power available at our finger tips.

TL/DR: Ryzen 7 5800x with some kind of nVidia GPU to assist with transcoding, or just keep it simple and go with Intel Comet Lake / Rocket Lake instead?

I run an ESXi all-in-one server (FreeNAS, Ubuntu with Plex Media Server, and other VMs like Ubuntu minecraft server, ubuntu factorio server, windows home server 2011 for client pc backups). I’m currently on an Intel i7-4790 which is starting to show its age with some of the VMs I’m running so I just started an upgrade cycle but I think I made a huge mistake going with an AMD Ryzen 7 5800x.

Somehow I was able to land a 5800x CPU (and a 5950x for my desktop upgrade) at MSRP and thought it would make a great ESXi CPU for all my VMs. I liked the idea of having the same platform as my desktop, so that in a few years when I itch to upgrade my desktop again I can take the beefy 5950x and drop it in my ESXi AIO box and extend the life of the server.

However I totally forgot about Intel QuickSync Video (QSV), and fear I severely miscalculated. I don’t do a ton of transcoding (i’m not one of those idiots running a server with 20 randos on the internet streaming my collection), but on occasion when I’m traveling (especially at holidays) I like to be able to stream from my home server to my family or friend’s. I’ve invested significantly in my 4K UHD bluray collection after I got my LG OLED 2 years ago, and as cheap as storage is I’m not going thru the hassle to double rip, convert, store a 1080p and 4k copy. So I want to be able to transcode 4k (and now that Plex finally fixed HDR tone mapping, it should all be doable)

My Intel i7-4790 struggled to transcode even a single 4k without buffering constantly and eventually failing for “server is not powerful enough”. I believe this is due to the Haswell i7-4790 not having any HEVC decode/encode support, the iGPU is simply too old in this generation. (please see the QSV reference table below). Can anyone confirm this is indeed the case?

The good news is that I finished the upgrade and everything is working with the Ryzen 7 5800x, and now I’m able to transcode a single 4k movie, although the CPU usage is nearing 80%. Due to the free version of ESXi’s limitations, only 8 vCPUs can be assigned to any VM (since the 5800x is an 8 core / 16 thread CPU, this means i’m effectively only letting Plex see half of the CPU). Honestly, this is probably a more than reasonable amount of vCPUs considering the load the server will have with the other VMs running. So while its working, I’m concerned there’s very little headroom and I’m barely limping by (let alone no room for future expansion, aka if I ever needed a 2nd 4k transcode at the same time). Basically without any Intel QSV on the Ryzen, I’m having to 100% rely on software transcoding which is eating up the CPU and going to be consuming a lot of power too.

I’ve considered adding an nVidia GPU (GTX 1050, or 1650/1660), but most of my PCIe slots are already filled (2x LSI 2308s and 1x 10Gbe LAN card are eating up the x16 slots on my x570 motherboard). My only option would be to put in an x1 pci-e GPU like the nVidia GT 710 or get one of those x1 to x16 PCI-e slot adapters that miners use all the time and hope the cut bandwidth doesn’t reduce the transcoding performance. Not to mention the extra $$ and power consumption of the dedicated nVidia GPU. I’ve also read multiple accounts of nVidia GPU transcoding not being as reliable as Intel QuickSync in my research).

So all of this brings me to the realization that I probably should keep this ESXi AIO VM/Plex box running on Intel hardware just for the purpose of maintaining QSV to lower power and cpu usage during transcoding. Is there anything I’m missing or any other alternative anyone can suggest for getting similar transcode performance on AMD Ryzen? Preferably without having to add some monster $300-400 nvidia GPU? Because honestly at that point, it would make more sense financially to just resell the 5800x (since they’re still in uber demand right now) and buy Intel…

If I do go Intel, I can see Kaby Lake / Coffee Lake / Comet Lake all have the same iGPU and thus same QSV features from the QSV wikipedia table. Most would recommend to buy the cheapest CPU in that family for Plex, but for me with the ESXi / VM usage I need more cores so I’m probably looking at an i7 10700k or i9 10850k. (Perhaps an i9 9900k if I go previous gen). But looking forward, there are some changes in Rocket Lake coming such as HEVC 12-bit, VP9, and AV1 support. I don’t believe any of those are currently in use in 4K UHD discs today, so would there be any real benefit to waiting for rocket lake? Since I’ve got the server ripped apart now, I’m leaning towards getting this project completed now with Comet Lake / Coffee Lake parts from my local Microcenter, but if Rocket Lake would provide a meaningful performance (Xe GPU) or feature set upgrade, I’d consider holding off until march and using the Ryzen 7 5800x for now.

What do you guys think / recommend? Truly appreciate any tips and tricks and build recommendations for Plex 4k Transcoding with HDR tone mapping in 2021!!

Reference: Wikipedia Intel QSV Reference:


2 Likes

You have already answered the quicksync question about your 4790 with the chart you posted, you need at least a 7000 series intel cpu with 600 series igpu in order to support 4k hevc decode/encode.

The equivalent chart for nvidia @ Video Encode and Decode GPU Support Matrix | NVIDIA Developer.

a gt 710 is not going to help 4k transcoding, a bare minimum would be a GM206 model, GeForce GTX 750 / GTX 950 - 960 Maxwell (2nd Gen) GM206 2nd Gen

a second bare minimum would a quadro p400, which can handle ONE 4k transcode.
the p400 2gig vram (half height) and p2000 5gig vram are the only single slot 4k transcode usable cards that I am aware of.

else, you are looking at a gtx 1050+ with as much video ram as you can find, each 4k transcode uses about ~1.4gig of video ram.

So in your situation (and most everyone’s to be straight), an intel 7000+ with quicksync will likely be the cheapest, simplest, most energy efficient, server for 4k transcoding.

as to the question of waiting for rocket lake, since they aren’t yet available, that is anyone’s guess, but if you have time to wait, then I personally would wait, simply to avoid buying 10th gen or earlier that already going to be obsolete.

if you need something right now, get a 10th gen of suitable power/affordability, and you should be set for a long time.

While AV1 etc is enticing to wait for, no doubt the software support will take even longer, and by that time, you could add a future gpu that supports it, if it is that important to you.

Besides, its not like 4k blurays are suddenly going to change from hevc to av1.

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.