I know this question has been asked a few times but the last post I found is from 2015. So maybe Plex change his way of transcoding since then.
I want to build a new server and I don’t know if I have to choose a CPU with more core and lower frequency or a CPU with less core but higher frequency.
My server struggle to transcode x265 and some others file format.
Fewest cores, highest passmark. Remember though that HW transcoding is coming for Plex - you might want to hold off on buying until it has been at least released (and pref, mature a bit - there are “issues” with stuff like x265 and 10bit) and go for the Kaby Lake iGPU in conjunction.
It is in the alpha stage atm. Not merged yet with the normal releases so it is only the link provided in that thread which contains the HW transcoding capabilities (so far, it will surely change in the near future).