Happy to help.
I am sure other opinions will be along soon!
Yes, that would be fine. But there are cheaper options, too. What you really need is just this: a semi-modern Intel CPU with a GPU supporting Quick Sync, a couple of DIMM slots, a good network port, and a comfortable amount of storage. A modern NUC is one way to do that but there are a lot of small form factor PCs. Or, bigger PCs are fine too if you get a deal and have a place to put it.
In my opinion there is one major decision point when looking at new Plex hardware, assuming you are going to be relying on Quick Sync. Either you look for a bargain priced system using yesterday’s GPUs (e.g. UHD 630) OR you go for a much more modern chip with the new generation Iris GPU.
The new GPUs are faster and supposedly look better when transcoding/tone mapping but the previous few generations are also generally thought to be fine. But since a new server might be something you have for many years, perhaps it’s worth getting something that isn’t already getting a little old. The Iris GPUs support more formats which may pay off down the road.
FWIW my server is an i5-10400 (UHD 630) and it’s a very comfortable amount of power. What is comfortable depends on your usage though, it’s tricky.
With your prior experience I bet you would have no problem installing a modern version of Ubuntu and PMS. It’s very trouble-free. Once in a while I log in and issue a couple of commands to update the OS and PMS at the same time. That’s it. That’s all I ever need to do.
Your existing NAS is easily good enough to serve media files to a Plex server. You’d just set up SMB or NFS shares, and mount them to the Plex server. There is no need to replace the Syno unless you want to for other reasons. (My 1813+ is still going strong.)
Yes. However Quick Sync under Windows will transcode (resolution change) but not tone map (HDR to SDR). To get total hardware acceleration under Windows you need an Nvidia GPU. Don’t quote me but I think it needs to be a 1050 or later? Someone here will know.
Being both a Win and Linux AND BSD guy, I think that goal is actually more easily met under Ubuntu than Windows. But this isn’t the place for religious arguments.
If you get a server which allows you to easily change the boot drive, you can always start under Windows… And then when you want to mess around, pop in a different SSD and start working on a Linux build. That way you can figure it out with no pressure, if you ever decide you want to try it out.
