Regardless of what machine you have Plex installed on, failing hard drives are failing hard drives. The only way to beat that is to have enough hard drives to backup your other hard drives.
I would advise that you ditch the two SSD’s in the DS1817+ and then fill it with either 10 or 12TB drives. Create two volumes with 4 drives in each disk group. Volume1 = Main data use, Volume2 = Full Backup of volume 1
With each volume using ext4 and SHR, you will have full backup and one disk fault tolerance for each volume.
When you purchase the hard drives, buy each drive from different vendors, because most drives are manufactured in batches and if one drive in a batch fails, it is highly likely that other drives from the same batch will also fail.
Buying from different vendors usually means buying from different batches, which lessens the chance of all drives failing at the same time, giving you the much needed time to replace a faulty drive should one fail.
If you need more drives, you can add two DX517 expansion units or buy another DS1817+ for about the same money.
As far as your files are concerned, you DO NOT have to transcode them, just make sure they are in MP4 H.264/AAC or AVC/AAC format with a High Level code of 4.1 or less and RefFrames = 4 or 5 with an overall bitrate of ~20Mbps. For 4K the overall bitrate should be ~40Mbps, but only if your systems will support it.
That may require you to re-encode some of your files and you can do that easily using HandBrake (free for PC/Mac) or any similar software. Yes it will take some time to re-encode your files, but if you get into a routine where you re-encode say 10 files a night, you will find that you get the job done in next to no time.
You can always replace or re-rip the files if it saves time.
Once the job is done, it’s done! and you wont have to worry about failing transcodes or underpowered NAS or the like, your files will play when you want them to and you will not have to fiddle and waste time trying to make things work which usually means re-encoding the file/s anyway.
It also puts far less stress on your NAS or other server machines which allows them to run cooler and use far less power than they would when being forced to transcode.