Just for the sake of whoever comes across this thread next (I went through every possible thread i could find regarding an AMD Plex setup and the first shred of hope was here from wiidesire before i really went in on this build), here’s a screen of my All AMD Plex Hardware Transcoder in action:
(AMD Opteron x3421 cpu & AMD Pro WX4100 gpu in a HP Proliant Microserver Gen10 on a bare metal install of win 10 1809 on a samsung evo 970 plus m2 pcie ssd)
This image depicts hardware transcoding a 4k 50gb+ 10bit hevc mkv file to 1080p h264 natively with no modifications to plex with plex using the dxva2 api & mf.
I’m using the latest drivers for everything involved & software pictured from left to right:
DXVA Checker, Plex Media Server - Dashboard, Windows Explorer, Task Manager
Questions I remember wanting answers to before i began:
Can this be done?
Absolutely 
Is this a great low power & tiny media server?
100% 
How many streams would this work for?
Probably only for a for a few simultaneous transcodes & probably not from 4k as a source in that circumstance 
That said, a better idea would be use something like an online sourced ffmpeg bulk convert app/script over your media to prevent the need to transcode on the fly in the first place, or at least lighten the load and reduce the need to transcode each video to only once 
Even better than that would be to write your own powershell script (or failing that modify one, plenty out there) to use whatever ffmpeg config your hardware can support to set this up in a way that suits your needs, for example to automate media file conversion in place as they land in your libraries or during a schedule of server down time or whatever works to your viewing habits, this way you have complete control of what happens to your media, and maybe you learn a bit about the whole decode encode process along the way 
