xbox PLEX app does not support atmos passthrough. While all output is in atmos format, only the xbox system and games use the atmos.
PMS will transcode anything HD audio. Transcoding audio isn’t quite as bad as transcoding 4k video, however for 4k content it is still alot of work (4k files are huge and high bitrate, that takes a lot of IO).
You can see this on the plex web status page, it will say transcoding the audio and/or video. That means that player doesn’t support that content type.
If you do not have an atmos/dtxhd receiver, one thing you can do, is select a different AUDIO stream when you play back.
For example, instead of using a default atmos audio stream, choose the dolby digital stream (or whatever audio is compatible with your playback device), this can help avoid audio transcoding.
You may not care about the audio, but you should care about what making sure your media direct plays.
As I said above, the nvidia shield is the only consistently good 4k hdr plex streamer.
They are on sale now, so I would suggest you invest in one if you wish to continue going forward with 4k.
as mentioned in the 2nd post, EVEN IF you get a cpu or gpu that can transcode 4k hdr, HDR will get washed out when converted to SDR.
you are treading down the path that many have already covered and there are tons of threads about 4k on this forum.
my setup
tcl/roku tv with 4k/hdr - can DIRECT PLAY 4k/hdr VIDEO but not HD audio (5.1 dd or dts only)
xbox one x - same as above
nvidia shield - can direct play pretty much every type of video and pass-through any type of audio
my xbox and shield are connected to yamaha receiver with atmos/dts-hd.
then receiver out to tv