Get a Nvidia Shield Pro and connect it to your Denon receiver.
Shield ← HDMI → Denon ← HDMI → TV
You’re running into the limitations of using Plex TV based apps and also your audio equipment.
You can get HDR video using the Plex app.
You cannot get lossless audio such as TrueHD and dts:X.
When playing media using the Plex app, choose an audio track that does not transcode (avoid TrueHD). Also, avoid SSA/ASS subtitles, as they can force a video transcode. PGS & SRT subtitles should be OK.
Longer Answer:
The TV does not support TrueHD (including TrueHD + Atmos). Therefore, Plex will transcode it to a supported format.
The Denon supports HDMI-ARC, not HDMI-eARC. Therefore, the TV cannot pass lossless audio such as TrueHD, TrueHD + Atmos, dts-HD, or dts:X to the receiver. You are limited to Dolby Digital/Dolby Digital Plus/dts at 5.1 channels or less and PCM 2.0 audio. Dolby Digital Plus + Atmos is supported, so you can get Atmos audio from streaming services such as Netflix.
Even if the Denon supported HDMI-eARC, you would still be out of luck. TV manufacturers do not support passing lossless audio from TV apps over HDMI-eARC.
The only solution to have both HDR video and lossless audio is the Shield connected to your Denon.
Note that audio passthrough was recently added to the Xbox. See Audio Passthrough support on Xbox beta for details.
Regarding Dolby Vision:
Dolby Vision on any platform is still very much a work in progress. Not all profiles are supported on all devices. Search the forum for “Sony Dolby Vision” for additional info.
Try playing one of your 4K HDR rips using the Plex app on the TV, but make sure you choose an audio format that direct plays - Dolby Digital 5.1 or dts 5.1.
Movies released on Blu-ray with TrueHD audio also have a Dolby Digital version of the audio track. If the DD audio track is not present, re-rip the disc with MakeMKV.
This thread has a lot of good info on 4K and Plex:
[INFO] Plex, 4k, transcoding, and you - aka the rules of 4k
Plex Dashboard → Now Playing
The AVR’s network connection has no bearing on Plex.
The TV’s Ethernet port is 100 Mbps. Many 4K HDR movies burst above 100 Mbps, resulting in buffering when using the Plex app on the TV.
If you’ve a strong 5 GHz 802.11ac signal where the TV is located, you may have better performance (less buffering) via WiFi than via an Ethernet cable.
