did you ever attempt to use a different audio stream?
like either ac3/eac3 dd, or even a stereo audio track ?
if it only has problems with dts ma, then change the audio and see if something else works.
if so, then you were chasing a bandwidth problem for no reason.
if you have not already read @ Plex, 4k, transcoding, and you please read the entire thread
