I am having an issue with AC3 5.1 audio. It sounds awful when its loud and it seems like the centre channel is messed up. I have no similar issue when I use Roku media player and DLNA to play the same files.
I have tried every other sound format I have and they all seem fine, its just AC3 5.1 that has a problem.
Have you tried flipping the direct play option off\on in Plex on Roku to see if it makes a difference?
Have you verified it isn’t transcoding?
When you say “awful” and “messed up” can you be a bit more specific? It’s hard to describe odd sound of course but is it garbled or staticy or cutting out … descriptions like that might help.