FYI: I am using** Plex Cloud** so no server settings were available to me.
I've had a very annoying issue where audio seems to start stuttering at various times and videos. There isn't really a correlation that I could fine. Strangely enough this doesn't happen with local playback on my android phone, windows pc, or mac osx chromcast. This really helped narrow the problem down.
This might not solve for everyone's issues, but it certainly did for me. I believe the bad combination was Android and Chromecast, and more specifically the fix lies in a chomecast setting.
One way to know if this will fix it for you is to try and CAST the video from a laptop or pc. If the stuttering continues this probably wont work for you. If it fixes it, you might be in this weird Android/Chromecast issue that I had.
*In Chromecast App find: Device Settings->'Let others control your casted media' and turn it OFF.
My only guess why this works is that this was creating a feedback loop during playback as my phone would have 2 different play buttons to start/stop media. One was plex, and the others was chromecast.