4K content can put your setup to a test. With high-quality / high-bitrate files you can put some network components to their limit – in your case the Nvidia Shield has a 1 Gbps port which is fast enough… you’ll need to verify the other parts of your setup have a similarly fast connection.
When it comes to troubleshooting why some of the files don’t play ok you should first look at the server’s now-playing dashboard – specifically check if any tracks are being transcoded.
https://support.plex.tv/articles/200871837-status-and-dashboard/
There’s an extensive thread in the tips & tricks section about how best to deal with 4K material.