Did you enable hardware transcoding? The synology nas is… kind of notorious for not liking having to burn subtitles in while also doing HW transcoding. I had a nice long back and forth about that a few months back. While they never did claim to find a problem to fix (no patches later that addressed this issue), they did have a workaround using a different VAAPI HW transcoder driver, which for me made my NAS do subtitle transcoding much better than it did before the change.
Try what ChuckPa suggests here: Plex Server cannot handle transcoding with subtitles - #51 by ChuckPa
