I had the same (or similar) problem. It turned out that it was the audio causing it to transcode. Samsung removed support for DTS audio in their latest TVs, so any files with a DTS soundtrack were transcoding not just the audio but the video too (Plex bug). To fix it, I used ffmpeg to add an AAC audio track to any files with DTS. Now my Samsung TV can use the AAC track and my LG TV can play everything without transcoding.
More recently I noticed in the latest Plex server for Synology there’s a bug which causes transcoding to occur when SRT subtitles are enabled. I rolled my server back to a slightly older version (1.15.0.659) and that fixed the issue. Some more details here.
