At the moment, 4K trancodes are typically a challenge with Plex. You can do some searches on the forums, others have seen this problem. See this post for more:
Your server setup seems quite reasonable. The 1050Ti is a perfectly fine transcoding GPU.
An alternative idea: You could encode a second copy of those 4K movies down to 1080p using tone mapping. It’s not easy though, there are a number of discussions about how to do it. Here’s one thread.
https://www.reddit.com/r/ffmpeg/comments/f0lrs8/convert_hdr_to_sdr/
If you’re successful, keep the transcoded file in the same folder as the original 4K file. Plex clients are far better now at choosing the best video file for a given client. So if your TV doesn’t support 4K, it’ll (hopefully) select the 1080p by default and skip transcoding.
