Recent Continuous Buffering on large movies

Okay. I’ve been though almost every thread on here and cannot find any solution that works so here I am, tail between my legs, begging for help. Let me start off by saying, I am a network engineer and former software developer so I welcome any tech-heavy suggestions.

The Problem
No matter what switch I flip or threshold I adjust, about 10 seconds into large movies (BluRay-grade movies) I get the dreaded Buffering wheel. I started digging through the logs and found nothing that jumped out at me aside from the log included below. It will continuously buffer on me regardless of what device I’m on on LAN. About 60 seconds after it recovers, it starts to play and then buffers again. This goes on until I pull all my hair out, stomp around, flip more switches, and drive myself insane.

Current Environment
Server:
HP Z800 (big guy)
Intel Xeon 2.27GHz (2 Processors)
96GB RAM
Recently downgraded from Server 2019 to Windows 10 (worked perfectly before and shortly after) - I plan to eventually go to Ubuntu

Network:
Ubiquiti ecosystem
WD NAS

Primary Streaming Devices:
Samsung TV Tizen
Chromecast

Due to the problem occurring on most devices, I’m led to think the problem is restricted to layer 7 on the server itself…comments, questions, concerns, and short stories are welcome…

Nov 10, 2019 09:38:24.339 [16692] ERROR - Transcoder: Failed to delete session directory (boost::filesystem::remove: The process cannot access the file because it is being used by another process: "C:\Users\jones\AppData\Local\Plex Media Server\Cache\Transcode\Sessions\plex-transcode-uepbzomus3jjl54uo7bn8kt9-625d66e9-7b9b-4c31-8406-438e243e299d\chunk-stream0-00002.m4s")
Nov 10, 2019 09:38:24.339 [16692] DEBUG - Whacked session uepbzomus3jjl54uo7bn8kt9, 0 remaining.

I have a similiar issue on my samsung tizen tv (q9fn)

Are you transcoding audio? (Newer samsung tvs dont support many of the usual movie audio formats)
When I play movies with ac3/eac3 audio = direct play, everything is fine.

I noticed a very similar problem just recently,
my server is based on an i7-8700 with a Quadro P2000 GPU, but everytime transcode is triggered it’s not able to keep running smoothly without buffering unless you limit transcode to 720p or less.

Notice that is limited to transcoding, with Direct Play there’s no problem at all, so it’s unlikely a network problem, and by the way both server and client are based on a gigabit internet connection.

Unfortunately I’m currently away for work, so I’ve only tried to disable GPU transcoding to see if that was related to the problem, but it doesn’t seems to be the case. I will dig through the logs when I will come back home.

Answering to @The8Darkness I’ve noticed the same behaviour with video transcoding and audio (ac3/eac3) via Direct Play, so I don’t think it’s related.