Losing server connectivity

So I’ve been using the plex app for Xbox one and loving it…do seem to run into and issue though. Alot of times when I load plex or back out into a different season or go Home etc…plex can no longer find my server. I click on Retry and then it pops in fine. It’s almost like my server (which is running on Windows 10, and comprised of some USB drives hooked up to it containing the media) goes to sleep and has to wake up again. Ill see the little red “!” and then it will turn back to green, or Ill go into media manager and it will come back. I also share this out to people who aren’t local, and they’ve told me the same thing happens. It’s not a huge deal but kind of annoying. I’ve gone into the network setting in Windows and turned of “sleep” on my NIC, I have no power down settings for any of my hard drives including the USB ones…still occurring. Thinking maybe because all my shared folders are on USB drives? Any insight anyone could give would be much appreciated.