Whitelist an ip address

This might not be possible but I’m really hoping it is cause it’s exactly what I need.

I’m running plex media server on my server and I’ve disabled the remote access.
Now I have a second home (not really but just easier to call it this then to explain) where I will be some days of the week
I want to be able to access plex on my xbox one at this place too without having to enable remote access entirely

So what I would like to do is whitelist the public ip of the second place so I have two spots that can access the server without problems
Can this be achieved in any way?

No

First, to access the server from outside your local network needs Remote Access to be enabled

Whitelisting is only needed if a Plex app does not have the capability of signing in to plex.tv to authenticate - example: LG TV Medialink or Smartshare

All access to the server has to authenticate and that is a requirement for local access and remote access.
For local networks, you can whitelist local IP address / range if your client apps cannot sign in. No option is available for whitelisting public IP Addresses

Unless you are using FreeNas plugin and its use of the internal DisableRemoteSecurity setting then all access from outside your network will need to authenticate and your server will only be accessible to client apps that know the plex.tv user and password.