No direct Stream on Nvidia Shield TV, No way to enter the server IP manually

The warning about an indirect connection has nothing to do with Plex not seeing your server.
It means the server and player are on different networks (could be different subnets based on intentional or accidental network segmentation). Hence Plex considers the stream to be remote – however there appears to be an issue with your remote access setup, requiring the connection to be routed through a Plex relay server (indirect).

Entering your server’s IP address manually won’t change that.