Are you using automatic (UPnP/NAT-PMP) or manual port forwarding?
If automatic, consider changing to manual port forwarding. Manual does not require any negotiation between your server and the router.
You will have to assign a fixed IP address to your server and configure port forwarding in your router.
https://forums.plex.tv/t/the-basics-of-remote-access-troubleshooting/769517/4
