This is usually based on a bad remote access setup. If the other server is located in the same network and has no issue, this implies the issue will likely be in your own router or home network configuration.
This guide is a good start to dig into what’s causing remote access issues:
