Plex web not showing my offline server

No. This is done to keep network traffic, client device app load, and load on the cloud infrastructure down.

If you need an alert when one of your servers is down, think about using external tools. e.g. Tautulli