There is a difference between dropping remote access (and access to the server remotely no longer possible) and remote access availability indicator saying it is unavailable.
If there is a port forward in the router and manually specified port setup then Plex Media Server does not the means to dis-establish that and it should remain working.
The availability indicator may be wrong sometimes - but there is a change in the latest beta in 1.19.4 to minimize the chances of the indicator being wrong
See release note
- (Remote Access) Status may initially show as unavailable on server settings web page. (#5114)
Logs would be needed to investigate the loss of remote access
