Plex remote works *sometimes* and I can't figure out why

Did you go thru the Remote Access Troubleshooting guide?
If you disabled UPnP on your router when you set it up for manual port forwarding, you’ll need to re-enable UPnP when you turn off manual port.