After a few hours of troubleshooting a roku not being able to connect to the server and a red remote access indicator (but everything else working fine remotely) I wanted to post my solution.
sudo rm /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Cache/certificate.p12
(or similarly named .p12 certificate file)(similar directory for Windows/MacOS)
This was the error in my console when I tried enabling remote access prior:
SSL certificate problem: unable to get local issuer certificate