It sounds like you’ve installed the PS4 app and pointed it at the computer as the main home screen ?
If so, all you need to do is reset it to point to the PMS server on the NAS as the default.
I’m not sure how the PS4 player is structured but for me, I’ve found it easier to just reinstall the apps and point them to the desired PMS server as I install them.
It seems like the PC is the centre for plex distribution because even if the NAS is on, if the PC is off I can’t access my media from my phone etc either.
Install PMS on the Synology, all setup and ready to go.
Usually the easiest way to make the phone go right to it is a quick re-install of the phone app then Pick the “Preferred Server”. ( I always nuke the app and start over – i’m lazy too haha)