Please define ‘Windows Plex’.
There are currently 3 different Plex apps running on Windows.
If you install Plex Media Player (free download below) you should be able to do it, provided that your Android phone is connected to the same local network as the windows box with PMP on it.
Switch PMP into its ‘TV mode’
I have found that this configuration change in Windows makes remote control work a little bit more reliably: