I just looked at the wiki page, and I'm pretty there's a typo in that command. The file should be trailers.der rather than trailers.cer (the command has 'der' in it).
That's the only difference I can see between the command listed on the wiki and the command not the blog that the wiki refers to (below) that worked the last time I installed PlexConnect.
C:\OpenSSL-Win32\bin\openssl.exe x509 -in C: railers.pem -outform der -out C: railers.der