SSL/TLS Content Inspection with Remote Access

Hi All -
I’m running a Plex server behind a firewall that supports SSL/TLS content inspection. I’d like to turn content inspection on, but this can’t work with the current mechanism by which Plex is issuing certificates for servers (https://blog.filippo.io/how-plex-is-doing-https-for-all-its-users/).

If I use a custom SSL certificate on my server, and try turning content inspection on (SSL decryption on a Palo Alto firewall), then i get a certificate error mismatch on the firewall, because Plex uses strict certificate pinning for all re-directed sessions through plex.tv or the mobile app.

The firewall will throw the error: "‘reverse proxy key ‘domain.com’ doesn’t match certificate issued to ‘*.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.plex.direct’’

Where that string of a’s is the unique hash signature Plex has issued for my server and ‘domain.com’ is the cert/key for the namespace upon which my plex server can be found (plex.domain.com, ex)

If I remove the custom certificate and instead let the Plex server import it’s own plex.direct certificate unique for my server, then i would need to extract the private key somehow such that I can import it into the firewall to perform content inspection.

Either method is acceptable. In the second approach, I would need to know where the Plex server stores the private key - which I am guessing is privileged information?

I’ve also fiddled around with just using a manual server configuration on the mobile app and plex.tv web frontend. That can work, but then voids the utility and ease-of-use of plex.tv autodiscovery and remote access.

Any suggestions from the community?

Server Version#: 1.15.6.1079
Player Version#: Varies

The plex.direct private direct domain is how plex identifies hosts on your LAN now.
The most common solution, which works even for my pfSense, is

Allow private domain 'plex.direct'

Thanks ChuckPA! Although I think maybe you misunderstand my question. SSL/TLS decryption, otherwise known as content inspection, requires the requisite certificate and private key be placed on an edge device which is performing that inspection. The same certificate and key as is used by the server. In this case, PMS.

I have no problems accessing content, remote access, etc. Plex uses the plex.direct namespace as a re-direct to locally hosted PMS servers. That’s all well and good - but it buggers up content inspection. So either we’d need a copy of the cert and key used for each PMS instance when linked to plex.tv, or the plex.tv service would need to ignore pinning for PMS servers with custom certificates.

It actually has nothing to do with firewall permission rules, or DNS for that matter. It’s all about certificates here :slight_smile:

I understood that but first wanted to respond to the internal LAN part.

As for inspecting, decoding and then re-encoding the internal, in two words “NOT HAPPENING”.

You are trying to inject yourself as a MITM (Man In The Middle) device. PMS will not function that way. It knows to watch for it.

2 Likes

Thanks ChuckPA! At the application level, no application can ‘watch’ for for content inspection. That’s because content inspection takes place at the session and presentation layers where a TLS handshake is negotiated (all protocols, all ciphers, it matters not). So as long as the endpoint of the TLS socket is the same, even a PFS cipher is viable for content inspection. The application really has nothing to do with it.

Just in case you’d like a more practical example…content inspection works fine for a manually configured PMS server, with a custom certificate. So let’s say one were to open the plex mobile app, or the Xbox app, and point the app directly to their hosted PMS server using the manually defined servers panel. Provided that PMS server is using a custom certificate, content inspection at an edge device works just fine. This is my ‘backup’ approach. You might be right in that PMS is ‘watching’ for content inspection, but it certainly can’t stop it :-).

Of course this approach voids the intent of my original question, which is to use the plex.tv service, which is wonderful!

I realize it’s a pretty complex topic, certificates, pinning, inspection. Not the end of the world!

Not sure I understand any practical reason for wanting this, especially a server already on your local network under your direct control with content that you are providing.

Yep, fair point, it’s a relatively remote use case.

The plex.tv portal, and mobile app (sans manual server configuration) make for a much more seamless user experience. And consider if there were several Plex servers behind a single gateway (personally I have two), the manual server configuration tops out at two instances.

But of course using the plex.tv portal, and Plex-provided SSL certificate mechanism means content inspection has problems, even when using a custom certificate.

So yeah, it’s practical, but not altogether prevalent if you take my meaning. If I knew where the private key for the PMS instance was stored…then it’s a pretty quick fix :wink:

But alas - moving on - sounds like I need to stick to a manual server definition and pointing my users straight at the custom domain vice plex.tv.

well anyone (other than the owner) with the private to key anything, means the bond of trust is broken.

which is obviously antithesis to the whole point of having a secure connection in the first place.

you might as well be asking your bank for the private key to their web banking portal.

1 Like

Oooo boy. This is why posting a question to forums is always a gamble.

I think you mean “chain” of trust. Actually, I shouldn’t be passive-aggressive. It’s “chain” of trust or a “trust chain”.

I am the owner of this PMS instance, and others.

And these certs that Plex has provided to us all are unique to the instance of PMS which is deployed. There is literally no parallel what-so-ever to exposing the private key of a web banking portal. I think you might be getting confused on the original ask here.

Listen, I get it, PKI, certs and inspection are complex topics. And an Internet forum isn’t the best medium to truly understand the complexities involved, especially for this corner case. Also add in that every forum contributor believes they are always right, and that everyone else is always wrong, yada yada.

No big deal - it’s a legitimate use case that just doesn’t have the prevalence to warrant a change anywhere. Moving on…

what I am saying, and I think you already understand, is that if you did find the private key to plex, then that means that plex’s certificate chain is broken.

because then you could obviously impersonate the real plex (which you apparently desire for your personal server(s) for “content inspection” purposes, coupled with the use of plex.tv domain system).

and while that may be all well and dandy for you and your own particular servers, but it means that everyone else that uses plex is compromised, in theory, if not in fact.

Close. I’m not looking for the private key to plex.tv. See the OP. Garnering the private key for “*.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.plex.direct” would solve the problem.

Where the string of a’s is the unique hash of the instance ID for the PMS server. Thus, this private key is unique only to my instance of PMS. Since I’m also the owner - it’s not like I’m exposing anything. No trust chain is broken because I’m not asking for the private key of the root cert, the CA, or any intermediary. No one else is compromised. Now if I were asking for the key for an intermediary, or the CA, or anything else in the chain other than my server cert…you’d definitely have a point. But that’s not the case.

This CN is unique for every PMS instance where remote access is enabled. It’s actually pretty well described in that link in the first post.

ok sure gotcha.

but while you/we may be the owner of a server, unfortunately that doesn’t mean we own the plex.direct domain or any of it is real or potential subdomains or wildcards.

in the end, I think you’ve answered your own question, and that is to continue to use your own custom certificate and private domain to accomplish your goal.

this leaves you to your purpose, and alleviates any confusion or liability or complications involved with a plex owned domain and its subdomains.

Returning to the thought that inspection is needed for *.plex.direct is the fundamental problem here.

*.plex.direct is a DNS concern, not a content concern. Allow the LAN-private domain and this issue is resolved. *.plex.direct us not used for WAN communication therefore negating the need of an edge device traversal.

To demonstrate how this is resolved by using my pfSense.

In the DNS resolver, allow declare the private domain (which allows it)

I will also add and clarify:

Plex is designed and intended for the HOME environment and not an enterprise environment where nothing is trusted and everything subject to inspection.

If, in one’s home, something is not trusted, stop using it
If, in one’s home, a person or computer is not trusted, don’t allow them to operate

1 Like

Peruse this support article.
https://support.plex.tv/articles/200430283-network/

You should be able to define a custom certificate for Plex to use (in conjunction with a custom domain).
If you use a certificate which doesn’t get broken by your gateway’s packet inspection, you should be good.


Side note: I have no personal experience doing this. So I’m unfortunately not good for any follow-up questions.

I’m also trying to get SSL decryption working. I’ve managed to extract the local certificate from the PMS server and set it up for inbound decryption, but it didn’t work.

I’m going to be spending some more time on this soon. The two things I want to get working are:

  • Inspect external connections
  • Inspect internal connections where I can deploy a trusted root CA (e.g., Apple TV)

My next steps are to test deploying the internal CA cert on my Apple TV and see if I can decrypt that. Then I’ll try the Plex certs again.

For those who don’t get why we want to do this:

  1. I own the server, the client, and the firewall. The bank analogy doesn’t really make sense, since I trust myself. (And for external connections, I STILL own the server… so they can either trust me and accept that I’m inspecting content, or go somewhere else.)

  2. I DO NOT trust the Plex application, since I didn’t write it. Who knows what sort of bug could creep in there? By decrypting and inspecting the SSL, the firewall can look for maliciousness and vulnerabilities/exploits.

  3. SSL is useful for protecting connections across untrusted networks, but it’s a huge security risk to have gigabytes of encrypted traffic flying around your private network and not have any visibility into it. That doesn’t mean not to use SSL, but it’s a perfectly normal use-case for your firewall to break it and inspect what’s going on.

you can do all that with plex > settings > network > disable secure connections

what you are doing is attempting to hack plex’s private certificate keys so you can keep using plex single sign on domain instead of using your own private domain.

I still want to use secure connections from the Internet, though :slight_smile:

Hack is a pretty strong word that has connotations of unlawfulness or maliciousness, which isn’t what this is about. I’m “hacking” in the sense of trying to get it to do something it clearly wasn’t designed to do with the goal of having more visibility over my network traffic.

get your own cert and domain then ?

and hack is accurate, you are attempting to gain unauthorized access to some other business private certificate keys.

there is nothing legal or un-malicious about it.

If I may provide some information here?

The packet inspection requested/attempted requires the Plex private key.
It will not be handed out.
Sorry.

@c.megalodon - virtual high five! It’s a complex topic, glad someone arrived who understands the request!

@ChuckPa - it’s actually only the last private key in the trust chain that matters, the one used just for the specific instance (our instance, the uniquely hashed instance) of Plex we are hosting. But I get it, it’s probably protected via password and Plex doesn’t want end users getting access to the private key of the certificate specifically assigned to their instance. Totally understandable.

My proposed solution persists, just use a custom cert and manually configure the plex.tv service and any platform app with the public and private (if necessary) addresses. Plex provides a facility for manual configuration - limited to two addresses - but it’s there all the same.

Thanks everyone for their thoughts!