It’s not the encryption which triggers the transcoding. At least not directly.
What is more likely to be happening here is this:
You are disallowing unencrypted connections. This leaves the client no other choice than to use an encrypted connection to your server.
And in your case this appears to be either the remote connection (which is usually bitrate-limited) or (worse) the Plex relay connection (which is limited to 1mbps without Plex Pass and 2 mbps with Plex Pass.).
It is this bitrate limitation which triggers the transcoding. Because you cannot control/change the bitrate of a media stream without transcoding it.
Now this means that there is apparently no direct encrypted connection to your server available when the client is within your home network. i.e. the same network as your server is.
Which naturally begs the question: “why?”
Encryption is done with security certificates. And such a certificate is always connected to a domain name. But normally, your server doesn’t have a domain name within your home network. It only has an IP address. And even if you set up an internal DNS server for your home network (in order to provide domain names), you also need a security certificate which is connected to that very domain. And it must be a publicly trusted certificate – i.e. not a self-signed one.
Plex is already providing your server with a domain name by default, together with the necessary certificate. These are issued automatically as soon as you connect your server to your plex.tv account.
However, many DNS resolvers you find in home routers or some web browsers won’t accept a domain name that has been defined by a DNS server that sits outside of the home network where the domain name is supposed to point at.
There are security reasons to do that. The name for this is “DNS rebinding protection”.
However in your specific plex-related case there is no malicious intent.
So, you want to check all these points, but number 2 in particular: Client says it won't play from "remote server", although the server is local
