Explanation, in technical detail, about how to avoid relaying?

If you client is using a relayed connection, it means it could not reach your server directly.
Switching off the relay connection would result in no connection at all.

Now why the client cannot reach the server directly, can have several reasons:

  1. The server has no public IPv4 address (if your internet service provider only assigns you a public IPv6 address, you cannot use direct Plex remote access at all)
  2. There are issues with the domain name resolution of the .direct TLD. Your server is contacted by clients with its own FQDN (xxx.yyy.plex.direct), which it gets assigned upon server start. Many ISP-provided DNS server mess this up. This can be rectified by using other DNS servers, Google’s for instance (8.8.8.8)
  3. If you also cannot connect directly to your server in your own local network, it may be also have to do with your router. see PMS using wrong IP for a quick test and some things you can try.
  4. If your server is running in a virtualized environment (Docker et al.) it could be indeed a double-NAT issue.
  5. If your local network has more than one router (or any other device which performs NAT [even some PowerLAN bridges do it], it could be a double-NAT issue).
  6. If your router is isolating the ‘wired’ from the ‘wireless’ portion of your local network, your local client can also have difficulty to reach the server directly.