Remember, that as far as sharing content to external users, there are seven things that factor in:
1.) Your server performance
2.) Your network
3.) Your upload speeds
4.) The Internet
5.) The remote user’s download speeds
6.) The remote user’s network
7.) The remote user’s equipment receiving the stream.
You can control only the first three. Unfortunately, no one controls item 4, and your remote user controls 5-7.
I have one friend who gets 15MB/s bandwidth, which is what I have my streams capped (can only get 20MB/s upload with both my ISPs). My brother gets about 2MB/s because he refuses to add an additional access point for the location he has his TV or wire his TV for better performance. You cannot control their network and you can talk until you are blue in the face to have them change but, it is up to them.
As a network architect for a large company, here are my tips:
1.) Pay attention to your upload speeds. At your 50MB/s and having fiber, I would say you have done well, there.
2.) Make sure your network equipment has enough horsepower and a good infrastructure between your server and Internet connection. Personally, my server is a Synology NAS with two 1GB/s ports to the switch that also has two 1GB/s ports connected to my router. Those dual 1BG connections are port-channeled together. You don’t get a single 2GB/s link. You just get two 1GB links that distribute the load (and it can get really complicated how that is done). I also pay attention to my network switch and router performance. My router has the horsepower to provide up to 890MB/s of throughput. I also have built in Quality of Service into my network equipment. I am not requesting you suddenly become a network engineer. I am just saying you can go down a deep rabbit hole on settings to improve things on your end. However; it has been my experience, it probably isn’t your network as long as you were thoughtful in your setup and use good equipment.
Look at your server’s performance (CPU, Memory, Network, Disk IOps).
The rest, as far as what I would recommend, is have the remote user take a serious hard look at their home network.
And I ALWAYS prefer wired connections over wireless. My brother’s problem is that the access point is a floor beneath his TV and he has kids, so phones, tablets, work, personal PCs, etc. Wireless is a shared medium, so if you got 20 devices all connected up, you are going to have problems. I have 4 access points in my house, all with direct wires coming back to my network switch for their backbone instead of mesh. Not a fan of mesh.
Good luck. I am only hoping to maybe expand your scope of where to look. I do not wish to confuse you. People make a good living knowing the ins and outs of this stuff and it isn’t something someone can pick directly up in the course of a few days.