The error “Your connection to the server is not fast enough to stream this video. Check your Network.” seems to be a common message on clients and often it appears to be a catch all error, leaving the end user unable to identify what the underlying issues are. I believe this error message pops up in the following cases:
Real infrastructure errors
- The network is actually too slow for the current bit rate
- transient network load changing network behavior (say increasing latency)
Client side errors:
- Client side buffer is not sufficiently large to handle a high latency but high bandwidth connection
- Generic client errors that wedge the client’s network stack
- Client side difficulty decoding a particular stream (reencoding at same bit rate has fixed, or turning off direct play/direct stream)
Server side errors:
- Server’s transcoding is not able to keep up with the stream, could be storage I/O, CPU utilization, other processes (cron) consuming resources, etc.
- Various other server side interferences creating latency issues or failing to provide the stream to the client fast enough
Primary objectives are to
- Allow the end user and plex support to quickly identify if the network is the issue
- Monitor for server side issues and notify the server admin with the details (storage I/O latency issues, too many clients, CPU under powered, CPU being hogged by other processes, etc) and improve error messaging on the client, indicating it may be a server side issue rather than blaming the network
- If no network or server side issues are detected, indicate as much and prompt for support bundle from the client, notify the server admin to send in a support bundle as well.
– additional thoughts about experience / implementation
For users experiencing this behavior consistently, an easy to do first pass capability would be to include a network bandwidth test embedded in both the client and the server, something like iperf3. This test server should be as independent from the Plex Server as possible, to keep any issues affecting the main Plex processes from interfering with the network testing, launching an independent iperf3 (or similar) server as needed would likely be an excellent option. This would allow the end user to establish what typical network behavior between the client and server actually is and determine if this is an infrastructure issue (something the end user needs to work through) vs an issue with Plex.
Three ways to implement this network test, First is an offer to do a ‘tuning’ run when a client is first introduced to a new server, this could even offer to set the global maximum bit rate options on the client. Second is to offer a manual test, the manual method should include an option for an extended test. In the extended test the pipes are kept under pressure for at least 20 minutes and the client caps the bandwidth in stages, starting out at low bit rate and monitoring latency, escalating to a reasonable maximum bit rate (tunable by the client, but not more than the server will allow that particular client to stream videos at). Throughout the test latency should be monitored by the client and if significant latency bumps are noticed (enough to allow that client’s cache to drain more than 75% at the current bit rate, faster bit rates mean a same sized cache is effectively smaller), the test should decrease to a slower bit rate. The reason for doing a longer test of 20 minutes is to try root out any reoccurring but still transient network issues, as well as trigger any upstream service provider throttling behaviors, which may not kick in till a stream has been active for some time. The test log / results should be sent to the server admin and to Plex so you can aggregate this and improve the testing over time. This test method should also offer to make changes to the global maximum bit rate settings on this client, additionally the test should provide some form of report (graph + csv?) showing target bandwidth, average bandwidth achieved and latency achieved over the test to the end user and server admin to help them diagnose any infrastructure issues on their end. Third would be some form of prompt to launch a test when the client is actively seeing what appears to be network slowness. All of these cases will help the end user determine if there really is a network performance issue they need to resolve, or even side step encountering it by properly tuning the client up front. It should also help Plex quickly weed out non-plex related customer issues.
For the client/server related issues, you might have an independent connection where the client and the plex server can communicate out of band, using independent processes from the video server/consumer client if possible so that if the main process gets blocked it doesn’t artificially increase observed network latency. This link could be used to transmit notices to the client about server side errors and to do a continuous periodic latency check on the network while video is actively playing. Details about the current status of the server (transcode buffer level, client buffer utilization, last frame sent/recieved, current client status) can also be shared to identify if a stall is being caused by the server or client side code and log the issue and needed debug details. If latency on this link remains low then any errors are unlikely to be network induced and the end user should be prompted to send in a support bundle. The client could also notify the server of any issues it encounters and then the server notify the server admin so they can look into it.
Additional health checking could be added to the server side to detect things like other processes consuming too many resources (CPU / storage bandwidth / network / etc ), monitor for overload on the server caused by too many clients playing at once and so on. These more useful messages could be provided to the client if it encounters any stalls and/or queued for the server admin.
