Random loss of remote - hundreds of pubsub async errors

[SwiftPanda16]

@sa2000

Well this is not the way to ask for a feature - by using an api that was intended for different scenario. As I said a refreshReachability will get the same result once the port number has been set to zero at plex.tv end. It will only change if the user goes through the steps of disabling and re-enabling remote access or possibly restarting the server - or some periodic action by Plex Media Server. The end point being used is there for plex.tv to use when the user configures remote access and enables it and the user would be on that settings screen.

This is a publically available API on plex.tv. If Plex does not want the public to use that endpoint in a way that is not intended, then they should be applying some separate restrictions to it such as rate limiting. It is not the job of the user of the API to determine if something is intended or not. That is the job of the person designing the API. If something is publically available, then it will be used.

@sa2000

There are other means for Tautulli to achieve what it wants to do. One example would be to request the machine identifier from your server via GET /identity and then use the identifier to lookup the non-local entry for that server within the resources.xml for which the end-point is already published in this support article https://support.plex.tv/articles/206721658-using-plex-tv-resources-information-to-troubleshoot-app-connections/ . From the resources.xml, Tautulli can get the url to use to reach the server through the WAN public IP.

I have already address this in my private message to you, but it seems like you completely missed those points. Because you seem to be “throwing me under the bus” here, I am publicly posting my response to your private message for the record below. See “Re: “Using /api/resources”” and “Re: “Why not ping the server’s http://PUBLIC-IP:PORT from /api/resources instead of using /myplex/account?”” in the copy of the private message below.

@sa2000

Of course hitting plex.tv with such a request every minute from every server with Tautulli would be unacceptable and could be seen as a DDOS action - but doing it every 60 minutes may be ok - but the volume of servers doing it would need to be assessed by the Tautulli developers and perhaps negotiate with Plex to allow for high volume of requests which Plex may not have allowed for when sizing the plex.tv infrastructure.

Again, that is not how public APIs work. If this is unacceptable, then Plex should implement rate limiting. I have already offered to work with Plex on a solution/alternative to this issue. I even provided a suggestion in my private message but I have not received any feedback or response from the Plex team. See the last paragraph in the copy of the private message below.

@sa2000

I should stress that if Tautulli wishes to hit a plex.tv endpoint periodically from all servers it manages then the developer should approach Plex Inc and ask if it is ok - or establish what volume Plex would deem acceptable

You are a Plex Team member. You should have taken all this information to the Plex development team, which you said you have done. You have not provided me with a response from the development team, nor has anyone from the development team contacted me.

---

Copy of private message between @sa2000 and me on July 19, 2018 to July 20, 2018

@sa2000

Hi

I have been looking at logs from @JamminR regarding remote access availability checks performed by Tautulli

It relates to this forum topic Random loss of remote - hundreds of pubsub async errors

First i think we have an issue here in that the volume of Tautulli requests of PUT /myplex/refreshReachability is not acceptable and the pubsub servers were not sized to cope with a request for reachability every minute of the day from every server (or in this case - every server monitored by Tautulli) - this level of loading was i believe not anticipated or allowed for.

The second point, is that we are aware that the remote access availabilty indicator may be false sometimes because of some timing issues and overlapping requests for connectivity tests. I would like therefore to understand exactly how Tautulli detects that remote access is not available. Is it really down or is it just the indicator that is giving a false indication?

I also wish to see the difference if the minute by minute checks are perhaps replaced with checked hourly or every 30 minutes

This is an extract of all requests from 192.168.1.4 which is where i believe Tautulli is running on (about an hour)

<removed logs from JamminR>

and this is what @JamminR provided for around that time from Tautulli

<removed logs from JamminR>

@sa2000

I suspect you are seeing something odd in response to /myplex/account
Client apps find the route to the server through resources.xml from plex.tv

@sa2000

Added two members of the plex.tv team

@SwiftPanda16

Note: I have all forum notifications disabled. I only found out about this message because my own Plex Forum bot notified me of a new post in my Tautulli thread.

---

Re: “How Tautulli detects that remote access is not available”

Tautulli’s remote access check solely mimics the remote access indicator found in the Plex server settings.

1. The remote access indicator retrieves the status from the mappingState and mappingError values found at /myplex/account.
2. The mappingState and mappingError values are updated through PUT /myplex/refreshReachability.

My team and I have done some extensive testing to confirm this:

1. Make sure remote access on the server is working and close all Plex Web clients.
2. Go to /myplex/account and it should say mappingState="mapped" and mappingError=""
3. Disable remote access by removing the port forwarding rule in your router.
4. No matter long you wait or how much you refresh the /myplex/account page, the mappingState and mappingError values will not change.
5. Using Postman (or curl) send PUT /myplex/refreshReachability.
6. Refresh /myplex/account and it should say mappingState="mapped" and mappingError="unreachable".

The reverse is also true:

7. Re-enable the port forwarding rule in your router.
8. No matter long you wait or how much you refresh the /myplex/account page, the mappingState and mappingError values will not change.
9. Using Postman (or curl) send PUT /myplex/refreshReachability.
10. Refresh /myplex/account and it should say mappingState="mapped" and mappingError="".

This is the reason Tautulli is sending the PUT /myplex/refreshReachability and GET /myplex/account.

---

Re: “Hourly or 30-minute check”

This is an unacceptable delay for Tautulli to notify of remote access errors. The intent is to have quick notifications of server status. Tautulli should notify you that your server is having remote access issues before your users send you a text message asking why Plex is down.

Currently, the mappingState and mappingError must report a failed status 3 times, to account for false statuses or temporary loss of connection, before Tautulli will send a notification (i.e. there is a 3-minute delay from initial remote access issues to receiving the notification). I would consider 5 minutes an acceptable maximum delay from initial issue to receiving the notification. However, only checking the remote access status once every 5 minutes does not allow me to account for temporary false statuses.

---

Re: “Using /api/resources”

The is nothing in the /api/resources endpoint that directly indicates whether remote access is working or not. The presence value only indicates if Plex Media Server is running or not, regardless of remote access availability.

My team and I have tested this as well:

1. Make sure Plex Media Server and remote access are working.
2. Go to https://plex.tv/api/resources and it should say presence="1".
3. Disable remote access by removing the port forwarding rule in your router.
4. No matter long you wait or how much you refresh the /api/resources page, the presence value will not change.
5. Exit Plex Media Server.
6. Refresh /api/resources and it should say presence="0".

The reverse is also true:

7. Port forwarding is still disabled in your router.
8. Restart Plex Media Server.
9. Refresh /api/resources and it should say presence="1".

---

Re: “Why not ping the server’s http://PUBLIC-IP:PORT from /api/resources instead of using /myplex/account?”

Tautulli uses the indicator in the server settings to avoid confusion such as:

a. “Tautulli says remote access works, but the indicator in the server settings says it’s down” or

b. “Tautulli says remote access doesn’t work, but the indicator in the server settings says it’s up”

The assumption is that the remote access indicator in the Plex server settings is correct.

---

The Tautulli “Remote Access check” is really a “Plex Media Server is published to Plex.tv check”. The intent is to mimic the remote access indicator in the Plex server settings, and it is named “Remote Access check” to match the Plex server settings. The remote access indicator in the Plex server settings is not really a true indicator of remote access, instead it is really an indicator of “Is the Plex server published to Plex.tv successfully for remote clients to lookup the connection URI?” (i.e. DNS in a sense).

Example: You can “Disable Remote Access” in the Plex server settings and still forward a port in your router. You will still have remote access your server through http://PUBLIC-IP:PORT but the server settings will say “Not available outside your network”.

Therefore, the accuracy of Tautulli’s remote access notifications should be identical to Plex’s own remote access indicator. However, you are telling me that it is inaccurate because Plex’s own indicator is inaccurate because the pubsub servers are being overloaded. If Plex can suggest an alternative method of getting this information that maintains the level of service people have come to expect, I would be happy to switch to that. My suggestion would be to provide a websocket event that reports the remote access availability.

@sa2000

I fully understand the reasoning and the benefits

(i have seen residual false information in mappingError field)

The point i was raising is that I do not believe the Plex infrastructure was sized for a capacity where every server would do a check every minute. If you have a requirement to do that, then my personal advice (and I am not Plex) is to ask Plex to add the hardware capacity you need for your tool and see what you get back in a response.

@sa2000

I presume you have now enabled PM email notification on the forum

@SwiftPanda16

There are two Plex team members in here. What are their thoughts on all of this? I also assume that you have passed on all this info to other Plex team members.

JonnyWong16:

If Plex can suggest an alternative method of getting this information that maintains the level of service people have come to expect, I would be happy to switch to that. My suggestion would be to provide a websocket event that reports the remote access availability.

And I have not enabled forum notifications.