Pub/sub mechanism broken

Server Version#: Version 1.43.3.10828

something’s broken with the pub sub mechanism my DNS logs have just started filling up with this:

23:02:32

pubsub.plex.tv

23:02:32

pubsub.plex.tv

23:02:32

pubsub.plex.tv

23:02:32

pubsub.plex.tv

23:02:31

pubsub.plex.tv

23:02:31

pubsub.plex.tv

23:02:31

pubsub.plex.tv

23:02:31

pubsub.plex.tv

23:02:31

pubsub04.pop.yyz.plex.bz

23:02:31

pubsub04.pop.yyz.plex.bz

23:02:31

pubsub04.pop.yyz.plex.bz

23:02:31

pubsub04.pop.yyz.plex.bz

23:02:31

pubsub02.pop.sin.plex.bz

23:02:31

pubsub02.pop.sin.plex.bz

23:02:31

pubsub02.pop.sin.plex.bz

23:02:31

pubsub02.pop.sin.plex.bz

23:02:31

pubsub10.pop.lhr.plex.bz

23:02:31

pubsub10.pop.lhr.plex.bz

23:02:31

pubsub10.pop.lhr.plex.bz

23:02:31

pubsub10.pop.lhr.plex.bz

23:02:31

pubsub01.pop.hnd.plex.bz

23:02:31

pubsub01.pop.hnd.plex.bz

23:02:31

pubsub01.pop.hnd.plex.bz

23:02:31

pubsub01.pop.hnd.plex.bz

23:02:31

pubsub02.pop.fra.plex.bz

23:02:31

pubsub02.pop.fra.plex.bz

23:02:31

pubsub02.pop.fra.plex.bz

23:02:31

pubsub02.pop.fra.plex.bz

23:02:31

pubsub04.pop.fmt.plex.bz

23:02:31

pubsub04.pop.fmt.plex.bz

23:02:31

pubsub04.pop.fmt.plex.bz

23:02:31

pubsub04.pop.fmt.plex.bz

23:02:31

pubsub12.pop.ewr.plex.bz

23:02:31

pubsub12.pop.ewr.plex.bz

23:02:31

pubsub12.pop.ewr.plex.bz

23:02:31

pubsub12.pop.ewr.plex.bz

23:02:31

pubsub00.pop.dfw.plex.bz

23:02:31

pubsub00.pop.dfw.plex.bz

23:02:31

pubsub00.pop.dfw.plex.bz

23:02:31

pubsub00.pop.dfw.plex.bz

23:02:31

pubsub05.pop.atl.plex.bz

23:02:31

pubsub05.pop.atl.plex.bz

23:02:31

pubsub05.pop.atl.plex.bz

23:02:31

pubsub05.pop.atl.plex.bz

23:02:31

pubsub.plex.tv

23:02:31

pubsub.plex.tv

Care to elaborate on the significance of your link?

Plex use Cloudfare, and as such is impacted by their situation
Ref: https://status.plex.tv/

Understood, however the status page you refer to shows that the pub/sub system is working perfectly (apparently)

Got the same issue. It has worked for a few minutes about an hour ago and now it’s broken again.

All of pubsubXX.pop.fra.plex.bz Servers return HTTP 500 Errors for me.

obviously this type of thing is unavoidable if bits of the internet go down that plex rely on but it’s the poor programming choices about how to repetitively and an increasingly generate requests for the same URL over and over again, flooding the local network with unnecessary traffic. Below is just 1 minute of request

00:11:44
pubsub02.pop.fra.plex.bz
00:11:44
pubsub02.pop.fra.plex.bz
00:11:44
pubsub02.pop.fra.plex.bz
00:11:44
pubsub02.pop.fra.plex.bz
00:11:39
pubsub02.pop.fra.plex.bz
00:11:39
pubsub02.pop.fra.plex.bz
00:11:39
pubsub02.pop.fra.plex.bz
00:11:39
pubsub02.pop.fra.plex.bz
00:11:39
pubsub04.pop.yyz.plex.bz
00:11:39
pubsub04.pop.yyz.plex.bz
00:11:39
pubsub04.pop.yyz.plex.bz
00:11:39
pubsub04.pop.yyz.plex.bz
00:11:39
pubsub05.pop.sin.plex.bz
00:11:39
pubsub05.pop.sin.plex.bz
00:11:39
pubsub05.pop.sin.plex.bz
00:11:39
pubsub05.pop.sin.plex.bz
00:11:39
pubsub10.pop.lhr.plex.bz
00:11:39
pubsub10.pop.lhr.plex.bz
00:11:39
pubsub10.pop.lhr.plex.bz
00:11:39
pubsub10.pop.lhr.plex.bz
00:11:39
pubsub01.pop.hnd.plex.bz
00:11:39
pubsub01.pop.hnd.plex.bz
00:11:39
pubsub01.pop.hnd.plex.bz
00:11:39
pubsub01.pop.hnd.plex.bz
00:11:39
pubsub06.pop.fra.plex.bz
00:11:39
pubsub06.pop.fra.plex.bz
00:11:39
pubsub06.pop.fra.plex.bz
00:11:39
pubsub06.pop.fra.plex.bz
00:11:39
pubsub02.pop.fmt.plex.bz
00:11:39
pubsub02.pop.fmt.plex.bz
00:11:39
pubsub02.pop.fmt.plex.bz
00:11:39
pubsub02.pop.fmt.plex.bz
00:11:39
pubsub03.pop.ewr.plex.bz
00:11:39
pubsub03.pop.ewr.plex.bz
00:11:39
pubsub03.pop.ewr.plex.bz
00:11:39
pubsub03.pop.ewr.plex.bz
00:11:39
pubsub03.pop.dfw.plex.bz
00:11:39
pubsub03.pop.dfw.plex.bz
00:11:39
pubsub03.pop.dfw.plex.bz
00:11:39
pubsub03.pop.dfw.plex.bz
00:11:39
pubsub06.pop.atl.plex.bz
00:11:39
pubsub06.pop.atl.plex.bz
00:11:39
pubsub06.pop.atl.plex.bz
00:11:39
pubsub06.pop.atl.plex.bz
00:11:34
pubsub06.pop.fra.plex.bz
00:11:34
pubsub06.pop.fra.plex.bz
00:11:34
pubsub06.pop.fra.plex.bz
00:11:34
pubsub06.pop.fra.plex.bz
00:11:28
pubsub02.pop.fra.plex.bz
00:11:28
pubsub02.pop.fra.plex.bz
00:11:28
pubsub02.pop.fra.plex.bz
00:11:28
pubsub02.pop.fra.plex.bz
00:11:11
pubsub02.pop.fra.plex.bz
00:11:11
pubsub02.pop.fra.plex.bz
00:11:11
pubsub02.pop.fra.plex.bz
00:11:11
pubsub02.pop.fra.plex.bz
00:11:06
pubsub01.pop.lhr.plex.bz
00:11:06
pubsub01.pop.lhr.plex.bz
00:11:06
pubsub01.pop.lhr.plex.bz
00:11:06
pubsub01.pop.lhr.plex.bz
00:11:06
pubsub03.pop.yyz.plex.bz
00:11:06
pubsub03.pop.yyz.plex.bz
00:11:06
pubsub03.pop.yyz.plex.bz
00:11:06
pubsub03.pop.yyz.plex.bz
00:11:06
pubsub05.pop.sin.plex.bz
00:11:06
pubsub05.pop.sin.plex.bz
00:11:06
pubsub05.pop.sin.plex.bz
00:11:06
pubsub05.pop.sin.plex.bz
00:11:06
pubsub10.pop.lhr.plex.bz
00:11:06
pubsub10.pop.lhr.plex.bz
00:11:06
pubsub10.pop.lhr.plex.bz
00:11:06
pubsub10.pop.lhr.plex.bz
00:11:06
pubsub01.pop.hnd.plex.bz
00:11:06
pubsub01.pop.hnd.plex.bz
00:11:06
pubsub01.pop.hnd.plex.bz
00:11:06
pubsub01.pop.hnd.plex.bz
00:11:06
pubsub02.pop.fra.plex.bz
00:11:06
pubsub02.pop.fra.plex.bz
00:11:06
pubsub02.pop.fra.plex.bz
00:11:06
pubsub02.pop.fra.plex.bz
00:11:06
pubsub02.pop.fmt.plex.bz
00:11:06
pubsub02.pop.fmt.plex.bz
00:11:06
pubsub02.pop.fmt.plex.bz
00:11:06
pubsub02.pop.fmt.plex.bz
00:11:06
pubsub12.pop.ewr.plex.bz
00:11:06
pubsub12.pop.ewr.plex.bz
00:11:06
pubsub12.pop.ewr.plex.bz
00:11:06
pubsub12.pop.ewr.plex.bz
00:11:06
pubsub00.pop.dfw.plex.bz
00:11:06
pubsub00.pop.dfw.plex.bz
00:11:06
pubsub00.pop.dfw.plex.bz
00:11:06
pubsub00.pop.dfw.plex.bz
00:11:06
pubsub06.pop.atl.plex.bz
00:11:06
pubsub06.pop.atl.plex.bz
00:11:06
pubsub06.pop.atl.plex.bz
00:11:06
pubsub06.pop.atl.plex.bz
00:11:01
pubsub10.pop.lhr.plex.bz
00:11:01
pubsub10.pop.lhr.plex.bz
00:11:01
pubsub10.pop.lhr.plex.bz
00:11:01
pubsub10.pop.lhr.plex.bz
00:11:00
pubsub01.pop.lhr.plex.bz
00:11:00
pubsub01.pop.lhr.plex.bz
00:11:00
pubsub01.pop.lhr.plex.bz
00:11:00
pubsub01.pop.lhr.plex.bz

I’m also having trouble with pubsub.
Specifically when scanning a Music library.

Here is a short summary of what I have done so far in troubleshooting:

Verified the server is signed in with a valid Plex account and Lifetime Plex Pass.
Verified Internet connectivity from the Plex server.
Verified the Plex server can reach plex.tv and retrieve /identity, /media/providers, and account information.
Verified all Plex Music provider services (Metadata, EyeQ/Gracenote, AcoustID, LyricFind, Songkick) report online.
Recreated the Music library.
Created a brand-new empty Music library pointing at a test folder.
Restarted Plex Media Server.
Reproduced the issue on two independent Plex servers:
Windows 11 (original production server)
Arch Linux (Docker) (new production server)
Bypassed Pi-hole completely by configuring the Windows server to use Cloudflare (1.1.1.1) and Google (8.8.8.8) directly.
Also tested Cloudflare DNS over HTTPS (DoH) on the Windows server.
No DNS configuration changed the behaviour.
Movies and TV libraries continue to scan and refresh normally.

Here is an excerpt from one of my logs when attempting to scan:
Jul 23 20:xx:xx DEBUG - Request: GET /identity
Jul 23 20:xx:xx DEBUG - Request completed: 200 OK (Signed-in)

Jul 23 20:xx:xx DEBUG - Request: GET /media/providers
Jul 23 20:xx:xx DEBUG - Request completed: 200 OK (Signed-in)

Jul 23 20:xx:xx DEBUG - Request: GET /library/sections/6/refresh

Jul 23 20:xx:xx WARN - Need to be signed in and connected to the Internet to refresh a plex music library.

Jul 23 20:xx:xx DEBUG - Completed: 503 GET /library/sections/6/refresh

Jul 23 20:xx:xx DEBUG - [EventSourceClient/pubsub/pubsub.plex.tv:443]
Failure: Unexpected HTTP status code: 500

Jul 23 20:xx:xx DEBUG - Switching to next PubSub host…

Jul 23 20:xx:xx DEBUG - [EventSourceClient/pubsub/pubsub.plex.tv:443]
Failure: Unexpected HTTP status code: 500

I asked our ops team and there is an issue with pubsub but should be fixed soon.

As far as the status page it looks like it stayed green because it was not completely down but they are looking into how to get that to be more accurate.

This started working again on my system at 02:22 GMT