Plex Server Crashing Randomly

From the research I did yesterday, if you have a GUI it’s as simple as going to appwiz.cpl, uninstalling Plex from there, and then running the .exe for an older version, which you can find at GitHub - axlecrusher/plex_download_urls: Archive of Plex mediaserver download URLs

For me was a bit more complicated as I run Server core, I had to find the Plex installed products with wmic product and then do msiexec /x xxxxxx.msi with the installed products I identified, and even after that it still told me I couldn’t install an older version as it was already installed.

In the end I realised I could run the original plex installer for the version I had, plexmediaserver_1.27…exe, and there it gave me an option to “Repair” or “Uninstall”. The uninstall cleaned up the last few bits, after which I ran plexmediaserver_1.25…exe

Now I’m running 1.25.9.5721 but it will be some time before I know if it’s actually stable.

A lot of posts but not many server logs posted here captured after a crash

I can see logs from @pvk8336 but these are more than a month old - from June

I am looking at these logs

I can see after the upgrade to 1.27.0.5878, it took over 2 minutes for the database to get migrated to a new schema - during such db updates the server is not accessible and Plex Relay may get started by plex client apps

However, after the completion of the update, there is evidence of requests not being seen by the server.

Do you have any 3rd party proxy software running on the PC ? You can check by going into “Control Panel” / “Internet Options” / “Connections” / “Lan Settings”

Also you can get netstat output using an elevated command line window
Start / Run / cmd
Right click on “Command Prompt” app and select “Run As Administrator”
Then run netstat with something like

netstat -a -b -n > "%LocalAppData%\netstat_2022-Jul-15.txt"

and then upload this file
"%LocalAppData%\netstat_2022-Jul-15.txt"
and info on proxy server in use

The example i saw in the logs was

2022-06-08 10:29:39,842 (2814) :  DEBUG (networking:143) - Requesting 'http://127.0.0.1:32400/servers'

which failed and got a 500 error code

2022-06-08 10:29:39,842 (2814) :  DEBUG (runtime:924) - Response: [500] 3603 bytes

But the request did not get to the server according to the log - which suggests some proxy software intercepted it

Also earlier on there is a lot of evidence of corrupt http requests - again could be because of some application intercepting the requests and corrupting them

One of many examples

Jun 07, 2022 19:04:43.922 [8664] DEBUG - Request: [100.xxx.xxx:19219 (WAN)] WqN  (26 live) Signed-in
Jun 07, 2022 19:04:43.922 [8664] ERROR - Error parsing HTTP request: qN;ì¤ ú>{ƒ˜VI§™u·ƒ[FZ»äauКT„SÝïŸ|}åTqšå¿¢Ïže¢c‚!j\M‚›l/ø¨7U
Jun 07, 2022 19:04:43.923 [13560] DEBUG - Completed: [100.xxx.xxx:19219] 400 WqN  (26 live) 0ms 265 bytes

For anyone else with crashes - please run with the current version of Plex Media Server and with debug server logging not disabled and upload logs zip capture after a crash

Is there any personal info in the logs that I need to take out before uploading? Will the logs even show the problem since I had to restart to make Plex work again (not sure how much time is kept in the logs)? Would you like me to upload the entire Zip file or is there something specific in the Zip that would be better? I am all in for helping the devs identify the issue, I just want to make sure my personal info doesn’t get exposed. I can PM the logs if needed too.

EDIT: I went into Debug, set server logging to “Enable”. I will be happy to send logs if it happens again.

2nd EDIT: From my reading, it sounds like my issue is a deadlock issue as Plex is not disappearing from the system tray. sa2000, is there something else I need to provide when/if the issue comes back again?

I think that the issue that I am having is what is called a “deadlock”. Plex icon stays active in the system tray, but when I try to open the Plex web interface nothing loads. The clients are unable to play anything from the server as well (local and remote). I found an old post from 2018 solving an issue called “503 Service Unavailable”. To everyone else here experiencing the issue first verify that the Plex icon is still in the system tray (Windows). Here are the diagnostic steps outlined by @sa2000 in the previous thread:

Diagnostics for a deadlock / lockout (Windows)

  1. Prepare yourself by finding the server security token string as it will be needed for the connections list request. To find the token string, see this support article Finding an authentication token / X-Plex-Token | Plex Support
  2. When the problem occurs (and please wait for sufficient time before capturing diagnostics or restarting eg 10 minutes of requests to the server not getting through) , get the output from a specific browser request on the server. In a browser, go to this url http://localhost:32400/connections?X-Plex-Token=xxxxxxxxxxxxxxxxxxxxx where you put the server token string instead of the xxxxxxxxxxxxxxxx. Copy the displayed output to a text file and save
  3. Next is to force a process dump for the Plex Media Server.exe process. We can do this with Windows Task Manager - but on a 64-bit windows and with Plex Media Server.exe being a 32-bit application, you would need to load the 32-bit x86 version of windows task manager. So load this program instead of the default task manager C:\Windows\SysWow64\Taskmgr.exe
  4. Using this 32-bit Taskmgr.exe find the Plex Media Server.exe process, select it and right click on it and choose Create Dump
  5. Check that the dmp file has been created
  6. Restart the server and capture the zipped logs
  7. Zip the dmp files and upload to dropbox or similar service
  8. send me by private message link to the zipped dmp files and send the saved /connections output text files and the zipped logs

Now on step 8, the “me” is not me, but an actual Plex dev (like sa2000).

Here is a link to the old thread for others who would like to read more: 503 Service Unavailable - #91 by ninja6o4

1 Like

@sa2000
Attached are my logs. I also am getting daily crashes. Appreciate any help.

Plex Media Server Logs_2022-07-17_08-17-59.zip (2.0 MB)

I’ve now started having this problem as well, with the same symptoms of the exe not actually crashing but the interface becoming unresponsive. However I’m still on 1.26.2.5797 and not on 1.27 like the others, which might be meaningful, or might not.

To be clear, my server has been issue free and rock solid, this only started in the last week where the same problem has happened twice within a few days. The Plex server stops responding despite the exe still seemingly running fine, there’s no actual crash. Since I’m on a different version I’m wondering if this new problem is plex-side rather than local.

In the log below the playback started buffering while watching, at which point I checked the server and saw that the exe was still active and the program was unresponsive. I couldn’t click through to any pages and couldn’t reload the interface, everything appeared to be dead other than the program still running. Eventually I just right-clicked and shut it down from the tray, which seems to have worked just fine, so perhaps it’s only the web server that becomes unresponsive. You can see some of this in the log, but there’s no obvious point where it crashed.

Playback starts sometime around Jul 17, 2022 19:48 with the unresponsive application starting at some point after that. Hope the log helps, but if you need full logs let me know and I can PM them.

rwoffice-server-stops-responding.log (5.4 MB)

Edit: sa2000 I PM’d you full debug logs, hope it’s helpful.

Logs may have public IP Addresses, Usernames and media paths within the filesystem

If providing me with a process dump and the /connections list for deadlock which would be best to be through a private message, would be ok to also include logs in the area you upload the process dump and text file for connections list to

I need three bits of diagnostics to start a deadlock investigation:

  • output from http://127.0.0.1:32400/connections?X-Plex-Token=xxxxxxxxx
  • process dump
  • debug server logs

and for this to be from the current public or beta release

@pvk8336 - further to my earlier response above, I noticed some memory allocation errors in the logs

Example

Jun 07, 2022 20:24:54.095 [1500] DEBUG - Request: [184.xxx.xxxx:49208 (WAN)] GET /photo/:/transcode?width=240&height=360&minSize=1&upscale=1&url=%2Flibrary%2Fmetadata%2F4332%2Fthumb%2F1653125423%3FX-Plex-Token%3Dxxxxxxxxxxxxxxxxxxxx (31 live) TLS GZIP Signed-in Token (xxxxxx)
Jun 07, 2022 20:24:54.095 [1500] DEBUG - Photo transcoder: Request for url [/library/metadata/4332/thumb/1653125423?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx] (is local: 1 upscaled: 1)
Jun 07, 2022 20:24:54.152 [1500] DEBUG - Auth: authenticated user 574970 as xxxxxx
Jun 07, 2022 20:24:54.205 [1500] DEBUG - It took 31.250000 ms to retrieve 124 items.
Jun 07, 2022 20:24:54.207 [1500] DEBUG - Calculated media file path for path [metadata://posters/tv.plex.agents.movie_880a22a3a5977cc775a65e76a747820545ca5656]: ["C:\Users\Pxxxxxx\AppData\Local\Plex Media Server\Metadata\Movies\5\8e035ab5b03db62cf0164cfd8438998ebe316d9.bundle\Contents\_combined\posters\tv.plex.agents.movie_880a22a3a5977cc775a65e76a747820545ca5656"]
Jun 07, 2022 20:24:54.305 [1500] DEBUG - Calling back into ourselves for photo to transcode, optimizing the process (status: -1)
Jun 07, 2022 20:24:54.337 [1500] ERROR - Exception handled: bad allocation
Jun 07, 2022 20:24:54.337 [13560] DEBUG - Completed: [184.xxx.xxx:49208] 500 GET /photo/:/transcode?width=240&height=360&minSize=1&upscale=1&url=%2Flibrary%2Fmetadata%2F4332%2Fthumb%2F1653125423%3FX-Plex-Token%3Dxxxxxxxxxxxxxxxxxxxx (31 live) TLS GZIP 241ms 405 bytes (pipelined: 1)

Could you also enable memory usage logging within Plex Media Server settings. This would need to be done by a registry edit
See Advanced, Hidden Server Settings | Plex Support
The setting is LogMemoryUse with value 1
It is a REG_Dword (32-bit) item

Also could you copy out this file, zip it and send it to me
C:\Users\Pxxxxxxx\AppData\Local\Plex Media Server\Metadata\Movies\5\8e035ab5b03db62cf0164cfd8438998ebe316d9.bundle\Contents\_combined\posters\tv.plex.agents.movie_880a22a3a5977cc775a65e76a747820545ca5656
Just want to see if it is a big file

thanks

Thanks for the logs

Can you check the Windows Event Logs through Event Viewer - opening the Application Event Log and see if there were any logged Application Errors / Exceptions at these times:

Jul 16, 2022 15:46:51
Jul 16, 2022 15:47:54
Jul 16, 2022 16:17:51
Jul 16, 2022 16:28:33
Jul 16, 2022 18:19:19

Also could you see if this file is zero length (empty) and let me know the date and time it was created/last modified
C:\Users\Scott\AppData\Local\Temp\55925e2a-916a-492c-bedf-ebb46c7d5b32

This was a dump file - but possibly empty if the crash was to do with running out of memory or some stack corruption

Could you enable the registry setting for Windows to capture the application crashes - see regedit on this support article
https://support.plex.tv/articles/201455336-crash-logs-plex-media-server/
and ensure this directory is created
C:\Users\Scott\AppData\Local\CrashDumps

Please Reboot after making the registry edit and creating the directory - and on next crash please get server logs and also look for any dmp file within C:\Users\Scott\AppData\Local\CrashDumps for Plex Media Server. Zip and upload and send me link by Private Message

I am not sure if Plex Media Server crashed or hung at the times I gave you to lookup in the Application Event Log, The server was processing a photos library and the stops happened after processing 5000 images. In one case it managed to process 10000

@scott_965 and could you tell me how many images are in the library - the library that has this path

C:\Users\Scott\Pictures\iCloud Photos\

(Would help getting zip of the database - you can send privately to me - you can download the database through Plex Web Troubleshooting web page)

It does appear to be a lockout and build up of requests. I do have an open issue at the moment with the development team on this.

If you are interested in trying out a development build, I can send you one - it has a fix for a compiler bug - I am not sure if it is relevant to this but coule be. It would be post 1.28.0 - which was released to beta today

Thanks @sa2000

There are no events in the Event View/Windows Log/Application at those times.

The file C:\Users\Scott\AppData\Local\Temp\55925e2a-916a-492c-bedf-ebb46c7d5b32 is a 0kb file last modified Jul 14, 2022 02:16. The file type is “file” but when I opened it with Notepad it had nothing in it.

C:\Users\Scott\AppData\Local\CrashDumps already existed. No registry edit was needed (that article talked about creating CrashDumps). I will send you that last PMS dmp file and my database file.

There are 33K photos in the iCloud Photos folder, but I don’t use PMS for photos (or music), so I can turn that (those) off. It could be as simple as deleting the photos library in PMS, but will wait for your reply.

Thanks

So the regedit was already in place

Were there any Plex Media Server dmp files in the folder

zero length files get created if Plex Media Server itself is unable to get a minidump created

I would like to see the database file before the photos library is deleted

(It may take a while for the DELETE library to complete - just leave it running)

No PMS dmp files in the CrashDumps file.

DB file link sent to you and I won’t change the photos or music library until you tell me too (as I said, I don’t use them).

This is exactly my issue. Everything had been stable up until this point. My Plex became unresponsive on 7/8/22 at approximately 1600 and on 7/14/22 at 1943. Plex server still showed as running in the system tray, not accessible from web interface or client side.

@sa2000 , if it happens again how much time can elapse before I get the output from the Plex Token connections and process dump? The reason that I ask is that I am at work most of the day, and if the error happens again, I might be home hours later before I can run the necessary diagnostic steps. The only time that I am aware of a problem is whenever a family member tells me that they are unable to stream. This also means that the problem could have been present for quite a while before I am able to try to get the logs. Thanks for all of your help with this, it is really appreciated!

2 Likes

@sa2000
I also noticed a “crash” on my Windows Plex, but it wasn’t.
The “crash” was caused by a Windows restart because of the patch day on 12./13. caused in July.

This would be helpful to know. I’m happy to provide the logs, dump file, etc, but the problem doesn’t happen very often, so catching it within a specific time frame could be difficult.

With deadlocks - the dump and connections list will still be valid hours later - however the logs would probably wrap round and not cover the time when the hang started and the build up of requests started.

So you could try to run with number of log files set to 20 as an example - see Advanced, Hidden Server Settings | Plex Support
The setting is LogNumFiles - a Reg DWord 32-bit field

Great, I changed it to 20. Thanks for the tip.

@sa2000 , had another “deadlock” incident. I PM’d you the connections text file, the dmp from 32 bit task manager on Plex, and the log files from Plex after restarting the server program (didn’t reboot PC) in a Dropbox link. Let me know if I can be of any extra help or if you have difficulty with the files. Seems to happen about once a week at this point.

1 Like