Repeated Plex Crashes (Memory allocation failures)

My Plex Media Server Process has been arbitrarily crashing about once a day, and I’m trying to figure out why. This started about two weeks ago, and has not let up since.

Plex is on the most current version, and there have not been any significant computer configuration changes during the time this started (I’m on Windows 7 64 bit). I’ve been running my server for well over a year at this point and have never had issues like this.

I’ve included a few crash dumps in the Plex Crash folder and my server logs in the other folder. Thanks in advance for any help you can provide!

Plex Crash.zip (18.8 MB)
Plex Media (1.6 MB)

The crash on 23 July at 04:41 appears to be due to running out of memory. There may be a memory leak leading to the Plex Media Server.exe process using up the whole 2Gb memory that it can occupy.

We have seen memory leaks when using VMWare and relating to the D3D9.dll. Also there were recent bug fixes - but these fixes would already be in the release you are running on v1.13.4.5271-200287a06

The logs show first sign of errors at

Jul 22, 2018 14:18:39.028 [3088] ERROR - Thread: Couldn't add a new thread to the pool of size 5: boost::thread_resource_error: An attempt was made to load a program with an incorrect format

Later on - other logged errors

Jul 22, 2018 17:05:51.759 [10388] ERROR - Format [JPEG] - Insufficient memory (case 4)
. . .
Jul 22, 2018 17:05:56.200 [11676] ERROR - Format [JPEG] - DIB allocation failed, maybe caused by an invalid image size or by a lack of memory

which eventually led to a crash at

Jul 23, 2018 04:41:21.977 [2964] WARN - Crash: Crash reporting disabled

To investigate memory leaks, need to use tools as was used in these investigations

using tools SysInternals VMMap and SysInternals ProcMon with VMMap running after PMS is launched and and ProcMon to be run for 5 to 10 minutes during a time when VMMap is showing stead climb in memory use and reached over 1Gb

Have a look at the other thread and let me know if you need any clarification

Thanks very much for the clarification. I’ll start by taking out the d3d9.dll, as that would have been a recent addition while I was attempting to get a Steam game to work. I’ll investigate the other tools after if I’m still having issues and report back, thanks!

Well, I can’t abandon that dll as it is needed for a critical program, so I started monitoring the process. Definitely looks to be a memory leak issue of some variety. I’m attaching a current batch of logs and an mmk file for you to look at. Plex hasn’t crashed again yet, but at the rate things are going, I think we’re headed down that road again.

Plex Media Server Logs_2018-07-25_01-10-25.zip (1.6 MB)
Plek MMK.zip (341.1 KB)

Thanks - it is the stack memory use as showing in VMMap that is going up - which as was seen in the D3D9.dll case before is due to threads being created - each occupying 64 Mb

need to now bring in ProcMon.exe - during a period of the commite dmemory climbing as can be seen here - would like to have a PML procmon capture for 5 to 10 minutes - to go with a VMMap mmp snapshot and logs for the period

Last time D3D9.DLL was showing in the procmon as creating those thresds, it was thought that it was VMMware doing it. Are you running in a VM or is this an actual Windows 7 SP1 installation?

With regards to Procmon capture - please zp the PML. Upload to dropbox or other service and send me link by private message. Please do not filter the procmon capture before saving the PML - ie save all events

Can do. In answer to your earlier question, this is not a VM - this is a Windows 7 SP1 installation.

Spent a number of days looking into the PML and VMMAP captures and logs. Nothing conclusive - have not been able to get all the PMS stack frames out of procmon.

The snapshots taken showed what looks like a threads leak

Snapshots
At 01:25:10 = 604 Threads
At 11:43:50 = 615 Threads 
At 14:02:09 = 761 Threads (significant increase - in 138 minutes +146 threads)

Would need PML capture from procmon for period when the number of threads ramps up like this - to go with vmmap capture and logs. May also be of benefit to increase log files to 100 using LogNumFiles
https://support.plex.tv/articles/201105343-advanced-hidden-server-settings/

Hi @bgardner1087

Spent a lot of time analyzing the process mornitor capture and logs from 13th August. Threads are being leaked with over 800 threads leaked over 4 hour period.

Again the evidence is pointing to D3D9.dll with the dll involved being this file
C:\Windows\SysWOW64\d3d9.dll

Could you check the version and see if there have been any bug fixes

Some more info after looking into more stack addresses

These were the DLLs on the stack for the threads that leaked new threads

26	<unknown>	0xfeed662	0x0feed662	C:\Windows\SysWOW64\atiumdag.dll
27	<unknown>	0xfe83607	0x0fe83607	C:\Windows\SysWOW64\atiumdag.dll
28	<unknown>	0x505b8fbc	0x505b8fbc	C:\Windows\SysWOW64\atiu9pag.dll
29	<unknown>	0x505b70aa	0x505b70aa	C:\Windows\SysWOW64\atiu9pag.dll
30	<unknown>	0x6b17791a	0x6b17791a	C:\Windows\SysWOW64\aticfx32.dll
31	<unknown>	0x6b4c81bf	0x6b4c81bf	C:\Windows\SysWOW64\d3d9.dll

So bugs could be in any of these 4 dll’s

That’s less than ideal, as those tie back to my graphics card and DirectX. I’ve been looking at the DirectX angle and haven’t come up with much, so I’ll see if there are any driver updates ATI has put out.

Looking at all the stack frames - these are all the dll’s that were involved (one being part of Plex)

C:\Windows\SysWOW64\atiumdag.dll
C:\Windows\SysWOW64\atiu9pag.dll
C:\Windows\SysWOW64\aticfx32.dll
C:\Windows\SysWOW64\d3d9.dll
C:\Windows\SysWOW64\MFPlat.DLL
C:\Program Files (x86)\Plex\Plex Media Server\avcodec-58.dll
C:\Program Files\Common Files\ATI Technologies\Multimedia\AMDh264Enc32.dll
C:\Windows\SysWOW64\evr.dll

They were not all in the same backtrace

As this is not a widespread issue, it is very unlikely that any further effort will be put into it. There has only been one other case of similar issue to do with codecs and that was seen was on a system with vmware running

My advice would be to ditch windows 7 and seriously consider Windows 10 for Plex Media Server. We have discussed this before.

If you have not tried it already, you could see if this windows integrity check and fix helps
in admin command line window, execute
sfc /scannow

Tried that this morning to no avail - says everything’s fine, but preliminary evidence suggests otherwise.

Windows 10 is on the shortlist of upgrades I’m looking at, but I’m trying to hold off until I can get a bigger SSD in a couple months. My primary concern is that d3d9.dll will likely be a part of my Windows 10 install too - is there something 10 does to prevent memory leak that 7 does not?

D3d9.dll was not in every thread leak stack.

Windows 7 does not come with Media foundation by default so there may be specific windows 7 aspects here

You could also check other dll’s in the list to see if they are right versions

May be some files are corrupt - did you ever have any disk issue needing a chkdsk repair to run ?

Have you been through a reinstall of windows 7 ?

With regards to Windows 10 - i have not seen this issue from users

A Windows 7 reinstall is next - but it’ll need to wait until a time where I can afford to have my primary computer offline. I work from home, so I’m trying to find a way to make that happen.

I do think some of the AMD dlls have issues - their drivers have never installed smoothly, even after uninstalling/reinstalling - and my administrative command prompt is missing a tool for uninstalling dlls, so corruption does seem probable. This particular Windows 7 install has been running a few years, so it could be any number of issues.

Thanks for looking into this, I’ll see what I can do with either a reinstall or Windows 10, and will work around things until then.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.