Memory leak since 1.19.1.2701 (possible > 1.19)

[GoodDogs]

Server Version#:1.19.1.2701 and 1.19.3.2764
Player Version#: N/A

I upgraded PMS for FreeBSD from 1.18.7.2457 to 1.19.1.2701 and then 1.19.3.2764 and since 1.19 then have seen swap pager errors (basically OOM). This was even after bumping the RAM by 512M to 2GB. My system is lightly loaded at worst, never more than 3 clients streaming untranscoded music (no video). It has 1GB of swap.

I notice there are other threads with OOM issues on Linux, and perhaps they are caused by the same code on FreeBSD. I’m starting this thread in case they are unrelated.

My system runs

• Plex Media Server
• Plex DLNA Server
• Plex Tuner Service
• Plex Plug-in

My server logs show:

# grep 'swap.*failed' messages | cut -c 1-10 | sort -u
2020-05-08
2020-05-10
2020-05-16

May 8 is when I first moved to 1.19

I’m still troubleshooting which process is to blame. It does appear to free the RAM soon after so perhaps some watching dog is restarting a process somewhere. Not sure.

[mikec_pt]

Not that PMS alone should consume much RAM but our suggested minimum configurations is 2GB.

Are you certain its PMS/other plex processes that are eating the RAM? What do you see in top ?

I’m on 1.19.3 now but don’t remember seeing such issues with 1.19.1 (on FreeBSD 12.1 jail at least).

Can you also give more info about the OS, what FreeBSD version? Host or Jail?

[GoodDogs]

Hi,

The reason why my current hunch is PMS or its other processes is that this machine runs ONLY PMS.

It could be rpc.statd as by music is backed by NFS. But pre 1.19 I had no memory pressure issues, even when the VM had 1.5G RAM. PMS is not running in a jail, in an ESXi VM.

# uname -a
FreeBSD plex-01 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 GENERIC  amd64

Here’s top:

# top -o size | head -20
last pid: 88418;  load averages:  2.16,  1.36,  1.15  up 5+05:46:25    18:15:43
30 processes:  2 running, 28 sleeping
CPU:  0.7% user,  0.3% nice,  0.5% system,  0.1% interrupt, 98.4% idle
Mem: 62M Active, 1310M Inact, 71M Laundry, 446M Wired, 200M Buf, 104M Free
Swap: 1024M Total, 436M Used, 588M Free, 42% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  794 plex         13  52   15   458M    68M piperd  12:58   0.00% Plex Script Host
52511 root          1  20    0   268M  2824K select   0:00   0.00% rpc.statd
  701 plex         19  52    0   157M    51M uwait   51:06   0.00% Plex Media Server
  799 plex         23  52    0   106M    20M usem     5:42   0.00% Plex DLNA Server
  800 plex         11  52    0    32M  5464K usem     0:14   0.00% Plex Tuner Service

I’m still getting my head around FreeBSD RAM allocation and I’m about to start a thread on FreeBSD Forums. The Free in top isn’t so bad, and really top is showing that 1.3GB is free (Inact). What appears to happen is a process uses up all (or almost all) RAM temporarily. And therefore swap kicks in (and occasionally runs out).

# swapinfo -h
Device          1K-blocks     Used    Avail Capacity
/dev/gpt/swapfs   1048576     436M     588M    43%

It’s a small data point but looking at the logs may indicate a regular run eating up the RAM:

# grep 'swap.*failed' /log/plex-01/messages | cut -c 1-13 | sort -u
2020-05-08T04
2020-05-10T04
2020-05-16T08

Maybe something that runs every 4 hours, although PMS’s scheduled tasks are configure to start at 2AM and complete by 8AM.

Just watching a library scan of my largest library (40K files) shows rpc.statd popping in and out of top, but never with more than 260MB of resident RAM (but always 260MB in Size).

[bjs59]

Just curious… What file system are you using, UFS or ZFS???

[GoodDogs]

PMS is running on a UFS partition. The media is mounted via NFS.

[bjs59]

Okay, just checking… On my PLEX server running 11.3 FreeBSD with ZFS, I had to limit the ARC or I would have problems with memory allocation and would bump into out of memory situations… Sometimes it seems ZFS has a hard time releasing memory quick enough for other applications to use…

Otherwise, I have not noticed any memory leak on my FreeBSD PLEX server…

[GoodDogs]

It does seem Plex (one of the processes) is requesting a lot of RAM, although it may not be a memory leak per se.

My machine has 2GB RAM and 1GB swap.

The graph shows the total of the Plex processes memory usage. The dotted lines coincide with swap failures and the Plex process(es) being terminated.

[GoodDogs]

So added 2GB of swap to the machine and I’ve yet to have Plex terminated due to swap usage for a week. Which is a good sign.

It does seem that every so often needs a log of RAM (1.5GB) over and above it’s average (272MB).

The red vertical lines are when Plex is killed, blue is when I added 2GB swap.

[GoodDogs]

I had another out-of-memory condition with Plex 1.19.5.3112.
So I upgraded the RAM to 4GB. I still see OOM from Plex:

Does anyone know if this is expected? All I have is 8 or so music libraries. The DB is 139MB.

[tentrey]

seems like this is still an issue. every morning I have to restart my plex server, sometimes I have to restart my entire freenas server

[GoodDogs]

I’m lucky enough to have some unused RAM in my ESXi host, so I bumped by Plex VM to 4GB. Which seems a lot for a music-only server, but it did stop the OOM crashes.
It seems Plex is often resource-hungry. My Plex data disk filled today - 40GB is not enough to hold the metadata for a mid-sized music library
The main culprits being:

# du -hcx -d 1 /1/plexdata/Plex Media Server
...
9.2G    Plug-in Support/Databases
5.9G    Cache/PhotoTranscoder
2.0G    Metadata/Artists
 11G    Metadata/Albums

Anyway that’s a bit off-topic. Good luck with your memory issues.

[yogi_en]

Hello,

Is there any update on this issue?. I am having the same issue, where Plex Media Server consumes around 5gb of RAM. This causes my NAS ( QNAP ) to hang and I will have to restart it most of the time. Happens every day morning. Currently I have stopped plex server and the issue disappeared now. Looks like a memory leak or some task is scheduled by plex in the night which is causing this. Any help is appreciated. Started happening recently only, around 2 to 3 weeks back.

[GoodDogs]

Well adding more memory (now 4GB) didn’t help:

Plex is still crashing out from out-of-memory conditions. And me with only music in my library…

[GoodDogs]

And again:

[mrchartcove]

Setup: VMware free ESXI 7.0U1, 32GB vm of FreeNAS 11.3-U5 running Plex Jail updated to 1.21.2.3918

I’ve been having this issue for some time now. Best I can tell it is related to the Plex maintenance runs of Plex Media Scanner. Each morning, FreeNAS was crashed/unresponsive with a screenful of “swap_pager_getswapspace(32): failed” messages. Logs showed high mem activity and swaps at 2am - same time as PLEX scheduled maintenance.

Logs point to Plex Media Scanner deep analysis etc. I set the “Generate intro video markers” to “never” and have not “crashed” since. (also added a freenas cron job to run page_in_swap.pl - look up in google - that supposedly reclaims swap space. Not sure if this helps, supposed to get an email when triggered and have not, but I mention for completeness)

I am still getting swap space errors although it seems that freeNAS can now handle things.

Jan 27 02:18:16 FREENAS kernel: pid 98304 (Plex Media Scanner), jid 1, uid 972, was killed: out of swap space

It would seem that there is either a memory leak or that the maintenance process is consuming/releasing much more memory than can be cleaned up by the freenas jail.

I’ve not really tried to dig into the command lines and have not been able to manually recreate things but thought I’d share in hopes of those more talented could investigate.