PMS on debian refuses to play media, unresponsive, slow most of the time

Server Version#: 1.14.5488
Player Version#: web/iPhone/AppleTV/PS4, does not matter

Have been running PMS on my debian sever for some time, but started to notice general sluggishness and crashes recently. Tried upgrading from 1.13 to 1.14, but no difference. Tried to optimize the library, but no difference. Clean bundles, no difference. Manually remove the
$PLEX_HOME/Library/Application Support/Plex Media Server/Plug-in Support/Caches, no difference.

The server usually allows me to log in and browse, but refuses to play anything.

Restarting the service sometime takes forever and requires service plexmediaserver force-stop.

Sorry about the somewhat verbose logs.

What can be wrong?

Plex Media Server Logs_2019-02-01_09-33-29.zip (3.5 MB)

Is this on Debian 7, 6, or older?

Also, for future reference, Please turn off VERBOSE logging until requested. Too much information is lost due to short log file retention.

It’s off now. Perhaps I can just “do more stuff” to fill it with non-verbose normal log stuff at the end.

You’re at the end of that version’s usefulness I’m afraid.

Debian 8 and above is what we support now (since Augst 2016).

It installed without complaints about any dependency, so I don’t see what could be incompatible. This started before I tried to upgrade PMS and has been working for quite some time before that.

What could be wrong?

I’m letting you know what is already in motion.

With regard to your machine, I would like you to reproduce the problem again (DEBUG logging only).

Also, how many items do you have indexed into PMS?
Is this on the native machine or in a container/VM ?

Runs natively.

About 1400 movies, few hundreds TV shows, tens of thousands of songs.

Saw a thread elsewhere that there might be a limit to how many items in a single folder/category. Could that be an issue here?

Mitigated the issue somewhat by crontabing a service plexmediaserver restart every night, but it’s just a band-aid.

Plex Media Server Logs_2019-02-04_09-35-14.zip (3.4 MB)

Fresh set of logs.

Check the health of the disk that the Plex database/install is on. Several messages regarding slow access.

WARN - Waited one whole second for a busy database

If all your listening is on the local network, I could be reading the logs incorrectly here but it looks like it is going through the relay.

Oh! It’s a raid 6 array, but I do have had several drives fail on me the last year (all installed at the same time so along with the Debian 6 they might be on the tail end of their lifespan). I did originally have the PMS database on the boot disk (SSD), but it outgrew it. Database is around 80 GB at the moment, which is unresonable by itself).

Perhaps take it down, run a e2fsck -f on the file system and force out any bad drive which might pop up?

Please quantify how many items indexed is “thousands” .

The easiest way to do so is

find /media-dir-1 /media-dir-2 /media-dir-3 -type f -print | wc -l

(listing each of your Top-level diretories in the find)

It will return a file count.

As supplemental information:

  1. Processor launch was Q1 2009. It’s 10 years old.
  2. Processor total passmarks - 4758

Great for its era but compared to current, it’s not much more than J-series NAS CPUs and only 1/2 the Passmarks of my 2012 i7-3740qm

https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5540+%40+2.53GHz&id=1245

Perhaps I should re-phase my question. PMS has been running fine up until now, which means responsive, working, could search, play music and so on. My 10 year old CPU had no problem in coping with streaming Full HD video to several clients, but now I can’t even listen to 128 kbps MP3. The CPU is not the issue here.

I did consider something similar but ended up using ExportTools plugin to count number of songs and movies. I’ll link to the thread here. Anyhow, find and wc -l revealed the following:
music 37815
movies 15479
TV 18091

e2fsck -f is running overnight. EDIT: Took a few minutes. No complaints.

Please confirm again the ls -la of the Databases directory?

total 848808
drwxr-xr-x 2 plex plex 4096 Feb 5 05:00 .
drwxr-xr-x 7 plex plex 4096 Aug 1 2018 …
-rw-r–r-- 1 plex plex 69632 Aug 2 2018 com.plexapp.dlna.db
-rw-r–r-- 1 plex plex 32768 Aug 2 2018 com.plexapp.dlna.db-shm
-rw-r–r-- 1 plex plex 292552 Aug 2 2018 com.plexapp.dlna.db-wal
-rw-r–r-- 1 plex plex 25103360 Feb 3 11:38 com.plexapp.plugins.library.blobs.db
-rw-r–r-- 1 plex plex 25103360 Jan 29 02:04 com.plexapp.plugins.library.blobs.db-2019-01-29
-rw-r–r-- 1 plex plex 25103360 Feb 1 10:36 com.plexapp.plugins.library.blobs.db-2019-02-01
-rw-r–r-- 1 plex plex 25103360 Feb 4 05:24 com.plexapp.plugins.library.blobs.db-2019-02-04
-rw-r–r-- 1 plex plex 32768 Feb 5 05:00 com.plexapp.plugins.library.blobs.db-shm
-rw-r–r-- 1 plex plex 0 Feb 5 05:00 com.plexapp.plugins.library.blobs.db-wal
-rw-r–r-- 1 plex plex 194862080 Feb 5 10:45 com.plexapp.plugins.library.db
-rw-r–r-- 1 plex plex 186520576 Jan 29 02:04 com.plexapp.plugins.library.db-2019-01-29
-rw-r–r-- 1 plex plex 190491648 Feb 1 10:36 com.plexapp.plugins.library.db-2019-02-01
-rw-r–r-- 1 plex plex 194845696 Feb 4 05:15 com.plexapp.plugins.library.db-2019-02-04
-rw-r–r-- 1 plex plex 32768 Feb 5 10:49 com.plexapp.plugins.library.db-shm
-rw-r–r-- 1 plex plex 1552120 Feb 5 10:50 com.plexapp.plugins.library.db-wal

Thank you.

Your actual database is 194.8 MB. Your total metadata, including all the chapter images, might indeed be 80GB. The part which matters most with respects to this issue is the physical DB (194 MB)

This is where I think Debian 6 is your root evil. Debian 6 support was offered by one of the Plex Ninjas, as a gift to the community, with permission from Plex. He has since moved on. There is no support for Debian 6. There is no support for Debian 7. Official support for Debian begins at 8 when it migrated from being init based to systemd based.

Feb 04, 2019 09:20:31.035 [0x7fe18d3ff700] WARN - Waited one whole second for a busy database.
Feb 04, 2019 09:20:31.166 [0x7fe1927fc700] WARN - Waited one whole second for a busy database.
Feb 04, 2019 09:20:32.047 [0x7fe18d3ff700] ERROR - Failed to begin transaction (../Library/MetadataCollection.cpp:173) (tries=3): Cannot begin transaction. database is locked
Feb 04, 2019 09:20:32.189 [0x7fe174bf5700] WARN - Waited one whole second for a busy database.
Feb 04, 2019 09:20:33.369 [0x7fe18d3ff700] WARN - Waited one whole second for a busy database.
Feb 04, 2019 09:20:33.409 [0x7fe1927fc700] WARN - Waited one whole second for a busy database.
Feb 04, 2019 09:20:34.386 [0x7fe18d3ff700] ERROR - Failed to begin transaction (../Library/MetadataCollection.cpp:173) (tries=4): Cannot begin transaction. database is locked
Feb 04, 2019 09:20:34.433 [0x7fe174bf5700] WARN - Waited one whole second for a busy database.

I say this because I have 66,367 items indexed.

[chuck@lizum Chicago.215]$ find /vie/*tv* /vie/*movie* /vie/animated /vie/season* /vie/*music* -type f -print | wc -l
66367
[chuck@lizum Chicago.216]$ 

and my database size is 137 MB

total 173M
drwxr-xr-x 2 admin administrators 4.0K 2019-02-04 02:02 ./
drwxr-xr-x 7 admin administrators 4.0K 2019-01-05 20:50 ../
-rw-r--r-- 1 admin administrators  68K 2019-02-02 03:07 com.plexapp.dlna.db
-rw-r--r-- 1 admin administrators  32K 2019-02-02 03:07 com.plexapp.dlna.db-shm
-rw-r--r-- 1 admin administrators 286K 2019-02-02 03:07 com.plexapp.dlna.db-wal
-rw-r--r-- 1 admin administrators 7.1M 2019-01-07 10:37 com.plexapp.plugins.library.blobs.db
-rw-r--r-- 1 admin administrators 7.1M 2019-01-20 02:01 com.plexapp.plugins.library.blobs.db-2019-01-20
-rw-r--r-- 1 admin administrators 7.1M 2019-01-23 02:01 com.plexapp.plugins.library.blobs.db-2019-01-23
-rw-r--r-- 1 admin administrators 7.1M 2019-01-29 02:01 com.plexapp.plugins.library.blobs.db-2019-01-29
-rw-r--r-- 1 admin administrators 7.1M 2019-02-04 02:02 com.plexapp.plugins.library.blobs.db-2019-02-04
-rw-r--r-- 1 admin administrators  32K 2019-02-02 03:07 com.plexapp.plugins.library.blobs.db-shm
-rw-r--r-- 1 admin administrators    0 2019-02-02 03:07 com.plexapp.plugins.library.blobs.db-wal
-rw-r--r-- 1 admin administrators 137M 2019-02-05 11:31 com.plexapp.plugins.library.db
-rw-r--r-- 1 admin administrators  32K 2019-02-05 12:44 com.plexapp.plugins.library.db-shm
-rw-r--r-- 1 admin administrators 1.1M 2019-02-05 12:44 com.plexapp.plugins.library.db-wal
[/share/CACHEDEV2_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/Plug-in Support/Databases] # 

Interesting, so the database handling works different depending on init/systemd?

No.

the flock() runtime library call behaves differently in different versions of Linux because of the version of the GNU C library (glibc) packaged with them.

PMS must minimally have glibc 2.17 to operate.

I cannot tell what version you have. The online resources end here.

What does dpkg-query -l | grep glibc return?

No glibc. I find gcc.

dpkg-query -l | grep gcc
ii gcc 4:4.9.2-2 amd64 GNU C compiler
ii gcc-4.8 4.8.4-1 amd64 GNU C compiler
ii gcc-4.8-base:amd64 4.8.4-1 amd64 GCC, the GNU Compiler Collection (base package)
ii gcc-4.9 4.9.2-10+deb8u1 amd64 GNU C compiler
ii gcc-4.9-base:amd64 4.9.2-10+deb8u1 amd64 GCC, the GNU Compiler Collection (base package)
ii lib32gcc1 1:4.9.2-10+deb8u1 amd64 GCC support library (32 bit Version)
ii libgcc-4.8-dev:amd64 4.8.4-1 amd64 GCC support library (development files)
ii libgcc-4.9-dev:amd64 4.9.2-10+deb8u1 amd64 GCC support library (development files)
ii libgcc1:amd64 1:4.9.2-10+deb8u1 amd64 GCC support library
ii linux-compiler-gcc-4.8-x86 3.16.43-2+deb8u5 amd64 Compiler for Linux on x86 (meta-package)

I will attempt to cross match that gcc version to glibc

This will take a while

To shortcut, check your actual files

-rwxr-xr-x. 1 root root  1901676 Aug 29 06:17 /lib/libc-2.26.so*

you want libc

I am also very concerned about this: Mixing 4.8 and 4.9 GCC… which apparently comes from Debian 8?

ii lib32gcc1 1:4.9.2-10+deb8u1 amd64 GCC support library (32 bit Version)
ii libgcc-4.8-dev:amd64 4.8.4-1 amd64 GCC support library (development files)
ii libgcc-4.9-dev:amd64 4.9.2-10+deb8u1 amd64 GCC support library (development files)
ii libgcc1:amd64 1:4.9.2-10+deb8u1 amd64 GCC support library

This is beyond my Linux knowledge but here goes:

/lib32/:
lrwxrwxrwx 1 root root 12 Jun 18 2017 libc.so.6 -> libc-2.19.so

I’ve crontab’ed a service plexmediaserver restart each night, so it works slightly better now, but plugins like ExportTools fail on a regular basis.