Slow library performance due to increasingly growing database

[sa2000]

Yes I can check the database for the extras problem -But best to also upload here debug server logs after a crash - in case it is a different issue

You may find products such as SQLiteStudio make it easy for you to check the database on a windows PC:

Shut down Plex Media Server
Copy db files out (if not shut down properly - copy all 3 files

com.plexapp.plugins.library.db
com.plexapp.plugins.library.db-wal
com.plexapp.plugins.library.db-shm

Add com.plexapp.plugins.library.db database to SQLite Studio
Select the db file
In Tools/SQL Editor - type in this query

select count(metadata_item_id),metadata_item_id from metadata_relations group by metadata_item_id order by count(metadata_item_id) desc limit 20

Run the query and see the first few lines for counts of extras

[sa2000]

The fix for the Extras issue and massive DBs is now out - in beta 1.25.4.5426
See release notice

Plex Media Server

Plex Media Server 1.25.4.5426 is now available to Plex Pass users in the Beta update channel. NEW: (NOTE) Database changes may cause longer than normal start-up times after initial installation of this update, especially on larger libraries. Please be patient and do not interrupt this process. (Library) Refresh movie metadata periodically during scheduled tasks (#13250) (Metadata) Add Bulgarian language option for modern movie and TV agents (#13263) (TLS) Server-side HTTPS connection support…

• (Metadata) Some users could end up with massive amounts of duplicate extras (#13241)

The immediate issue of crashes, memory allocation failures get resolved on first launch. The database will shrink slowly over a number of weeks. For example if there are a million extras on a movie - it would take about 11 week to purge all the orphaned database items after the initial deletion of the extras - the purge is up to 100K at a time when the database is optimized and the scheduled task runs this once a week. Of course users can run manual database optimizes which would reduce the period. Note that during an optimize the server is effectively locked out and it may initially take 20 minutes or more

Anyone with the early alpha build with the initial fix should update to this beta

Thanks to everyone for help in tracking down this bug with investigations and diagnostics from @shayanzadeh @Volts @jakehoms @spartan018 - just to mention a few

[dEEPZoNE]

Still getting the " 200ms to retry busy db " and the server stops.
Running unraid docker.
Have disabled extras, trailers aso on the server.
Got 42 000+ items in the movies library ( have split up the servers in 2 dockers. one for movies and one for series )

Running plex on a threadripper 16 core with 40 gigs of memory.
The database and metadata is located on 3x m.2 disks in raid 0.

[drzoidberg33]

Full server logs please.

[dEEPZoNE]

Plex Media Server Logs_2022-02-14_22-13-28.zip (4.8 MB)
Here you go

[dane22]

@dEEPZoNE

( have split up the servers in 2 dockers. one for movies and one for series )

Does that means two servers accessing the same database?

[dEEPZoNE]

No, it’s different databases. Totally seperated.

[drzoidberg33]

Can’t see anything wrong in those logs. Your issue probably happened earlier and the logs have already rolled over.

When the issue occurs make sure to grab the logs as soon as possible then provide those.

[dEEPZoNE]

Plex Media Server Logs_2022-02-17_17-03-48.zip (4.2 MB)
Here is the new logs right after the issue

[dEEPZoNE]

more logs :
Plex Media Server Logs_2022-02-20_16-31-56.zip (3.5 MB)

[Volts]

It looks like both of those times the server shuts down because it received a Signal 15, or a normal “please shutdown” message.

At least, I think so. Plex responds differently to signals on FreeBSD.

Is this a personal server, or on a VPS?

Responding partly just to make more noise on the thread for @drzoidberg33 to see. I’m curious what’s actually happening.

[dEEPZoNE]

It’s a personal server running unraid

[dEEPZoNE]

The signal 15 might be from when i press “restart” on the docker when it stops up ( when it get the sleep 200ms messages ) and i have to restart it

[Volts]

How long was it “stopped” before you forced the restart?

I see a few of these messages:

Feb 17, 2022 16:28:20.449 [0x14a882597b38] ERROR - Soci Exception handled: sqlite3_statement_backend::loadOne: database is locked

Those are bad.

There’s a million threads across the Internet about Unraid breaking SQLite.

unraid sqlite database is locked - Google Search

[Support] Linuxserver.io - Sonarr - Page 42 - Docker Containers - Unraid

Where’s the Plex metadata (especially including .../Plug-in Support/Databases/ stored?

[dEEPZoNE]

Not sure how long before i restarted it. A friend of mine notified me about it being offline.
changed the plex appdata folder to /mnt/cache/appdata/pledmediaplayer/ from /mnt/user/appdata/plexmediaserver/ now to see if this helps.
All plex files are on the cache drive from before ( 3x m.2 drives in raid 0 )

[HTPC_Lover]

@Volts

There’s a million threads across the Internet about Unraid breaking SQLite.

If using an old Unraid release please avoid using the FUSE “User Share” mount (/mnt/user/appdata/…). Use the corresponding “Disk Share” mount (e.g. /mnt/cache/appdata…, /mnt/diskx/appdata/…, or whatever) instead.

BTW, while the User Shares are great to combine user data folders on multiple disks, they suck for the system folders (e.g. Docker, VM, appdata, domain, …).

[dEEPZoNE]

Looks stable for now
Will keep you posted ! Thanks !