Library.db size more than doubled in latest version

Yes.

The bloat problem exists only with 1.41.7 beta builds.

2 Likes

Welp maybe I’m unlucky but been running PMS v1.41.8.9834 for a few weeks now and still have a database size at 58.8gb. No improvement since the update with the scheduled task weekly optimize.

I’ve already tried the manual DBrepair tool and manually run the scheduled task to no avail. I know the database was fine beforehand per the screenshot above.

Any ideas? Wanting to avoid a full re-install but seems that is where this is headed :frowning:

delete from statistics_bandwidth where account_id is NULL; has been running for more than 12 hours now. This file is around 73 GB, and there is a .db-shm file that is now about 380 MB and growing. This is still running but extremely slowly. What happens if I interrupt this operation? I made a backup copy of the original DB. Thanks!

@eknee1417

DBRepair will not reduce the bloat. The capability was removed when PMS 1.41.8 was released.

Manually initiating the Optimize Database (Settings → Troubleshooting) will not reduce the bloat. It works only when run as a scheduled task.

I do not know the order in which scheduled tasks execute. However, if the Scheduled Tasks window is too small, Plex may never get to the database optimization.

The Database Optimization scheduled task can be initiated immediately instead of waiting for it to run once per week.

To initiate using WebTools-NG, see this post for additional info.

To initiate via a POST or curl command, see this post for additional information.

Two important reminders:

a) The system must have enough free space, at least as much as the size of the bloated database. Relocate or delete the bloated backups if you need to free up working space.

b) Plex will be unresponsive during the cleaning, which will take several hours. If you share libraries with anyone, consider letting them know the server will be offline.

2 Likes

@eknee1417

To add to FordGuy,

If you are comfortable using the Linux command line then you can effect the repair on the command line with some database commands.

Please advise your skill level and if you wish to do this.

1 Like

@ChuckPa

Definitely comfortable at the linux command line and with all databases :slight_smile: Let me know what to execute.

For context I’m running plex in a docker on unraid and have around ~350gb+ of space on my cache to mess with after I delete the backups. Server specs are beefy as I use it for personal hobbies so its currently better than my gaming rig :slight_smile:

@eknee1417

You docker types !

:roll_eyes:

LOL LOL

I can give you the command sequence and (99% complete) commands.
You’ll likely need tweak the pathnames to suit your system.

While PMS now can do the reduction by itself, albeit more slowly because PMS is running, I’m about 90% of the way to having (yet again) DBDeflate …

I’ll finish and let folks use it until this issue is resolved if it helps and Plex Engineering doesn’t mind.

The prototype reduced a 32GB DB, on an ARMv8 DS418j (1GB ram) Synology - no SSDs, in 34 minutes.

The same operation on my Dragon Canyon NUC (i9-12900) took 33 seconds
:see_no_evil_monkey:
LOL

So ------

Which do you prefer?

  1. Wait and try my scripting
  2. The commands with instructions

@ChuckPa I am currently trying to debloat my 274GB plex database (also running in docker) which also didn’t get fixed after running optimize. If you can give the commands that would be helpful!

@ChuckPa I’m down for trying your scripts so option #1 please :slight_smile:

Folks,

Here is a script for you all.

You can run it outside a docker container if you give it the real paths.

PMS must be stopped to use this

You MUST CONFIGURE the PATH variables at the top

This script is for Linux. Mac / Win can be adapted from this.

#!/bin/bash

#  1.  Configure PSQL and DBDIR to match your system
#       PSQL - Path to "Plex SQLite"   (Do not use  SQLite3)
#       DBDIR - Directory containing the database files
#
#  2.  Run this as 'root' user (UID 0) or 'sudo' root.


# Where is Plex SQLite really stored (native or external to container)
PSQL="/usr/lib/plexmediaserver/Plex SQLite"

# Where are databases really stored (native or external to container)
DBDIR="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
CPPL="com.plexapp.plugins.library.db"

# Sanity checks
[ ! -d "$DBDIR" ] && echo "ERROR:  No such directory '$DBDIR'" && exit 1
[ ! -f "$PSQL" ] && echo "ERROR:  Cannot find 'Plex SQLite' at '$PSQL'" && exit 2
[ ! -w "$DBDIR/$CPPL" ] && echo "ERROR:  Cannot find  / no write permission for '$CPPL'" && exit 3

echo ==== Current listing
ls -lah

## Run in actual databases directory
cd "$DBDIR"

# Remove output if exists
rm -f "./new.com.plexapp.plugins.library.db"

echo ==== Deflating into new DB
"$PSQL" com.plexapp.plugins.library.db << EOT
  CREATE TABLE temp_bandwidth as select * from statistics_bandwidth where account_id not null;
  DROP TABLE statistics_bandwidth;
  ALTER TABLE temp_bandwidth RENAME to statistics_bandwidth;
  CREATE INDEX 'index_statistics_bandwidth_on_at' ON statistics_bandwidth ('at');
  CREATE INDEX 'index_statistics_bandwidth_on_account_id_and_timespan_and_at' ON 'statistics_bandwidth' ('account_id', 'timespan', 'at');
  VACUUM main into './new.com.plexapp.plugins.library.db';
EOT
Result=$?

# Make sure no errors and non-null DB
[ $Result -ne 0 ] && echo "ERROR: Error code $Result from deflate.  Seek assistance" && exit 4

# Confirm we have a DB
OK=1
[ ! -f "new.com.plexapp.plugins.library.db" ] && OK=0

# Confirm not zero
NewSize="$(stat -c '%s' "new.com.plexapp.plugins.library.db")"
[ "$NewSize" -le 300000 ] && OK=0

# Abort if not OK
[ $OK -ne 1 ] && echo "ERROR:  Something went wrong.  Exiting."  && exit 31

# Get the owning UID/GID before we proceed so we can restore
Owner="$(stat -c '%u:%g' $CPPL)"
Perms="$(stat -c '%a' $CPPL)"


echo ========== DEBUG ================
echo Owner: $Owner
echo Perms: $Perms
echo New: $NewSize

# Move existing DB's
echo "Moving existing DB to '-bloat'"
mv "$CPPL" com.plexapp.plugins.library.db-bloat
[ -e "$CPPL" ]     && mv "$CPPL" "$CPPL-bloated"

echo "Removing WAL and SHM if exist"
[ -e "$CPPL-wal" ] && rm -f "$CPPL-wal"
[ -e "$CPPL-shm" ] && rm -f "$CPPL-shm"

# Move new DB into position
echo "Making new DB active"
mv "new.com.plexapp.plugins.library.db"  "com.plexapp.plugins.library.db"

# Set owner and permissions
chown "$Owner" "$CPPL"
chmod "$Perms" "$CPPL"

echo New databases directory contents
ls -lah

echo Done.
exit 0

At the bottom is where the file moving is done.
You can insert a exit 0 before this to “dry run” it.

HOW THIS WORKS:

  1. It creates a new statistics_bandwidth table where the bloat is removed
  2. It then deletes the old table
  3. It renames the new (temp) to take its place
  4. It VACUUMs the DB into a new file (safeguard)
  5. When all looks OK, it renames the existing DB and moves the new into place.

Space required — the size of your DB before bloating.

5 Likes

@MHedgy @eknee1417

See above please

I would appreciate feedback about how long this takes on your systems.

Thank you so much, Chuck! I ran the version of this your posted 4 days ago and it worked well for me (once I manually modified the permissions afterwards). Looks like the latest version you have does all of this and is more robust.

My library.db file was 24GB. I lost the exact timestamp, but this took somewhere under 3 hours for me. I ran it within my plex docker container on my unraid server (after stopping the PMS process).

The unfortunate part for everyone is the SELECT operation with qualifier must walk through every record in the DB (of which there are hundreds of millions in this case) to determine which ones are valid. That unfortunately takes time.

I hope this script will help those who need to reduce the size now and are unable to wait for PMS to reduce during each nightly tasks.

1 Like

Ok, I suspect I have the same problem as described in the thread above, only I never subscribed to the beta PMS releases :face_with_raised_eyebrow:

I’m running PMS as a docker (plexinc/pms-docker:plexpass) on unRaid (6.12.15) on a ASRock Z370M-ITX/ac with 32GB RAM and a Intel Core i3-8100 CPU

The docker.img file was only 70GB (40GB in use) and for the last 4 weeks, on Friday night, the docker.img file would fill up to 100% and then goes back to normal.

As it is weekly and plex doesn’t backup it’s database anymore, I started looking into it and found this thread.

Normal backup size of the database is around 430MB. The last successful backup was on 2025-05-16 with 4820MB. The database of Plex at that moment was 58GB

I’m running the latest release (Plex 1.41.8.9834) and thus I increased my docker.img to 300 GB and started the database optimization process through the troubleshooting menu of Plex.

After reading this thread, I expected it to take a while for plex running through the database and after a while, I started to see an increase in the wal file. This increase kept going until it stopped increasing after 24 hours. the database is now 54GB and the wal file 56GB. I let my system be for an other 6 hours, but no further file edits where made so I force stopped Plex and restarted the docker.

During the optimization process, plex is unresponsive and apears to be offline. the log of unraid for the docker also does not appear to hold lots of information.

My question is the following: will plex manage to clean up the database when it will do it’s maintenance loop? And what will happen when the database cleanup is not complete when the maintenance time is up?

That is unknown to me. I’ve not seen what it does.
(Engineering is not around until Monday (US) to ask)

2 Likes

I was always sceptical that this was truly fixed hence I am still on the previous public. Plex employees have been playing extra coy on this one.

It is not.

Que? There are clear explanations in this very thread.
The scheduled task which would automatically reduce the bloat runs only once a week, so the issue is not magically gone immediately after installing 1.41.8.xxxx

This very thread also contains several different manual approaches for removing the bloat on the command line, plus now @ChuckPa 's shell script which does the same.
Particularly for those users who need to perform the de-bloating on a different machine (which may be faster or have more memory to store the database wal files while the operation is in progress) .

This is one hell of a long thread to wade through though, and for less technically competent people like me, the methods of repairing it are very confusing. I’m on Windows and have no idea how to adapt the script or what to do with it once adapted.

Will a later version resolve it and repair all the bloat? Or is everyone going to have to manually do the repairs? If so, I may be better off uninstalling Plex and reinstalling fresh, which will be a complete ballache as well.

No. PMS 1.41.8 will try and debloat the DB automatically during the scheduled server maintenance. This will happen once a week.

Manual processing is only necessary if you are low on storage space on the volume which has the plex data folder. Or if you want to perform the procedure on a higher speed device (e.g. on a desktop computer instead of a slow embedded system)

Mine hasn’t managed to do it itself yet, and when I’ve tried to set it running using the webtools-ng it just hangs for hours and never seems to complete, even when I left it overnight.

In the scheduled tasks settings there is a setting to set the time scheduled tasks should start and end by. What’s the best end setting to allow it time to finish?