Hi,
I am running plex for me and my family on a windows 2016 Server and I have no Problem with the functionality itself, but it is a crazy data hog:
I have a few thousand audio files, viedo files and photos. Plex is hardly used, but here and then somebody streams something. The media amount is hardly growing: every few weeks some few videos or audio files are added.
The first Problem is, that I am missing a oppurtunity to define the place of teh data directory, which is in my case C:\Users\username\AppData\Local\Plex Media Server. On other drives I have terabytes of free space, but since this is a virtual machine, I try to keep the system disk small.
The second problem is, that it is again at roundabout 20GB, after I deleted everything in the summer and set it up new. The biggest Folders are Metadata (6,5GB) and Plug-in Support (6,5GB), Both dbs com.plexapp.plugins.library.db and com.plexapp.plugins.library.db-wal are around 3,2GB.
How can this grow? A few hours after setting up, all metadata is loaded and the folder had maybe 200MB. A few moths later, after no big changes to the Media and maybe using it once a week its already exploded again to 20GB.
I know about the path I can set for reencodes and database backups, which I used, but I am seeing, only the blob database is backuped into this folder. There are no backups from the other databases, not in the external nor in the original folder.
From my point of view plex is missing a cleanup strategy for all the data it is hording.
Are there any settings that are relevant to both of my problems?
Hi,
thanks for all the information. I decided to try it one more time with plex: I had no interest in preserving all the metadata and started with a clean slate: Deinstalled plex, deleted all the plex config and data folders and made a fresh install, added all my libraries and waited after the libraries were build completely. Here is the data occupation at this time:
whole plex folder:7,5GB
just Metadata 6,5GB
main db 87 MB,
blobs db 3MB
From my moint of view, this is a lot of metadata, but ok.
My aim is, keeping it in that size.
Now three weeks later: Nobody used it during that time, the underlying media data did not change much in that time. It should be in the same ballpark like after installation:
whole plex folder:12,2GB
Cache 2,7GB
Media 1,1GB
Metadata 7GB
PluginSupport 1,25GB
main db 570MB
blobs 80MB
What is happening? For an unused system this is crazy. these 12GB are just consumed for organizing itself, while it has nothing to do.
During nightly server maintenance, additonal data are generated. If the collection is big, and/or the server is weak, it can take weeks to create them all.
These are not within the Plex data folder, immediatley after the media are added.
Among these generated data are:
in-depth bandwidth analysis of every video, audio and subtitle stream
preview snaphots/thumbnail pictures of each âchapterâ marker of a video
preview thumbnail pictures with 1 picture every 2 seconds of any video (optional per library)
progression of loudness of any audio track inside of a âmusicâ-type library, with 5 samples per second
more data is being downloaded to power the EPG, if you have set up the Live TV & DVR feature. This includes small pictures for almost every show thatâs programmed â on any station in the lineup.
supplimentary metadata in music libraries, like album reviews, artist biographies, concert dates, related artists & albums are going to be refreshed at a regular basis
the server will cache any downscaled version of a poster, album art, or background (fanart) it had to create to serve the needs of the various client types. The cache folder will therefore get bigger if you (and your shared users) are using a bigger variety of different Plex client devices and are doing a lot of library browsing. (This cache folder is essentially what is getting purged if you activate the âRemove old cache files every weekâ butler job.)
When creating the libraries, I unselected all the âpreviewâ, âthumbnailâ etc functions. Just the posters for the movies and tv shows. And what is happening with the plex databases? The main db is seven times bigger.
I will wait for a few more weeks and check, if the folder is expanding without limitsâŠ
It is now 6 weeks later, and the following happened: Overall most of the plex subfolder sizes changed not much, some even got smaller, BUT the plex library has grown to around 2,5GB. Keep in mind, this was a fresh install two months ago, on a server with no downtime, restarts, etc.
I tried the debug logging like described for 5 minutes after start, but I saw nothing horrible, no corupted or malformed database. I opened the database with sqlite browser, but I really donât know, which table could be problematic, so I tried to use sqlite3_analyzer on the database and fould out that 95,3% of the data was sitting in STATISTICS_MEDIA with 66Mio entries. Now I looked into that data and made you a screenshot from the middle of the table, but the whole table looks like the screenshot: accountId always null, and look at the date: there are also dates from current times, but I installed the server in october 2020 and the dates are mostly 2020-01-01.
There is no need to dump the whole table. The offending rows can be cleanly identified and selectively removed. â As long as one can still work with the database and itâs not damaged. DELETE FROM STATISTICS_MEDIA WHERE account_id IS NULL;
Please see Plex Database sqlite file has more than doubled in 10 days for the complete procedure.
How big is the file when you compress it with ZIP or RAR (before removing the NULL rows)? Just in case, the developer likes to take a look.
This solution would only be sufficient, if the bug had been fixed. It is no solution to manually stop plex, open the database with an external tool, run sql statements and then compress the file, and non the less do the procedure once a month. This should have been fixed now. Why is this still open, since it was communicated at least half a year ago?
One goal is to determine whether the latest server update brought improvement.
Another is to give you a workaround to keep your server running.
Not all users are affected by that issue. The question is what is special about either your server or your network.
Do you also have DLNA activated?
Iâm the person that started the thread âPlex DB doubled in 10 daysâ. I have been watching this issue and it hasnât been fixed - I havenât tested it against the latest release yet.
There are some workarounds which I havenât chosen to implement because I want to keep testing it and see if it actually gets fixed. But there was a procedure done by one of the posters in that thread that stopped the problem for him - see this post the third paragraph.
I believe this doesnât happen unless you have clients that arenât signed into the plex.tv servers. Only my clients not signed in are causing the balooning of entries in statistics_media. If you just want it to stop try these things:
Follow âphilipswâsâ suggestions in the post I mention above
Make sure all your clients are signed in
You could try disabling your DLNA server if you arenât using, some have suggested it can exacerbate these problems
I plan to every 4-6 weeks clean up my database and upgrade to the latest, then watch it again to see if it is continuing. So on purpose I continue to do the things I know cause the problem so I can be the âcanary in the coal mineâ on this problem. Otherwise Iâd just give up and sign all my clients in and do the stuff philipsw suggested. And believe me, Iâve thought about it. But it would be really good to see this bug fixed, it is a NASTY one for those it affects!
We all know, that the issue is connected to not logged in devices, and some people already posted which devices in their households are responsible. In my case it is an Amazon FireTV. Congrats to the people without such devices, but it does not help us. I am not willing to log into devices, I will never use with plex just to mitigate some bug, but I am willing to delete a table, when plex ist not willing to fix the bug. The bugfix is supereasy: check what your writing into that table and ignore not logged in devices.
