Intro Detection: Temp Folder Won't Clear

Hi All,

I’ve noticed a “Detection” folder within my Plex Transcode directory that is absolutely eating up storage. I assume it’s the temporary files used for intro detection but it appears they’re never cleared away once detection is finished.

I originally had my transcode location set to RAM but noticed high usage from this folder so moved it to disk to see how big it would grow.
I moved it to disk Monday evening, it’s now Wednesday morning and the folder is 76GB, it’ll probably continue to grow as i add more media and it detects intros.

It’s full of files like this (Sorted by date modified so you can see the oldest file in there)

I thought at first maybe it was permissions stopping it from being able to remove the files but the permissions are the same for the “Sessions” folder which still grows & clears as i transcode when people watch content, so plex doesn’t appear to have any problem with clearing the data down for the general “Transcode” location, it just doesn’t seem to be doing it for the “Detection” folder.

I tried searching to see if anyone else had this problem but couldn’t find anything so I thought i’d post and ask. It seems like it’d be an obvious enough problem to notice, especially with those who transcode to RAM so I was surprised not to hear anyone else with the same issue.

If nobody else seems to have the problem and it’s likely to be something wrong with my setup I can attach logs. I just didn’t immediately because I’d accidentally left verbose logging on, only just unchecked it so now I need to wait for new content to come in with intros to detect.
(I assume debug logging can still be left on and won’t leave my token in the logs?)

Unraid Version#: Version: 6.8.3
Docker Image: Hotio/Plex
Server Version#: 1.19.4.2935
Player Version#: N/A

We should be deleting after each file - may be there is a lock giving rise to error on the delete

Would like logs captured after detection

(tokens are not logged in the Plex Media Server.log files even with verbose)

Thank you for getting back to me so quickly.

Attached latest copy of the logs (thanks for confirming tokens aren’t capture in the log)
Plex Media Server Logs_2020-06-24_14-58-35.zip (4.8 MB)

I imported a few new shows and also kicked off analysis on a few existing shows to ensure it ran through the intro detection process. Creating about 9GB of files in the last 3 hours.

Thank you

Can you copy out these db files and zip and attach

com.plexapp.plugins.library.blobs.db-wal
com.plexapp.plugins.library.blobs.db-shm
com.plexapp.plugins.library.blobs.db

They are in this directory
/config/app/Plex Media Server/Plug-in Support/Databases/

Please also check what date/time were last backup files for these databases

com.plexapp.plugins.library.db
com.plexapp.plugins.library.blobs.db

What date was when there was a backup of both databases

Auto backups happen every 3 days

Couldn’t upload those files to here, i assume the 800MB is probably well above the upload limit, so hopefully this firefox send link works
https://send.firefox.com/download/439c07cdfe1aca3f/#CA8emVCCykq6PAz99JFRPQ

Checking the database backups, looks like no backups available from Plex. Turns out i had my backup path set for an old container image layout, so the path doesn’t exist and I assume the files couldn’t be backed up because of it.
I should have at least a week of daily manual backups though if they’re needed?

On this note of database, I did have some database corruption a week or two ago, i followed the guide to repair it and all seemed to work fine at the time. Thought i’d mention it as I’m assuming something might be up with the database if you’re looking that way, rather mention it and not be relevant then not mention it and possibly waste time.

was that the blobs database or the other one?

The logs show corruptions

Could you leave the uploaded zip of the db there as I will be sharing a link with the development team

I ran the fix steps in the guide against com.plexapp.plugins.library.db so i assume it was against that and not the blobs databases.

Happy to leave the zip uploaded, i think it’s set to 100 downloads/7 days as a max in firefox send.

Thanks - can you shut down Plex media server and then do a repair on the blobs database and also do a pragma integrity check on the main DB

These are the exact commands i ran, line by line:

Repair Blobs:

cp com.plexapp.plugins.library.blobs.db com.plexapp.plugins.library.blobs.db.original
sqlite3 com.plexapp.plugins.library.blobs.db "DROP index 'index_title_sort_naturalsort'"
sqlite3 com.plexapp.plugins.library.blobs.db "DELETE from schema_migrations where version='20180501000000'"
sqlite3 com.plexapp.plugins.library.blobs.db .dump > blobs.dump.sql
rm com.plexapp.plugins.library.blobs.db
sqlite3 com.plexapp.plugins.library.blobs.db < blobs.dump.sql

Check main database for corruption:

cp com.plexapp.plugins.library.db com.plexapp.plugins.library.db.original
sqlite3 com.plexapp.plugins.library.db "DROP index 'index_title_sort_naturalsort'"
sqlite3 com.plexapp.plugins.library.db "DELETE from schema_migrations where version='20180501000000'"
sqlite3 com.plexapp.plugins.library.db "PRAGMA integrity_check"

Both ran with no issue, no output from the first as expected and just the ok from the second.

I assume start plex up now and check that Detection folder? Or analyse some more shows and send logs?

You can purge the old the temp files
/mnt/user/media/downloads/plexcache/Transcode/Detection/xxx

and launch Plex Media Server and do some intro detection and then get debug logs to see if we have any issues and also check if any new temp files in that path get left behind

Okay so unfortunately it seems like when i run the final command of
sqlite3 com.plexapp.plugins.library.blobs.db < blobs.dump.sql

The file it outputs is 0 bytes. The blobs.dump.sql file it’s created from shows 1.7GB though so unsure why it’s doing that? It’s causing Plex to not start so i have to revert back to the .original file

P.s thank you for all your help so far

Could you try to launch without a blobs database and let me have logs if Plex Media Server does not start

The dump file might have an action need the end to indicate if there was an error in it

Removed the blobs database and Plex started up.
The detection temp files are now being cleared immediately and the detecting intros task actually seems to take some time now whereas recently it’s been zipping through them, fool me for assuming it was just a fast process.

Latest logs from this morning after running a few intro detections.
Plex Media Server Logs_2020-06-25_08-55-18.zip (5.7 MB)

Should i expect any data to be missing which needs to be recreated due to deleting the com.plexapp.plugins.library.blobs.db?
Tried to search to see what that database actually does but couldn’t find anything.

Let me know if you also wanted the blobs.dump.sql that didn’t seem to work. Thought i’d double check before uploading a 1.7GB file.
I did have a quick search for “error” in it though and found these 3 lines that were spread out:

CREATE TABLE IF NOT EXISTS 'media_grabs' ('id' INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, 'uuid' varchar(255), 'status' integer, 'error' integer, 'metadata_item_id' integer, 'media_subscription_id' integer, 'extra_data' varchar(255), 'created_at' datetime, 'updated_at' datetime);
/****** CORRUPTION ERROR *******/
ROLLBACK; -- due to errors

Good news.

My understanding was that the intro detection would be re-run. The blobs database is also used for loudness analysis for music

Awesome, will kick a full library intro detection to anything missing and will also kick off a music library analysis too.

Thank you for your help and helping me get this fixed :slight_smile:

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.