Is there any way for Plex to create a status page on their internal Agents? I’ve restarted Plex, both service and server itself, which leads me to believe there is something wrong again on Plex HQ side with the matching algorithms… I’ve had 3 days now with neither Plex Movie or The Movie DB agent being able to get any information whatsoever…
Just now I was able to manually fix a match with The Movie DB but it’s a serious pain in the butt when scanning in bulk and not knowing when the agents decide to work or not…
And no, it’s not my file names – small extract of my naming:
./a/Arlington Road (1999)/Arlington Road (1999).mkv
./a/Any Given Sunday (1999)/Any Given Sunday (1999).mkv
./n/Notting Hill (1999)/Notting Hill (1999).mkv
./d/Deep Blue Sea (1999)/Deep Blue Sea (1999).mkv
./o/October Sky (1999)/October Sky (1999).mkv
./r/Ravenous (1999)/Ravenous (1999).mkv
./r/Runaway Bride (1999)/Runaway Bride (1999).mkv
./v/Varsity Blues (1999)/Varsity Blues (1999).mkv
./v/Virus (1999)/Virus (1999).mkv
./s/Stir of Echoes (1999)/Stir of Echoes (1999).mkv
./p/Play It to the Bone (1999)/Play It to the Bone (1999).mkv
./p/Payback (1999)/Payback (1999).mkv
./t/Two Hands (1999)/Two Hands (1999).mkv
./t/True Crime (1999)/True Crime (1999).mkv
./i/The Insider (1999)/The Insider (1999).mkv
./i/The Insider (1999)/Theatrical Trailer-trailer.mkv
./i/The Iron Giant (1999)/The Iron Giant (1999).mkv
./i/The Iron Giant (1999)/Theatrical Trailer-trailer.mkv
./i/Inferno (1999)/Inferno (1999).mkv
./g/Go (1999)/Go (1999).mkv
./e/eXistenZ (1999)/eXistenZ (1999).mkv
./e/Eyes Wide Shut (1999)/Eyes Wide Shut (1999).mkv
The lack of feedback is a serious concern lately… 
You’re the third person to tell me you’re having these problems.
After you go to one of the files with trouble and try to get the metadata again, after it fails, Please go immediately to Settings - Server - Help - Download Logs. It will present you with a ZIP file you can upload here.
We’ll look at it and see what it’s complaining about but I suspect you and the other user might have found a regression we need to fix (again) as well as alert QA to change how they test for it.
Hi @ChuckPA log files sent via PM, 2 while Plex is spinning its wheels and a 3rd after the fail…
I got your log files and your database also needs Optimizing .
May 04, 2017 20:29:41.658 [0x7fe1ab7fb700] DEBUG - It took 60.000000 ms to retrieve 27 items.
May 04, 2017 20:29:41.658 [0x7fe1dffff700] DEBUG - It took 60.000000 ms to retrieve 27 items.
May 04, 2017 20:29:41.787 [0x7fe1a7bf5700] WARN - SLOW QUERY: It took 1100.000000 ms to retrieve 65 items.
For reasons unknown, which I’m gathering data on, sometimes one Optimization pass doesn’t get all of it.
I would like you to see what impact this has:
- Optimize the databse
- Restart PMS
- Optimize the database again
- Restart again
- See if normal operation is restored and for how long.
I am trying that now… Is there a linux shell command to optimize the db perhaps?
do you know how to dump and reload a SQLite3 database?
never stop learning… point me to the info?
sudo sh
systemctl stop plexmediaserver
cd "/var/lib/plexmediaserver/library/Application Support/Plex Media Server/Plug-in Support/Databases"
mv com.plexapp.plugins.library.db com.plexapp.plugins.library.prev.db
rm com.plexapp.plugins.library.db-???
echo .dump | sqlite3 com.plexapp.plugins.library.prev.db > dbdump.sql
sqlite3 com.plexapp.plugins.library.db < dbdump.sql
chown plex:plex com.plexapp.plugins.library.db
systemctl start plexmediaserver
ok, so that sorted the crap out of it… refreshing metadata on unmatched files ticked over like clockwork, it is currently running a full library scan on my main library of currently 5217 movies… thank you @ChuckPA !
which begs the question… at what level of interactions per day/week does Plex start to struggle to keep up? I’ve been nailing the server the last 2 weeks or so, fixing naming of movies at a steady rate of about 50/70 a day as well as adding 10 or so a day… Plex has been set to do auto maintenance, me and my Plex-dance is on Tango level 7 while I’m working the folders, and for some reason it seems it is still not enough to be able to keep up with the demand…
granted my renaming is crazy level, currently standing at 73.12% completeness until Plex should have the cleanest library this side of PlexHQ…
any ideas and/or suggestions? why does it seem the db got corrupted and only affected scanning and metadata?
what I gave you above only takes care of the database. It does not address the metadata out in the bundles.
This is still handled by Updating the library. Emptying the trash after every scan is a good idea as is a routine ‘clean bundles’ , both of which can be handled by the butler during nightly maintenance. The only thing you must adjust, based on library size, is how much time you give it to complete your tasks.
You can take some optimizations in Settings - Server - Library and enable ‘Partial scan’. This allows it to skip over what hasn’t changed.
If you’re adding ~50 pieces of media a day (which is a lot), I’d have it run nightly (using my Synology NAS as the baseline CPU where PMS runs). You don’t want the DB to become ‘sluggish’ on such a small processor.
With all your ‘crazy renaming’ going on, I would complete each library and, before moving on to the next library , optimize the DB. What has happened in the DB is the old records were deleted, new ones added.
The optimal sequence would be:
- Rename the media (e.g. FileBot)
- Update the Library
- Empty the trash
- Clean the Bundles
- Optimize the DB
- Advance to the next library.
- When done with all libraries, if you feel any sluggishness, do the ‘PMS offline optimization’ I showed above.
As a general rule of thumb, I keep an eye on the size the the DB itself and the -wal and -shm files. Those two are its cache. When those are larger than the DB itself, it’s time to do the big compaction.
I hope this gives guidance / makes sense?
I almost run your sequence to the T… Only thing is I only have 2 libraries in which the renaming is happening… One is my SORTED folder containing those movies which have been renamed properly with the extras files etc, and the other is my main folder which includes the SORTED folder and UNSORTED folders in the same library…
After the initial first scan it looked all good, and then scanned newest 52 renames, and it looks like it fails with those again… 0 metadata and 0 matches… shipping newest logs to your inbox…
The command sequence I have above (the direct sqlite3 command block) makes no internal (data) changes to the DB and is the beauty of it.
I will look at the logs and see what’s going on.
Quick question: Is there any way you can stop the active streaming while doing this?
ChuckPA,
As per your PM, is there any better way to do this other than dumping complete library folder and rebuilding?
I have only about 10 libraries set in Plex, but those 10 libraries cover something to the effect of 37TB right now… not ideal for a quick pump and dump and rebuild, especially if not all those are locally attached to the server itself…
I imported a quick 12 new movies this afternoon, but instead of moving files to my cloud storage and then scanning, i moved it into local unionfs and then scanned, and both sorted/main libraries picked up 0 issues and metadata was loaded for some before the scan was even complete…
now i know Plex is not responsible for offsite storage connectivity blah blah blah, but this has only been an issue it seems since 1.6.xxx versions, and nothing else in my setup has changed…
when you posted the timeout logs, can you indicate what the timeout is for? for plex getting info from their own servers to match file names, or for plex not being able to access what it thinks are local files…
A point you made earlier ... scanned newest 52 renames..., Are these renaming after PMS has scanned them?
If so, did you remember to “Plex Dance” them out and back in so it picks up the name correctly the first time? If you didn’t, PMS is still hanging on to the incorrect information in the DB. It will associate the new, correct, name with the previous ‘unmatched bad name’ and not list it.
If that’s the case (it HAS to be simple like this), are you using a tool like FileBot or TheRenamer to ‘rename and move’ simultaneously?
When I process new video, however I rip it, I run it through FileBot. My movie template for FileBot puts it in the directory structure how I want it. There’s no reason you can’t do the same. Yes, it can be automated too 
My FileBot template for movies is as simple as /syno/movies/{n} ({y})/{n} ({y}) . I hand edit the extras (adding the -tag as needed and moving in manually.
My template for Series, which Sonarr also uses, /syno/tv/{n}/Season {s.pad(2)}/{n}.{s00e00}.{t}
macro names:
n = name
y = year
s = season
t = episode title
.pad(2) = directive to pad the season number to 2 digits (leading zero fill)
A possibility?
those 52, were taken from main/hdd1/movie.name.2048.bluray.720p/movie.name.2048.bluray.720p.mkv -> moved to completely seperate sorting folder, renamed to Movie Name (2048)/Movie Name (2048).mkv… then movied to sorted/2048/m/Movie Name (2048)/Movie Name (2048).mkv…
after that scan library, empty trash (to get links to extras removed)…
normally all plex does is pick up the new name, shows the extras as unavailable (gone after trash empty) and everything is working as it should…
whether i use filebot or not, my method for updating which has worked for the previous 3200 movies, will probably not change… yes filebot will probably save me 10 seconds or so, but i have great comfort in my manual naming of folders and files in the exact place i want it… and if you are renaming extras anyway, you might as well do the movie yourself as well…
As for the ‘extras’, I have a secret script for that haha (When ready for prime time, I’ll share)
scanning and emptying the trash isn’t enough to clear it out completely. You have to ‘Clean Bundles’. Only when the bundle files are gone does PMS remove the record from the database. Until that point, it’s progressively removing it from the different tables.
- Update — removes it from the section table and moves to Trash
- Empty Trash - does just that…
- Clean Bundles - removes the unused “Media Item” entries which still exist. These include the posters, etc and anything else PMS knows including the previous file’s fingerprint from the DB. ← finger prints are important… It’s part of duplicate detection.
Can you tell me what is the timeout setting for Plex scanning a folder/file before it decides to give up? Local files are scanning without issue whatsoever… not sure why if plex can scan the file and see the name, why it would then timeout further unless it is trying to do a video trace/signal compare with its own servers to try and find the movie details…
is there a curl command or something that i can run to try and figure out why plex times out if files arent local even though it obviously picked up the name correctly before scanning for metadata
also, if it fingerprints a file in a certain location, why exactly can it not just retrieve that information when the folder/file is included in a second/third library?
