I noticed that the library scan time is slower with the new Plex Movie agent compared to the legacy when the library has duplicated items (remux and non-remux).
For example, my Anime Movies library has about 172 items, 28 of them are duplicates (remux and non-remux), with the legacy agent, it takes about 2-3 seconds to complete the scan. Now with the new Plex Movie Agent, it takes about 35 seconds to complete the scan.
Another example is with my 4K Movies library (1187 items, 776 duplicates as in remux and non-remux), with the legacy agent it takes less than a minute to complete the scan. With the new agent it takes about 35 mins to complete the scan.
I also tested this with a fresh library using the new movie agent, same result. Scan times are definitely slower than the legacy agent. But if I make a new library without adding the duplicated version (remux only or non-remux only), scan time will be as quick as the legacy one.
Can confirm, incredibly slow on Linux for me as well. Scan of a library with old agent takes ~30 min. New Plex Movie agent takes ~24 hours. It seems to run through each folder as opposed to the old one which only looks through the changes or something.
Did you upgrade the library or are you using the new agent on the old library?
Can’t speak for OP, but I Upgraded my old library, then refreshed metadata so everything would get new IDs. I just setup a new library and on the same old media, and I will let you know how fast it does on subsequent scans once the initial scan is complete.
I will caveat in that I do have perfect naming and structure
I also have perfect naming structure, imdb ids and everything.
Also check your “Settings - Server - Library”. Make sure they didn’t revert to full scan in the upgrade.
I’m not seeing any such setting. Are you talking about:
Scan my library automatically: Unchecked Run a partial scan when changes are detected: Checked Scan my library periodically: Unchecked
I run all my scans manually from the Web UI. Adding just a few movies takes an incredibly long time since it starts at Z > A, and scans every single movie folder (each of my movie is in its own Movie (Year) folder to play better with Radarr).
This is interesting. I am assuming your “4K New” library was already fully scanned in and there was nothing changed when you performed the scan from the log file?
I can see in the log it saying File xyz didn't exist, can't skip which means it would have to be processed again however this shouldn’t happen if it’s already been scanned into the database.
Your folder set up may be the clue, I’ll try and reproduce the issue using a similar folder structure and see if I find something. Thanks for providing such beautiful logs
Yes, the “4K New” library was already fully scanned in, nothing was changed. It is just the subsequent scans that are really slow.
I did a couple of tests the other day (don’t have a log for those as I already removed the libraries), I made 2 new Anime Movie (less items to scan) libraries using the new agent.
One library had non-remux movies and the other one had remux movies, the initial and subsequent scans were fast but when I add the remux folder to the non-remux library or the non-remux folder to the remux library, the subsequent scans for those libraries will be really slow compared to the legacy one even after everything is fully scanned in.
After digging a bit I saw that this was an issue we already have on our radar but it’s not an easy fix and stems from a decision that was made very long ago that is now causing the problem you’re seeing.
This will only affect you when you have more than one movie folder added to your library that are in different locations but share the exact same name, e.g.
/Movies/A Movie (2020)/
/Movies-4K/A Movie (2020)/
Everytime we scan this library we’ll be forced to process the files in one of those folders again even if it’s already been scanned in.
There are a few ways to work around this problem for now:
1. Give the folders unique names, e.g:
/Movies/A Movie (2020)/
/Movies-4K/A Movie (2020) [4K]/
2. Put all media versions in the same folder, e.g.
/Movies/A Movie (2020)/A Movie (2020).mkv
/Movies/A Movie (2020)/A Movie (2020) [4K].mkv
3. Use separate libraries. This might be a solution for some people but not for others.
This doesn’t affect the old agents anywhere near as much due to the way the responses from the Python agents were cached.
I have a solution in mind for the new agent too, I just have to implement and test it so not sure when that will land in a release.
I’m having the same issue. I had same movie spread across multiple folders/locations, and new agent just takes forever to do re-scans. It’s been so bad i just did my whole plex install again and went back with the old agent. I did not have this issue before.
@drzoidberg33 do you have a timeline for when a potential solution could be implemented.
I too am having this issue, really disappointing that this was released without a caveat for users with large mixed libraries if it was a known issue. Now I need to rebuild my library again?
@drzoldberg33 I too would like a timeline for a solution to this issue.
I do have separate libraries, with no overlapping locations like you mentioned.
My Movies - HD library:
/Movies/A/A Movie (2020)/
/Movies/B/B Movie (2020)/
My Movies - UHD Library:
/Movies-4K/A Movie (2020)/
/Movies-4K/B Movie (2020)/
Note the difference in file path. HD is alphabetized and 4k is NOT. And they are in separate libraries. So it’s definitely not a solution…
This will only affect you when you have more than one movie folder added to your library that are in different locations but share the exact same name, e.g.
Use separate libraries. This might be a solution for some people but not for others.
Can I ask why Plex would be looking at folders in a completely different location and library when a scan no where near it was triggered? Either way, I am VERY disappointed, my new movies take several hours to add.
For the past month, I’ve had to learn command lines and issue commands through the terminal just to be able to watch a new movie i added.
I had the same issue. I fixed it by renaming all folders in the 4k library by adding [4K] at the end. Scan speeds returned to normal after an initial scan.
No, it’s not super high priority right now and is easy to work around for the few users this specific issue affects. I’ll get to this in the new year at some point.
Just upgraded to the new movie agent and my library now takes hours to scan!
Frustrating that the solution/recommendation is to create separate libraries - considering the changes being made to Recently Added etc are pushing users towards aggregated libraries.
The other two solutions were to give the folders unique names, or to move all versions within the same folder. You’ve highlighted only one of three potential workarounds for this issue.
Neither of those are listed in the Plex Naming Convention recommendations though?
Will adding “4K” to 4K folders break Plex’s ability to place both standard and 4K files under the same library listing and auto select the appropriate file based on. Will having 4K in the folder title have any impact on file matching?
And if I have to change my entire directory structure , does that mean I have to delete the old directory references in each library and re-add the new ones (with 4K added)?
No, it won’t break anything or have any impact on the file matching, if done as shown in the example. You should first add the new directory structures, scan, and then once it’s scanned all the new directories, you can remove the old ones, as described in https://support.plex.tv/articles/201154537-move-media-content-to-a-new-location/
