Music scanner scans entire music collection when new content added

I found only one set here Music scan still deadlocks which I responded to. Could you give me the link for the other and I will check it

Also having plex constantly scan my entire music library over and over again without end.

I have 3 music libraries as well as some TV and Movie libraries. What appears to happen is a large number of scans for the music libraries get queued up for whatever reason, and each scan appears to be for the ENTIRE library. So if I try to add some TV shows, and start a TV show scan for instance, I’ll either have to wait for all these queued music scans to finish, or I’ll have to cancel them all.

This may be what the other user above was seeing about not being able to cancel scans. Since I have 3 libraries, the behavior is more obvious, but it’s definitely a high number of scans getting queued. If I cancel a scan, the next scan in the queue starts up instantly, and sometimes its the same library, and sometimes it’s one of the other 2. Then if I cancel that one, it could go back to the first again, or scan the second library again, or switch to the 3rd. There doesn’t really seem to be much of a pattern as to which order they get queued in. Sometimes it does have the same library queued multiple times in a row, which could feel like an inability to quit the scan if you only had one music library.

After finally cancelling all the scans (I’ve had over a dozen queued quite often), then whatever scan I tried to trigger myself will start (e.g. the TV library).

The only problem area I have found and I am looking into is that at the end of a scan after adding some music, a metadata refresh is instigated for artists that have some unmatched tracks. The problem area being for artists such as ā€œVarious Artistsā€ where there may be thousands of tracks involved and this may take an hour or so

I have not seen evidence of scans being scheduled repeatedly - that would only happen when auto scans are enabled and changes are continuously being made to the media directories

@sa2000 so to clarify as discussed in my case if I add 2 new albums, the scanner runs twice taking a couple of hours trying to match the various artists and excessive scanning for lyricfind as well. I hope this gets resolved . Thanks.

1 Like

maybe my issue is different then. I do have auto scan on, but I’m certainly not adding things throughout the whole day. I drag 6-10 albums into the folder in the morning sometimes and it’s still scanning over and over again well past midnight quite often. I find myself canceling a dozen queued scans before I can initiate a scan of my own on say a TV library. And each of those scans scans the WHOLE music library even though I have the setting to only scan changes on.

1 Like

I’ll chime in also on this topic. My music library scan (PS 1.18.7.2457 on Mac OS 10.14.6) also takes several hours since around a month or two ago, it now takes about 12 hours. And the status icon shows it scanning quickly through most artists until it has run for some hours when it seems stuck on Unknown Artist for a long time. After the scan finishes viewing Alerts at this point shows it continuously matching Various Artists albums and updating metadata. So I have disabled my regular daily scan which worked for years without issue (and only took less than one hour). I scan manually now and when the scan completes I have to restart Plex Server in order to stop the continuous metadata refreshing.

I have about 8,000 Various Artists albums. Not sure why but when I view Artists there are three Various Artists groups: one has most of my VA albums, one has around 300 albums, and one has only one album. All three are spelled the same in iTunes and in Plex.

I do have several tracks (1753) with blank album artist which are ripped from singles (45s). All of these have correct tags for artist, track, etc. except for Album Artist.

I have, reluctantly, deleted my music library. I’d urge a fix to this so that I can readd it, but for now, I’ve just found the trade-off isn’t worth it. Fortunately I do have music services (Youtube music and GPM), but I like keeping things aggregated in Plex.

I have been looking at other evidence from another user and again it was tracks - some relating to Various Artists and some not - that were not marked as refreshed and ending up being refreshed repeatedly. For this, it appears to be residual actions needed after the upgrade of the Premium Music Library as outlined in this support article https://support.plex.tv/articles/upgrade-music-libraries-new-metadata-system/

I have asked the user to do manual artist refreshes on those affected as per section headed Upgrade Existing Music Artists in that article

For your libraries, I would like to have a look at your Plex Media Server. Could you download the database zip using the help plex web page and send me a link by Private Message where I can download the zip from. Please do this before you take any actions concerning artists refresh

1 Like

@sa2000 Thanks for your response, I have at this point deleted my music library created a new one and over a week added back the files using these settings ā€œPrefer local metadataā€, Album Art ā€œlocal files onlyā€, scanner ā€œplex media scannerā€, agent ā€œpersonal media artistsā€ and unchecked everything else.
These settings result in reasonable scanning time but unfortunately disable most of the features of the plex music library.
Let me know my logs would still help, I hope Plex can solve this issue !

I’m not sure how related our issues are, but a lot of what’s being said here feels very similar to what I’ve been seeing. In my case however, all 3 of my libraries were added AFTER the plex music changes happened.

1 Like

Even when disabling most of the scanner settings I listed above, new Plex music scanner still takes far more time to scan than the old system. Probably would go unnoticed with a smaller music collection.

For each one of these where adding an album or two results in very long period of metadata refresh, I would need logs plus zip of the main database

There may be different causes.

So far two different problem areas discovered

  1. Many artists had unrefreshed items for artists and tracks from old Premium Music Library - that have not been manually upgraded - the action described in this article https://support.plex.tv/articles/upgrade-music-libraries-new-metadata-system/
  2. Thousands of Various Artists tracks were trying to get lyrics and failing

There may be other reasons - so would need logs + copy of the database - download in same way as logs zip
and please indicate what album was added and when and how long the scan/metadata refresh took. If it is hours, do capture logs every hour in case they wrap round too quickly

1 Like

I’m still having this problem and I have a had a Premium music library for years so I can’t manually upgrade it. And I have lyricfind disabled.
I just updated to PMS Version 1.18.8.2527 yesterday, ran a scan of music library last night after adding or modifying 44 albums since my last scan a week ago. The scan was still running this morning (about 8 or 9 hours later). Alerts showed constantly refreshing metadata. Tried to cancel scan but it would not stop so I had to restart Plex.

I will proceed with your request for more info but I need help with that.
On a Mac where is the database located?
I know of:
~/Library/Application Support/ Plex Media Server/
But, then where? Some folders are quite large:
Media - 1.5GB
Metadata - 15.6GB
Plug-In Support - 8.7GB
Updates - 2GB
I know where the logs folder is but please give instructions for how to capture logs every hour. Which logs do you need?

You can use the Plex Web interface to download the logs zip
Settings / Manage / Troubleshooting / Download Logs

Or you can copy out the Logs manually and zip the copy - do not zip in-situ

See https://support.plex.tv/articles/200250417-plex-media-server-log-files/

The web page for Troubleshooting also has option to Download the database - do this when Plex Media Server is idle as it may lockout the database for the time it takes to download it.

the database is held in
~/Library/Application Support/ Plex Media Server/Plug-In Support/Databases
There may be 3 files

com.plexapp.plugins.library.db
com.plexapp.plugins.library.db-shm
com.plexapp.plugins.library.db-wal

For logs make sure debug logging has not been disabled - see
https://support.plex.tv/articles/201643703-reporting-issues-with-plex-media-server/

I have tried to keep out of this thread, because my library scans relatively fast. - usually not more than 2 or 3 minutes, often less. But it does scan the whole library each time an album is added, or even if the embedded metadata for a single file already in the library is changed.

In my experience, this is not true. Adding more than one album at a time initiates multiple scans of the library at least equivalent to the number of albums added. I do have auto scan enabled,as well as the option to only scan folders that have changed enabled. I have those options enabled so that the whole library isn’t scanned when the library changes, only the folder that has changed.

If I wanted a complete library scan, I would initiate that manually.

I’m attaching logs that I believe prove multiple scans are initiated. I’m also attaching what I believe from the logs are a txt file showing specifically at what point each new scan was initiated and completed. The entire process is quick, but that is not the point. The fact that there are multiple scans of the complete library seems redundant, particularly when each scan is full of the lines similar to:
Skipping over directory 'Various\Grease (Original Soundtrack)', as nothing has changed; removing 24 media items from map.
for all but the first scan of the library.

I fully realize my interpretation of the logs could be wrong, but adding 3 albums appears to have initiated 8 complete scans of the library. Obviously, the scanner ignores the rules to only scan the folders that have changed. I can only imagine what happens if any of the other ā€œglitchesā€ already identified come into play.

MultipleMusicScans.txt (7.3 KB)
Plex Media Server Logs_2020-03-14_04-31-37.zip (2.4 MB)

Curious, why the release notes for Plex Server version 1.18.8.2461 said in the ā€œFixesā€ section
ā€œSpeed up repeat scans of large music librariesā€ when a better fix would have been ā€œEliminate repeat scans of large music librariesā€ :thinking:

2 Likes

Plex Media Server Logs_2020-03-14_11-08-31.zip (6.9 MB)

Last night I started a scan of my music library. It finished approximately 12 hours later. Attached are my logs. I downloaded and zipped my database but the zip file (250MB) stops uploading at 2%. I tried several times and restarted my computer but it still cancels out at 2%.

PMS 1.18.8.2527
Mac OS 10.14.6
2.3 GHz Intel Core i7
16GB RAM

I only added 3 albums since the last music scan:
https://musicbrainz.org/release/a9d0837b-4428-41b3-ad49-b3ddb3efeccb
https://musicbrainz.org/release/64a6d345-8603-4720-816b-fe4621bfeb00
And a web comp which isn’t in Musicbrainz called Western Ranch Party Vol. 4 by Various Artists.

Thank you for the logs and the excellent summary

I have already acknowledged that we do scan the whole library - in one of my early posts here

The new scanner is a lot faster and as you can see from your logs - although the whole library section was scanned each time you added an album, the 7 scans all completed within 3 minutes in total.

Nevertheless, I have submitted a request to the development to look into implementing partial directory scanning for the new music scanner. I think it is only the traversal down the directory list which has been seen as harmless and of no impact

I guess the point I am trying to make, is that adding 3 albums initiated 8 scans of the library.

Even accepting the acknowledgement that the whole library is scanned for each addition of an album, that leaves five complete scans of the library un-accounted for and seemingly unnecessary.

In the example, I added 3 albums by 2 artists . Both artists previously existed in the library, and were matched properly. The only way I can account for 8 scans of the library is this way:

Jimi Hendrix -- 2 scans initiated, 2 new folders added to the artist
   Before London -- scan initiated - new folder added
   Get That Feeling -- scan initiated - new folder added

Jimi Hendrix Experience -- scan initiated - new folder added to the artist
   Re-Experienced -- scan initiated - new folder added
      Disc 1 -- scan initiated - new folder added
      Disc 2 -- scan initiated - new folder added

I have deleted and re-built the music library at least 3 times since the new Plex music system came on line. I think I probably have it as close to perfect as possible within the apparent limitations and glitches that do exist in the scanner. I have to agree, in my case, the complete scan is of little impact in most cases.

That changes if I add a compilation album, scans are longer because it spends time refreshing the metadata for ā€œVarious Artistsā€ - a completely useless endeavour, because no new metadata is actually retrieved for that artist. That is - none exists, it is not an actual artist. But the metadata for each and every album is refreshed, even though only one album was added.

That would be a process that should occur during the scheduled maintenance period in most cases, especially when the artist is already matched properly in the library. That should apply to all artists, not just ā€œVarious Artistsā€ I have quite a few albums by the Beatles, for example, and any changes to that folder, initiates a complete refresh of the metadata. I’m reasonably certain there are no new concert dates to download for them I might be interested in. :wink:

1 Like

I can see the issue here - whilst we are not doing partial scanning we should at least consolidate the scans into one.

I am following this up.

1 Like

This is my problem exactly and why the each scan takes 12 hours. It tries to refresh metadata for 7,844 V.A. albums.