Plex Media Scanner 300% CPU on Version 1.18.0.1944

I am not sure what I can offer here but will attempt if desired.

To boil down some of the things I see quantitatively,

  1. 700,000+ tracks
  2. Grossly overloaded database query (database itself needs urgent optimization) - SLOW QUERY warnings. (1 minute per query versus 10 ms per query normal = 6000% slowdown )
  3. CPU overload due to fingerprinting process for 700,000+ tracks.

Are there any other discrete facts I’m missing?

So I used to have 90 gig and 6 million files. A few weeks ago I optimized the database and cleaned bundles and trash and all that and it dropped to 2.3 million files and it is still 90 gig.

Please clarify as this is important (to me).

  1. Size of the Plex data (all metadata) space on disk is 90GB -or- the physical single DB file ?
  2. How do you have your media curated? Is it curated by Genre, or time period, or some other method, e.g.
/music/
      Pop/
      Folk/
      Classical/
      .
      .

@Swervie

Please do not snipe or hijack a thread I’m trying to help someone with.
Thank you for your cooperation.

1 Like

That size is the size of the entire Plex Server folder.
the music is spread out over several directories.
Pop and Rock is alphabetical so
Music/A
ABBA
Aerosmith
Music/B
Beatles
Beach Boys
With albums in each folder (and a lot more groups in each alpha letter.
Also Classical in sub folders by performer or composer
Also collections in sub folders by soundtrack, blues collections, etc.

And that is just music - I also have thousands of Audio books by fiction, non fiction, etc.

Thanks, That helps me a great deal. Give me a few minutes please to lookup some things and do some math.

Which Xeon CPUs are those please? (model)

Xeon X5650 six core running at 2.67GHz

@Tramp78,
Thanks for that info.

  1. That’s a nice workhorse you have there. No speed demon but will pound through anything.
  2. I was able to find out,
    a. The update to the new scanner is scheduled to be released today in PMS 1.18.1
    b. It is specifically targeted at lightening the fingerprinting CPU load which everyone is seeing
    c. It will also shorten the latency because the album identification technique was optimized.

Given you have your music so wonderfully curated, I see a few options. Before typing both out, I would like to ask if you have a preference as to which seems to be easier:

a. Wait for the update and apply it then let it slug through the entire process
b. Parition / parcel the task of getting the library re-established the way you like it

1 Like

Will the update slug through at the same rate as it is now? I can’t really update this thing to 1.18.1 since I can’t access it and it I stop the process and start it again it just picks up where it left off. I don’t know of a way from within PLEX to go back to the last server version.
I can, however, just restore the entire box via Acronis to where it was a few days ago. that takes abot 45 minutes. (I can’t recommend Acronis enough for just this purpose).
Abour option B, what exactly is that? Is this just removing those libraries and putting them back in after the upgrade one at a time? Or something else?

Will the new scanner allow the server to function while it is scanning? Right now the entire thing in unavailable from any direction while it pounds on whatever it is pounding on. (that sounds a bit kinky)

Update. So it’s been about 2h and it is still on the same file. So, would it be safe to assume I need to submit a bug report at this point? I am currently tailing the "plex media Scanner Matcher Log and the Plex Media Server Log. Is there another log the I might watch to help me diagnose this problem?

you can’t downgrade versions within plex, but you can stop/exit PMS then (re)install an older version.

Unfortunately plex doesn’t provide access to old versions, so you would have needed to downloaded and save any old installers, or risk googling for 3rd party downloads (not necessarily trust worthy).

I’m hoping 1.18.1 has the fix that was determined for linux, so I can move my ‘1000+’ file folders back into my music library. (I had to remove those particular folders to get plex to finish scanning everything else).

@arejay73 I would recommend holding tight at this point, wait for the new PMS version that Chuck mentioned above, and see if it resolves the issues for you.

1 Like

Would it be better to just call me? LOL

Regarding older versions:

Not sure about Linux and Apple, since I run mine on a Windows server, but if you have updated through the GUI, every update is retained within a folder called “Updates” within the Plex Media Server" folder. Mine went back to almost 3 years ago.

2 Likes

i’m recovering from surgery, working a bit from home, to keep my mind busy… 'nuff said?

you’ve managed to move stuff out of the way. that works for now.

The update, iirc reduces the number of fingerprints required to get a definitive match for an album. As you know, once an album is identified, PMS then maps tracks in the album directory to the album itself.

The change will require media be well curated at this point. CPU loading, based on 20 tracks per album will be reduced to about 1/10th of previous.

doing the math: 700,000 tracks will result in 7,000 fingerprints.

Before the change, I fingerprinted all 18,000 tracks I have in about 90 minutes on 40 Mbps internet.

Does this give some sense of scope?

Thanks. Right now I’m dealing with a bad Acronis restore so the entire server is down until I fix that. But when it is back I may just wait a bit and try again in a few weeks.

First off, how you get better soon. I restored my server to an earlier version so now my PMS is at Version 1.17.0.1841 and it will stay there for a while. May I make a huge suggestion? I know you are folding the scanner into the music product. Can this PLEASE be an option at the library level like it was when it was first rolled out? My initial experience with the fingerprint scanning process was songs that were moved from the album they were in to some other album the scanner thought they should be in. I also had a few large folders of songs (specific example was a Multi Disk Glenn Miller set that the scanner would hang on and never get past. So the things the scanner really needed - the ability to be turned off and the ability to error correct and move on if it got stuck - have to be there before I upgrade to a version that has it hard wired in. You mentioned that my music is well curated and you are right. But that curation goes out the window if the scanner doesn’t agree with it.
Does PLEX have some location for release notes to let us know the details of what a version does before we upgrade to it?
For now I’m going to bring up a second test server on my regular PC (Ryzen 7 1700 and 16 gig ram) to test new versions before I roll it to the main server. I can’t risk going through what I went through yesterday again.
Granted MOST of the pain was Acronis not wanting to do a restore without hitting it with a stick. Again, what I really would like to see is the scanner being an option, not hard coded. I have too many audio files to not be able to cancel a scan and turn off that functionality if need be.

1 Like

Did 1.18.1.1966 include the anticipated CPU load improvement? I reviewed the release notes, and saw the item below, but I couldn’t identify a point that indicates a lightening of CPU usage for the music scanner.

(Metadata) Improved reliability of music lookups

Thank you!

  1. Yes it should include it
  2. Keep 1.17.x handy just in case I’m still too medicated :rofl:

I would start by making a controlled test.

  1. Create a “test music” library section
  2. Add a directory section with a small - moderate number of items in it (5000 - 10000 ?)
  3. Measure its performance.

One thing that would be a BIG help is don’t have the upgrade instantly start scanning. Just upgrade. If I want to rescan I will. If it hangs then I can stop the scan. But having it start scanning before the server is up essentially kills the server. Know what I mean?

Your Settings control that.

It won’t start scanning right away AFAIK…

Mine never does.