New music scanner - high 100% cpu on half number of cpu cores

Plex Media Server Metadata & Adding Files
1

Server Version#: 1.18-1906
Player Version#:

Like @ Can only add basic music library - #8 by theonlycure

and @ Plex server update 1.18.0.1846-f62172e99 is borked - #37 by bleachjt

I also have the new music scanner get hung up on certain folders/files.

12 out of 24 cores are maxed out 100% cpu.

some example screenshots

image
image956×523 121 KB

image
image956×523 113 KB

I have not yet had time to separate out and identify the specific files, but I have narrowed it down several folders each with a bunch of (not random) mixed files, some exceeding 1000 tracks.

1 Like
2

these 2 pictures are only a couple minutes apart

fingerprinting before hang

image
image956×523 122 KB

fingerprinting after hang

image
image956×523 118 KB

logs shortly after hang manifested
Plex Media Server Logs_2019-10-08_23-31-10.zip (16.9 MB)

linux server 1.18.1913

I will let this run all night, and post another screenshot with what I assume will be exactly the same as above.

1 Like
3

screenshot after ~8 hours of letting plex process, appears to be the same threads going.

image
image956×523 114 KB

1 Like
4

Rebuilding my libraries from scratch now.
So far, unable to replicate but will keep at it.

5

to provide some background, these sxsw folders have anywhere from a few hundred to over 1000 mixed mp3s and possibly other file types.

I have copied the folders to my windows desktop, installed 1.18.1913, created a new music library with only those folders, and windows pms appeared to scan the folders with no hangs.

Before I left home this morning I moved these folders out of my main linux music folder/library, and into a temp folder.

I then created a new music library with only that temp folder containing the sxsw folders/files.

The linux pms has repeated the hang with half my cores stuck at 100%.

so behavior between linux and windows is different.

It was suggested that I try to break down the folders into sets of 50, which I will work on when I have time.

But


  1. these folders/files did not have any problems with the previous plex premium or regular music libraries.
  2. I know I am not the only one who has various folders filled with random/semi-random/related files in a single mixed folder.

whether this is an issue with specific file(s) that may be not the best tagged, or even possibly corrupted in some manner, or if its because there are too many files in the same folder, I don’t know.

6

How well are they curated? (names and structure) – Percentage wise ? 90% on spec?

7

they are named like below, with artist - title as the file name. There are no track numbers, as these are not part of a traditional album.

These were tracks available from the sxsw website at whenever they were posted.

C:\Users\Tekno\Documents\test\2012 - SXSW 2012 Showcasing Artists [MP3]\Fake Problems - 5678.mp3
C:\Users\Tekno\Documents\test\2012 - SXSW 2012 Showcasing Artists [MP3]\Ambition - A.M.B.I.T.I.O.N..mp3
C:\Users\Tekno\Documents\test\2012 - SXSW 2012 Showcasing Artists [MP3]\Rachael Sage & The Sequins - Abby Would You Wait.mp3
C:\Users\Tekno\Documents\test\2012 - SXSW 2012 Showcasing Artists [MP3]\RapSody - ABC-Guilty.mp3

They would be tagged with proper artist-title, the album would be approximately the same as the folder name (SXSW 2012 Showcasing Artists).

The folder name itself is
<year> - <album> [<filetype>]

while my files have no doubt been retagged/renamed from the originals, the original files can be downloaded @ http://www.sxswtorrent.com/

8

so you’re using someone’s private relabeling of selected artists and expecting a match?

At this point in the music overhaul, I would not expect any success .

I have perfectly curated music and am finding quirks with 1.18.0.

9

no no no, I am not expecting a match.

I am expecting plex to use the existing tags and files, as the per the library settings.

I am expecting plex to NOT get stuck burning up 12 cores at 100% utilization indefinitely.

This also blocks any other library content updates.

Very very few peoples music libraries are perfect, and expecting everyone’s to be is absurd.

It is bad enough that plex requires such rigid naming for video, but that can be accepted due to lack of established file metadata standards for video files (ie video files don’t have id3 tags).

But music files all have established tags.

Plex simply does not respect them all and does not expose them all to the user/library.

Plex is at the bottom of the list for any music library manager.

As a music player and streaming system, it is awesome. (or can be when not obstructed by other glaring deficiencies)

To manage and curate, it is crap.

  • doesn’t read all tags
  • doesn’t WRITE any tags
  • no renaming or movement of files according to user specified tag templates
  • you can only choose at the library level whether to use local metadata or plex metadata, when nothing is that simple
 I might want to use ONLY local tags for some files, local+plex-augmented metadata on other files, and plex-only metadata other files.

and so many other complaints, but this thread is not about the deficiencies of plex music, so much as it is about trying to find and fix why plex scanner hangs, blocking all other library updates, and causes cpu cores to go 100% indefinitely.

10

Here is PMS on windows, test music library with only sxsw music in it, that scanned without hanging up.

image
image.jpg709×665 158 KB

this is what I expect to have on linux, without the scanner hanging.

11

I’m going to continue investigating here but in my MPC, MP3, FLAC, and WAV files, nothing gets hung up.

There is something else afoot here

12

well hopefully any others will chime in and provide any details and logs that could help.

@theonlycure
@bleachjt

13

@TeknoJunky let’s continue our PM chat in here for visibility.

I am working to see what is differently about how we scan on Linux vs Windows as your results seem to suggest that the issue is limited to Linux :thinking:

14

yeah, seemed ok on windows.

I created a new test music library on linux server (the same one having problem scanning existing library).

I moved all of the sxsw folders to a test folder and added that test folder to the new library.

Shortly after attempting to scan that new library, the scanner hung again.

I shutdown pms, moved all but 2 folders out of the test library and I am rescanning now to see if it completes or hangs.

From there I will continue to add or remove as applicable until I can either get a full scan or hang, in order to see if I can narrow it down to either specific files, or specific number of files in a folder.

15

@TeknoJunky out of curiosity which “year” of SXSW are you using for your “all but 2 folders” test?

I would like to ensure I am using the same ones over here.

16

right now I have scanning 2 folders

  • 2005 sxsw containing 768 files
  • 2006 sxsw containing 934 files

these appear to have completed processing, according to the ‘alerts’ log.


4:09PM Finished processing SXSW 2005 Showcasing Artists
4:09PM Loading metadata for SXSW 2005 Showcasing Artists
4:09PM Finished processing SXSW 2006 Showcasing Artists
4:09PM Loading metadata for SXSW 2006 Showcasing Artists
4:08PM Finished processing SXSW 2005 Showcasing Artists
4:08PM Downloading metadata for SXSW 2005 Showcasing Artists
4:08PM Downloading metadata for SXSW 2006 Showcasing Artists
4:08PM Downloading metadata for SXSW 2005 Showcasing Artists
4:08PM Finished processing Various
4:08PM Loading metadata for Various
4:08PM Loading metadata for SXSW 2005 Showcasing Artists

image

17

From the screenshots earlier, it looks like the 2010 SXSW year is what it was stuck on.

18

yup 2011 and 2012 caused problems as well, I started at the earliest folders and working my way up.

  • 2007 sxsw containing 724 files appears to have been successfully scanned.

image

  • 2008 sxsw containing 761 files appears to have been successfully scanned.
19

as a side note, it appears that plex attempts to re-process each of the ‘albums’ every time this test library is scanned.

I am assuming it is because it is trying to match up with the ‘correct’ metadata and unable to.

I hope there are limits/restrictions that are built in (or should be built in) to prevent plex from wasting bandwidth (yours and mine) trying and failing to match content repeatedly indefinitely.

  • 2009 sxsw containing 1250 files ----- plex appears to have hung upon adding and rescanning with this folder.

When I get home, I will attempt to remove half the files in that folder and rescan to see if it continues to hang or not.

20

Hello.

Some interesting results.

I don’t think I have actually stated it, but whenever the hang has occurred, I have to stop plex and wait for those hung threads to die, then restart plex.

plex being hung after adding the 1250 files, I stopped then I reduced the 2009 sxsw folder from 1250 files, to 500.

After restarting plex and rescanning the test music folder, it appeared I was able to complete the rescan, including the 500 files that were left in the 2009 sxsw folder.

image

plex is still re-matching/re-pulling metadata from the other sxsw folders as I post this, but the 500 files were actually added to the music library (which was not the case during the hung threads).

this suggests either-

  • the quantity of files in the folder DOES have a limit and if exceeded, causes the plex scanner/import process to hang.

OR

  • that I removed any problem files that caused something to hang.

now, while watching plex scan these 500 files, I noticed something else.

first it appears that plex runs bulk accousticid look ups on the entire folder, then it tries to match/reconcile the acousticid metadata with the tracks and/or local metadata.

watching log, it was repeatedly saying something about 398 files, which I suspect is the number of files that it matched, out of the 500.

going further, when plex is doing the get /user/services/acoustid requests, the cpu load is fairly low.

however when it was attempting to match, the cpu usage shot up wildly.

this leads me to suspect, that the problem is not in the SCANNER, but it is in the MATCHING process.

I suspect that the matching table has some type of upper limit or runs out of memory or other resources, and when it is exceeded, that part of plex hangs.

Anyway, at this point I am going to continue adding/rescanning that folder until either I find the quantity limit, or determine which specific file(s) trigger the hang.

TLDR- I highly suspect the issue is related to quantity of files in a folder. You could probably recreate hang with ANY 1000+ files in a folder and scan it into a library.

more to come

1 Like