Server Version#: 1.18.0.1944
Player Version#: N/A
My Plex Media Server since update to version 1.18.0.1944 has been pegging the CPU at over 300% when it runs the media scanner to update libraries. First it had access remote files and folders enabled. Next I tried with full disk access. Neither has seemed to solve the problem. I can upload or send logs to development for review if required.
the new music library uses a lot of cpu during the conversion.
is this linux ? do you have folders and/or albums with over 1000 tracks in them?
if the above is true, this may apply @ New music scanner - high 100% cpu on half number of cpu cores
otherwise, you may simply need to be patient and let it finish processing (it may take multiple days depending how large your library is and how weak/powerful your system is).
But I was not having any issues before this update.
I am able to run the updates to my Movie and TV Libraries without issues. It seems localized to my music library.
it’s ruined the music library here completely. Was perfect now it’s sh**.
So I have over 700,000 music files. I got a PLEX Pass to try the music mapping functionality and I didn’t like it so none of my music libraries are of that advanced type. Now I updated my server to 1.18.0.1944 and the CPU has been pegged for 12 hours and the server is not reachable.
Luckily I have a backup of the entire server and if this blows it up I guess I’ll just go back to the old version and never update again until PLEX makes this rescan optional.
@arejay73 yes its a new music library, plex combined the premium and basic library and it is now converting whatever music library you had to the current format.
yes it can take a lot of cpu
yes it can take a lot of time
if you are not on linux, and/or if you do not have folders/albums with 1000+ tracks in them, then most likely you just need to wait it out.
I have 220k+ tracks, and it took over 1.5 days to process the majority of them.
also you can turn of plex web > settings > library > uncheck ‘scan my library periodically’ since this can trigger unwanted music rescans
So right now the server is completely inaccessible until this scan is done? You would hope that it would bring up the server so you could see stuff that isn’t music but I can’t get in at all.
I don’t know what else to tell you other than, if it appears like it is processing (you can still see it processing files), then let it be.
If it appears completely hung, then I would suggest collecting logs, then shutting down pms (make sure no old processes/threads still running), then restart.
no one on this thread has detailed exactly what their system specs are (specific cpu/os etc), nor has anyone provided logs.
providing no details and saying its ‘inaccessible’ does not help get the problem(s) identified or resolved.
if/when plex is accessible, you can get logs @ plex web > settings > troubleshooting > download logs
otherwise see @ https://support.plex.tv/articles/200250417-plex-media-server-log-files/
for reference, my server is dual xeon 6 cores (24 threads) and it took ~1.5 days to process ~220k+ files.
if scan library periodically is enabled, it will scan music library and attempt to continue to match anything is not matched.
you can see what is not matched @ music library > filter unmatched artists/albums
at certain times it was slow to respond, and hubs slow or timed out, but I don’t think I lost access completely (except for the specific hangs that are outlined in my thread linked above).
but my server is not used by very many people, and is sufficiently powerful enough to handle the load, both cpu wise and disk IO wise.
you are on windows, so the hang that I experience would not apply to you (only affected linux).
is this a log while plex is not responding ?
Yeah, Plex is cranking like crazy (100% CPU) but is not accessible in any way. Went into the server and copied that file out.
ok, I am no log expert, but this looks like a slow system drive and/or an extremely large database (mine is ~1.5 gig db)
Oct 15, 2019 02:38:16.228 [9864] WARN - SLOW QUERY: It took 62453.125000 ms to retrieve 20 items.
Oct 15, 2019 02:38:26.884 [9864] WARN - SLOW QUERY: It took 63046.875000 ms to retrieve 0 items.
Oct 15, 2019 02:38:27.900 [9132] DEBUG - Auth: authenticated user 1 as tramp78
Oct 15, 2019 02:38:37.103 [9864] WARN - SLOW QUERY: It took 60750.000000 ms to retrieve 20 items.
Oct 15, 2019 02:38:45.040 [8176] DEBUG - BPQ: [Starting] -> [Processing]
Oct 15, 2019 02:38:47.228 [9864] WARN - SLOW QUERY: It took 60125.000000 ms to retrieve 20 items.
Oct 15, 2019 02:38:47.228 [9864] DEBUG - External Metadata: Found 5548 items of type artist in our libraries, and a total of 30410 items to look for in 77610 ms.
Oct 15, 2019 02:38:47.243 [9864] DEBUG - It took 93.750000 ms to retrieve 49 items.
Oct 15, 2019 02:38:53.696 [9864] DEBUG - It took 36968.750000 ms to retrieve 9881 items.
Oct 15, 2019 02:39:03.837 [9864] DEBUG - It took 59828.125000 ms to retrieve 931 items.
Oct 15, 2019 02:39:14.040 [9864] DEBUG - It took 60453.125000 ms to retrieve 2713 items.
Oct 15, 2019 02:39:23.961 [9864] DEBUG - It took 58109.375000 ms to retrieve 329 items.
Oct 15, 2019 02:39:34.680 [9864] DEBUG - It took 63265.625000 ms to retrieve 574 items.
Oct 15, 2019 02:39:44.914 [9864] DEBUG - It took 60531.250000 ms to retrieve 771 items.
Oct 15, 2019 02:39:54.742 [9864] DEBUG - It took 58265.625000 ms to retrieve 545 items.
Oct 15, 2019 02:40:04.304 [9864] DEBUG - It took 56687.500000 ms to retrieve 515 items.
Oct 15, 2019 02:40:14.273 [9864] DEBUG - It took 58953.125000 ms to retrieve 105 items.
Oct 15, 2019 02:40:25.069 [9864] WARN - SLOW QUERY: It took 63171.875000 ms to retrieve 0 items.
Oct 15, 2019 02:40:35.272 [9864] WARN - SLOW QUERY: It took 60296.875000 ms to retrieve 0 items.
Oct 15, 2019 02:40:45.725 [9864] WARN - SLOW QUERY: It took 62203.125000 ms to retrieve 0 items.
Oct 15, 2019 02:40:56.006 [9864] WARN - SLOW QUERY: It took 60812.500000 ms to retrieve 0 items.
Oct 15, 2019 02:41:06.084 [9864] WARN - SLOW QUERY: It took 57781.250000 ms to retrieve 0 items.
Oct 15, 2019 02:41:16.412 [9864] WARN - SLOW QUERY: It took 61218.750000 ms to retrieve 0 items.
Oct 15, 2019 02:41:27.005 [9864] WARN - SLOW QUERY: It took 63031.250000 ms to retrieve 20 items.
also, this does not look good, hundreds or more similar lines
Oct 15, 2019 02:57:34.717 [5756] DEBUG - It took 78.125000 ms to retrieve 1 items.
I do not see any log lines about actual scanning the music library, you would see a bunch of lines about get ‘acoustid’ around other pretty obvious stuff about music
hopefully one of the ninjas or dev’s can take a look for more details/info.
The media files are on a Synology NAS. The server itself is connected via ethernet to that NAS. The database IS big. Do we know what it is trying to retrieve there? Oh, and thanks for looking at it. I really appreciate it.
how big exactly ? and you still have not detailed the actual server specs… i mean if you running a 10 gig database on a 10 year old cpu with 4 cores and 16 gig of ram, on an old 500 gig laptop drive (just being silly here) then yeah I would say you need to upgrade…
that said plex has not exactly made it easy/simple/possible to ‘opt out’ of the music upgrade (other than not updating pms).
to me, this seems like some type of ‘pre-processing’ getting ready to scan files for pulling external metadata… the actual scanning does not appear to have started.
on your windows task manager, you can sort the process by CPU, DISK IO and network IO.
you should investigate that to see if plex is still doing disk/network IO while it is processing.
just because plex may be unresponsive, does not necessarily mean it isn’t doing something
The last time I measured the folder where the database was it was about 90gig and 2.6 million files. The server is a Dell Precision T5500 server with 18 gig RAM and an Intel Xeon X5650 six core running at 2.67GHz. This isn’t a multithreaded processor so those six cores equate to six cores.
Disk I/O is 0
Network I/O is .1 Mbps
ok, so server, is an almost 10 year old one (don’t worry I have 3 old dells with similar cpus, that I no longer use with plex).
fyi here the pass mark for that cpu https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+X5650+%40+2.67GHz&id=1304
you need to look at the database itself, 90 gig for only the database folder is still pretty friggin huge, and I mean hoooooge in the trump sense… If the actual plex db itself was only 9 gig, that would still be huge, and I would suggest a very extreme example.
I would suggest taking a look, just to know for certain what size the db is, and how that compares to anyone else having problems.
when plex is maxed out/unresponsive, how many cpus are pegged?? is it 3 out of the 6 ? because that would be similar to what happened on my linux server (12 out of 24 cores @ 100%) when plex would hang.
In my case, it was due to some folders with 1000+ tracks.
In your case, on windows, it might simply be the shear size of your library that exceeds some (possibly same or possibly different) internal resource limit(s) and appears to hang in a similar to manner to what happened to me on linux.
at this point, you are probably going to need dev attention to investigate further.
All CPUs are pegged. My problem with some of this is that server, while old, was perfectly fine for what it was originally asked to do. run a PLEX server that multiple people could stream from. It can handle about 4 at at time. I really don’t want to upgrade that box just because the new server software is a CPU hog doing something I would rather it not do.
By the way, how does one go about getting DEV attention?
I understand, and can agree.
But, you have to admit that your PMS is of an extreme size and any bulk processing is going to be taking a large amount of resources, and time.
Well, I have about 23K audio tracks. I have a 2018 MacMini i5. I have let it run for about 20 mins and then the CPU is pegged at 300%. I will leave it running for a bit and watch the logs to see if it is moving through the files. If not then I will create a bug report.
@arejay73 that log looks good and as long as it continue to process new files, then I would suggest you let it be.
it will do a lot of acousticid stuff, matching fingerprints, downloading any matched metadata.
i usually tail the main server log (a lot more noise), but that scanner log might be clearer.
