Plex Scanner freezing during scan, server unresponsive until reboot

Plex Media Server Desktops & Laptops
1

Server Version#: 1.21.3.4046

Log attached

I’ve had an issue that started about a week or two ago where my scanner would hang. I first noticed this when I tried to set up a new media source, and the scanner would not add my content. I’d have to manually stop the scan. I tried deleting the library, optimizing the database, emptying trash and trying again but it’s very hit or miss.

When searching for others with this issue, I hear it can be a bad file. But the server is hanging on files that were on the server before the issue started.

Digging into the logs, a lot of them end with something similar to this: “Directory had 22 files, database had 13 files, can’t skip.”

The files are random though, it will encounter a file, get stuck trying to skip it and then it brings the server down if I leave the scan going. Anyone have any ideas? I’m not sure what to do besides nuke this server and start fresh which I would prefer not to do. I’ve got a ton of libraries and media.Plex Media Scanner.1.log (14.6 KB) Plex Media Scanner.1.log (14.6 KB)

2

May I have all the DEBUG log files (ZIP file) ?

one scanner log is insufficient.

These logs show the scanner talking to PMS. They don’t show PMS attempting to resolve the match

3

Here’s the entire zip,
Thanks,Plex Media Server Logs_2021-02-18_18-00-58.zip (5.9 MB)

4

@Joshuaisarocker

I am seeing PMS doing a lot of matching cleanup work.

Please notice the “dead item count”.

Feb 18, 2021 17:49:43.355 [0x7fa803b43bc0] DEBUG - HTTP requesting GET http://127.0.0.1:32400/library/changestamp
Feb 18, 2021 17:49:43.355 [0x7fa7f9802700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/notify/toggleItemActivity?librarySectionID=2&metadataItemID=8749
Feb 18, 2021 17:49:43.356 [0x7fa803b43bc0] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/library/changestamp
Feb 18, 2021 17:49:43.357 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8743, is has a dead item count of 59.
Feb 18, 2021 17:49:43.357 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8527, is has a dead item count of 5.
Feb 18, 2021 17:49:43.357 [0x7fa803b43bc0] DEBUG - Looking for path match for [/media/Plex/TV Shows/Pokémon/Season 05/Pokémon - S05E07 - Dueling Heroes.mkv]
Feb 18, 2021 17:49:43.358 [0x7fa803b43bc0] DEBUG - Path matched, we're reusing media item 8741
Feb 18, 2021 17:49:43.358 [0x7fa803b43bc0] DEBUG - Bringing back media item 8741 to life.
Feb 18, 2021 17:49:43.358 [0x7fa7f9802700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/notify/toggleItemActivity?librarySectionID=2&metadataItemID=8749
Feb 18, 2021 17:49:43.358 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8750, is has a dead item count of 0.
Feb 18, 2021 17:49:43.358 [0x7fa803b43bc0] DEBUG - Marking item 8750 as alive and well.
Feb 18, 2021 17:49:43.358 [0x7fa803b43bc0] DEBUG - Downloading document http://127.0.0.1:32400/library/changestamp
Feb 18, 2021 17:49:43.358 [0x7fa7f9802700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/notify/toggleItemActivity?librarySectionID=2&metadataItemID=8750
Feb 18, 2021 17:49:43.359 [0x7fa803b43bc0] DEBUG - HTTP requesting GET http://127.0.0.1:32400/library/changestamp
Feb 18, 2021 17:49:43.359 [0x7fa803b43bc0] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/library/changestamp
Feb 18, 2021 17:49:43.360 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8743, is has a dead item count of 58.
Feb 18, 2021 17:49:43.360 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8527, is has a dead item count of 5.
Feb 18, 2021 17:49:43.360 [0x7fa803b43bc0] DEBUG - Looking for path match for [/media/Plex/TV Shows/Pokémon/Season 05/Pokémon - S05E08 - The Perfect Match.mkv]
Feb 18, 2021 17:49:43.361 [0x7fa7f9802700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/notify/toggleItemActivity?librarySectionID=2&metadataItemID=8750
Feb 18, 2021 17:49:43.361 [0x7fa803b43bc0] DEBUG - Path matched, we're reusing media item 8742
Feb 18, 2021 17:49:43.362 [0x7fa803b43bc0] DEBUG - Bringing back media item 8742 to life.
Feb 18, 2021 17:49:43.362 [0x7fa803b43bc0] DEBUG - Updating deletion state for metadata item 8751, is has a dead item count of 0.
Feb 18, 2021 17:49:43.362 [0x7fa803b43bc0] DEBUG - Marking item 8751 as alive and well.
Feb 18, 2021 17:49:43.363 [0x7fa803b43bc0] DEBUG - Downloading document http://127.0.0.1:32400/library/changestamp

Have you recently moved / renamed media ?

5

Yes I have. I started peeling items out of TV Shows for example, and moving them into a Documentary Series library. I’m also cleaning up/renaming files to get them in their correct naming structure. I guess I went pretty heavy with renaming/moving over the course of a weekend.

6

@ChuckPa
So I uninstalled Plex from my server using the instructions in this link, and removed the /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/ directory.

[Uninstall Plex Media Server | Plex Support]

I then reinstalled Plex Server, claimed the new server and started adding my content back. My scanner is hanging right away. What do I need to do to fix this? Do I need to do a clean Ubuntu install? Fresh log attached.

Plex Media Server Logs_2021-02-19_14-37-39.zip (121.7 KB)

7

In your logs, you have 3 CIFS mounts.

Is this where PMS is hanging?

Feb 19, 2021 14:33:30.457 [0x7f31e67fc700] DEBUG - Filesystem Type: cifs Path: /media/Plex Name: //192.168.1.6/Plex
Feb 19, 2021 14:33:30.457 [0x7f31e67fc700] DEBUG - Filesystem Type: cifs Path: /media/Plex2 Name: //192.168.1.6/Plex_2
Feb 19, 2021 14:33:30.457 [0x7f31e67fc700] DEBUG - Filesystem Type: cifs Path: /media/Plex3 Name: //192.168.1.6/usbdisk1
8

Yes, all my media is stored on 3 drives in a NAS. Nothing has changed recently with my CIFS setup. Unless something has changed with how Ubuntu handles CIFS shares?

9

Being a NAS, would you consider NFS (NFS is the native protocol for Linux) ?

10

Unfortunately I’m using an older 2 Bay Buffalo NAS that only supports SMB 1/2. Or I would be using NFS. I’ve had no real issues with this server up until recently. The NAS and Server have been working well together for 3+ years.

11

Fair enough ; I had to try :rofl:

I don’t know if Ubuntu changed CIFS – I don’t use it

When you look at the activity monitor (upper right), while scanning, do you get a sense of where it’s at ?

I’m asking because I’m wondering if the issue is on the NAS itself or maybe the naming of something new you just added ??

12

:grin: Trust me this is making me want to move everything over into a rack server and ditch the nas entirely! And move over to Windows or Windows Server.

As far as what I’m seeing during the scan, it shows to be hanging up on a TV Show that has existed on the server for quite some time now. No changes have been made to it, and the naming structure is a TVDB match.

I’m not going to pretend like I know what these logs are saying/doing, but the logs keep ending like this,

Feb 19, 2021 14:29:48.379 [0x7f8c4f7dcbc0] DEBUG - Directory had 26 files, database had 0 files, can't skip.

Now what’s strange is, my Plex Activity monitor shows to be scanning “Aaahh!!! Real Monsters”
image

If you go up a little bit in the log, it shows to be looking at a different folder before it hangs. Looks like it’s trying to scan Adventure Time (Again, no changes to this folder) but the Activity monitor is showing Aaahh!!! Real Monsters as the hang up.

> Feb 19, 2021 14:29:48.282 [0x7f8c4f7dcbc0] DEBUG - Checking descendants of Aaahh!!! Real Monsters/Season 01
> Feb 19, 2021 14:29:48.283 [0x7f8c4f7dcbc0] DEBUG -  -> FOUND metadata item (show)
> Feb 19, 2021 14:29:48.283 [0x7f8c4f7dcbc0] DEBUG -  -> We found a local media item with rooted metadata in Aaahh!!! Real Monsters/Season 01
> Feb 19, 2021 14:29:48.283 [0x7f8c4f7dcbc0] DEBUG - Found existing show 11
> Feb 19, 2021 14:29:48.284 [0x7f8c4f7dcbc0] DEBUG - Downloading document http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.284 [0x7f8c4f7dcbc0] DEBUG - HTTP requesting GET http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.285 [0x7f8c4f7dcbc0] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.286 [0x7f8c4f7dcbc0] DEBUG - Downloading document http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.286 [0x7f8c4f7dcbc0] DEBUG - HTTP requesting GET http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.287 [0x7f8c4f7dcbc0] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/library/changestamp
> Feb 19, 2021 14:29:48.288 [0x7f8c4f7dcbc0] DEBUG - Added new metadata item () with ID 67
> Feb 19, 2021 14:29:48.288 [0x7f8c4dff3700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/flushMetadataCounterCache
> Feb 19, 2021 14:29:48.289 [0x7f8c4dff3700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/flushMetadataCounterCache
> Feb 19, 2021 14:29:48.309 [0x7f8c4f7dcbc0] DEBUG - HTTP requesting PUT http://127.0.0.1:32400/activities/a2be1061-b009-481b-8b0f-1629af741a91?percentComplete=%2B0%2E108552635&subtitle=Aaahh%21%21%21%20Real%20Monsters%20-%20Season%2004
> Feb 19, 2021 14:29:48.312 [0x7f8c4f7dcbc0] DEBUG - HTTP 200 response from PUT http://127.0.0.1:32400/activities/a2be1061-b009-481b-8b0f-1629af741a91?percentComplete=%2B0%2E108552635&subtitle=Aaahh%21%21%21%20Real%20Monsters%20-%20Season%2004
> Feb 19, 2021 14:29:48.312 [0x7f8c4f7dcbc0] DEBUG - Scanner: Processing directory /media/Plex/TV Shows/Adventure Time (parent: yes)
> Feb 19, 2021 14:29:48.349 [0x7f8c4f7dcbc0] DEBUG - Scanner: Processing directory /media/Plex/TV Shows/Adventure Time/Season 01 (parent: yes)
`> Feb 19, 2021 14:29:48.379 [0x7f8c4f7dcbc0] DEBUG - Directory had 26 files, database had 0 files, can't skip.`

I was hoping that by removing Plex Service and deleting the directory, I’d be starting with a fresh database and it would fix whatever the problem is. What did manage to scan is playable in my Plex Players. But if I leave the scanner going on a hung up item, eventually the server will become unresponsive.

EDIT: I ran a disk check on the NAS and found no errors.

13

It must be trying to match that.

> Feb 19, 2021 14:29:48.312 [0x7f8c4f7dcbc0] DEBUG - Scanner: Processing directory /media/Plex/TV Shows/Adventure Time (parent: yes)
> Feb 19, 2021 14:29:48.349 [0x7f8c4f7dcbc0] DEBUG - Scanner: Processing directory /media/Plex/TV Shows/Adventure Time/Season 01 (parent: yes)
> Feb 19, 2021 14:29:48.379 [0x7f8c4f7dcbc0] DEBUG - Directory had 26 files, database had 0 files, can't skip.`

Why it’s doing it is a mystery but by the message itself, the files were never previously matched.

May I ask the full filenames , including extension?

14

Hopefully a screenshot will suffice. This isn’t the only library giving me this problem, multiple folders are giving me the same issue. TV, Movies…

image
image1222×718 79.9 KB

As far as TV goes, I removed the Adventure Time, and Ahhhh!!! Real Monsters folders from the media folder. I rebooted, started a fresh scan and now it’s hanging on Animaniacs. Looking at the console log, this is what I’m seeing a lot of. XML Errors,

image
image1178×424 125 KB

15

  1. Ignore the SSDP / XML errors. I’ve worked with engineering and they agree they are A) noise B) should not be blasting into the logs and C) have already changed how they will report. You will only see them if VERBOSE logging is enabled (to aid debugging discovery problems)

  2. You’re using MP4 files. PMS will always read the embedded info in them. It’s important to control that impact.

  3. I recommend using mediainfo to see what the embedded title / etc show you . Make sure the names are clean or , even better, completely absent.

  4. many folks like to use MP4 files. I use MKV exclusively. FileBot does the naming (they are exactly as yours are) and don’t have an issue.

  5. I will make some tests , using MKVs, of the series you have and see what happens.

16

This is my QA library. It has 911 full TV series in it (all test files).

I added “Adventure Time”.

within seconds it was started and then done within the minute.

Screenshot from 2021-02-19 13-21-33
Screenshot from 2021-02-19 13-21-33928×539 134 KB

To share the size of this server install: 157K indexed items.

[chuck@lizum /vie.507]$ find ./movie* ./tv* ./music* ./qa* -type f -print | wc -l

157950
[chuck@lizum /vie.508]$

17

Got it. I have the full series as well, and I’ve had it for some time now.

OK i’m about to format and reinstall a fresh Ubuntu Server build. I’ll reinstall Plex and let you know what happens.

18

BEFORE YOU DO THAT!!! :slight_smile:

  1. Remove the existing server name -OR- remember to give it a different Friendly name when you reinstall (if installing cold)
  2. Make a tar ball of ./Library to save the existing - if you want. chown to the new UID/GID after reinstalling & restoring.
19

Thanks for the reminder, especially changing the server name. That’s gotten me into trouble before.

So after buttoning up a fresh install, adding my CIFS shares, I’m having the same problem. I really don’t know what changed. So many of these files that look to be causing me trouble have been on the server for a very long time.

I’m at a loss. A week ago my server was running fine. Now it’s un-useable .

Plex Media Server Logs_2021-02-19_22-20-36.zip (170.2 KB)

20

So I figured it might be easier for me to troubleshoot on Windows. I set up a server on my windows Desktop and added it as a new server. Scanning TV Shows stops on the same files. I guess that rules out potential CIFS problems with Linux. I’m running some antivirus scans on my drives right now as a precaution. Once those clear, I’ll move a large chunk of TV Shows over to one of my desktop drives and attempt a rescan. If those go through, I guess I can trace the problem back to the NAS itself or one of the drives.