Command Line Scanner and Partial Refresh

Could the Q-S be the problem. That is where you would normally see a SEASON folder. Is this even supported?

I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched. Though I used to have it is the manner you are designating too previously.

@eldon.mcguinness said:
I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched.

But would like you to do a test with a structure like this
/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
as opposed to
/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv
and specifying the Season 01 directory path as parameter

@sa2000 said:

@eldon.mcguinness said:
I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched.

But would like you to do a test with a structure like this
/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
as opposed to
/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv
and specifying the Season 01 directory path as parameter

I can try to set that up tonight, but I used to have it setup just like that less that two weeks ago. I recently changed to the way I have it now as directory listing was getting rather slow. As noted all the way back last year, the issue is the same with that directory structure too.

Just to be clear, you’re not asking me to move all the other files as well, just to move this one and try it out. Correct?

@eldon.mcguinness said:
Just to be clear, you’re not asking me to move all the other files as well,

I am not at this point in time asking you to do that.

I have not been able to replicate the issue with the structure that I have and the partial directory being scanned being the season directory

So at this moment it is a test to establish if that is the cause.

Another test to try is to temporarily enabling automatic library updates and see when you add a specific TV show what the log file contains and if the problem is there as well or not

Ok I am running the test now to see what it does if I run:
/usr/lib/plexmediaserver/Plex Media Scanner -p -r -s -c 2 -d /mnt/nt/Jamie's Great Italian Escape/Season 01/

Additionally, adding automatic library updates will do nothing in my case as it is a network filesystem that does not support polling.

@eldon.mcguinness said:
Ok I am running the test now to see what it does if I run:
/usr/lib/plexmediaserver/Plex Media Scanner -p -r -s -c 2 -d /mnt/nt/Jamie's Great Italian Escape/Season 01/

Additionally, adding automatic library updates will do nothing in my case as it is a network filesystem that does not support polling.

J-M ?

/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/

@sa2000 said:

@eldon.mcguinness said:
I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched.

But would like you to do a test with a structure like this
/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
as opposed to
/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv
and specifying the Season 01 directory path as parameter

I don’t think that’s by spec either UNLESS he adds a new path/folder to the TV show library which would point to /mnt/nm/j-m/

Because from the root it only wants to see Show Name/Season/filename.

@cayars said:

@sa2000 said:

@eldon.mcguinness said:
I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched.

But would like you to do a test with a structure like this
/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
as opposed to
/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv
and specifying the Season 01 directory path as parameter

I don’t think that’s by spec either UNLESS he adds a new path/folder to the TV show library which would point to /mnt/nm/j-m/

Because from the root it only wants to see Show Name/Season/filename.

/mnt/nt/j-m is one of the 13 locations for the library

I moved the Jamie’s Great Itallian Escape shows to the root of /mnt/nt and give it an appropriate structure. It is currently running, but a snippet from the top shows the same thing.

But, after looking at the logs, this will not work as /mnt/nt is not a valid section, I would have to add a new directory to the list and the like. I guess, from Plex’s point of view, the scan is actually looking at the root of the section which is /mnt/nt/J-M in this case.

I ran /usr/lib/plexmediaserver/Plex\ Media\ Scanner -v -p -r -s -c 2 -d /mnt/nt/J-M/Jamie\'s\ Great\ Italian\ Escape/Season\ 01/

Here is what a top snippet off the top of the scanner log, it is currently still running and I will upload a full copy for ya. It looks to be doing the same thing.

Here is, what should be, the full log
nextcloud.progressivethink.in/index.php/s/XYMsKreo30MRWxW

May 11, 2017 17:29:44.755 [0x7f3ee6638700] INFO - /usr/lib/plexmediaserver/Plex Media Scanner -v -p -r -s -c 2 -d /mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/
May 11, 2017 17:29:44.755 [0x7f3ef24f4800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1
May 11, 2017 17:29:44.853 [0x7f3ef24f4800] DEBUG - It took 20.000000 ms to retrieve 2269 items.
May 11, 2017 17:29:49.798 [0x7f3ef24f4800] DEBUG - It took 0.000000 ms to retrieve 2269 items.
May 11, 2017 17:29:54.871 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:29:59.887 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:04.927 [0x7f3ef24f4800] DEBUG - It took 20.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:09.940 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:15.167 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:20.147 [0x7f3ef24f4800] DEBUG - It took 20.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:25.069 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:30.242 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:35.473 [0x7f3ef24f4800] DEBUG - It took 20.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:40.444 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:45.405 [0x7f3ef24f4800] DEBUG - It took 10.000000 ms to retrieve 2269 items.
May 11, 2017 17:30:50.306 [0x7f3ef24f4800] DEBUG - Scanning TV Shows using en(Plex Series Scanner) with 0 current media items in the database...
May 11, 2017 17:30:50.306 [0x7f3ee4fff700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=2&sinceTime=1494537050
May 11, 2017 17:30:50.308 [0x7f3ee4fff700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=2&sinceTime=1494537050
May 11, 2017 17:30:50.367 [0x7f3ef24f4800] DEBUG - Performing a scan with 'Plex Series Scanner' (language: en virtual: 0).
May 11, 2017 17:30:50.367 [0x7f3ef24f4800] DEBUG -   * Scanning /mnt/nt/J-M/Jamie's Great Italian Escape/Season 01
May 11, 2017 17:30:50.432 [0x7f3ef24f4800] DEBUG -     * Scanning directory /mnt/nt/J-M/Jamie's Great Italian Escape/Season 01 (parent: yes)
May 11, 2017 17:30:50.435 [0x7f3ef24f4800] VERBOSE -     * Noting that 'Jamie's Great Italian Escape/Season 01' was updated at 2017-05-11 21:28:49
May 11, 2017 17:30:50.435 [0x7f3ef24f4800] VERBOSE -     * Taking directory 'Jamie's Great Italian Escape/Season 01' (25687) out of the map (0 left)
May 11, 2017 17:30:50.438 [0x7f3ef24f4800] DEBUG - Adding file for scanner: /mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
May 11, 2017 17:30:50.440 [0x7f3ef24f4800] DEBUG - Adding file for scanner: /mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e05.mkv
May 11, 2017 17:30:50.445 [0x7f3ef24f4800] DEBUG - Directory had 2 files, database had 0 files, can't skip.
May 11, 2017 17:30:50.454 [0x7f3ef24f4800] DEBUG -       * Scanning Jamie's Great Italian Escape Season 1 Episode 5
May 11, 2017 17:30:50.454 [0x7f3ef24f4800] DEBUG - Looking for path match for [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.038 [0x7f3ef24f4800] DEBUG - Checking by hash to see if we can find a match with 2aec3186b5517429058a7ae2047891246386939a (display offset: 0, not part -1)
May 11, 2017 17:30:55.038 [0x7f3ef24f4800] DEBUG - We found a hash match for [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e05.mkv] which was [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e05.mkv].
May 11, 2017 17:30:55.051 [0x7f3ef24f4800] INFO - Part rename detected [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e05.mkv] was renamed [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.083 [0x7f3ef24f4800] DEBUG - Updating media item 415707, size=424717925, metadata_item_id=266480
May 11, 2017 17:30:55.133 [0x7f3ef24f4800] DEBUG - Updating part with ID=431546 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.137 [0x7f3ef24f4800] DEBUG - Turbo analysis on modified item 415707 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.142 [0x7f3ef24f4800] DEBUG - Updating media item 415707, size=849435850, metadata_item_id=266480
May 11, 2017 17:30:55.150 [0x7f3ef24f4800] DEBUG - Updating part with ID=431546 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.159 [0x7f3ef24f4800] VERBOSE - Added new media part with ID=440541 [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e05.mkv]
May 11, 2017 17:30:55.210 [0x7f3ef24f4800] DEBUG -       * Scanning Jamie's Great Italian Escape Season 1 Episode 6
May 11, 2017 17:30:55.210 [0x7f3ef24f4800] DEBUG - Looking for path match for [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.241 [0x7f3ef24f4800] DEBUG - Checking by hash to see if we can find a match with 5cde9aca6e8402c8dd4e40c91c2ee748458b07b1 (display offset: 0, not part -1)
May 11, 2017 17:31:01.241 [0x7f3ef24f4800] DEBUG - We found a hash match for [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv] which was [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv].
May 11, 2017 17:31:01.245 [0x7f3ef24f4800] INFO - Part rename detected [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv] was renamed [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.269 [0x7f3ef24f4800] DEBUG - Updating media item 415708, size=402451229, metadata_item_id=266481
May 11, 2017 17:31:01.278 [0x7f3ef24f4800] DEBUG - Updating part with ID=431547 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.282 [0x7f3ef24f4800] DEBUG - Turbo analysis on modified item 415708 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.294 [0x7f3ef24f4800] DEBUG - Updating media item 415708, size=804902458, metadata_item_id=266481
May 11, 2017 17:31:01.303 [0x7f3ef24f4800] DEBUG - Updating part with ID=431547 [/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.326 [0x7f3ef24f4800] VERBOSE - Added new media part with ID=440542 [/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv]
May 11, 2017 17:31:01.343 [0x7f3ef24f4800] DEBUG - Removing 0 media items that were left.
May 11, 2017 17:31:01.343 [0x7f3ef24f4800] DEBUG - Removing 0 directories that were left.
May 11, 2017 17:31:01.343 [0x7f3ef24f4800] DEBUG - Updating directory 'Jamie's Great Italian Escape/Season 01' (ID 25687) to time 2017-05-11 21:28:49.
May 11, 2017 17:31:01.417 [0x7f3ef24f4800] DEBUG - Refreshing section 2 of type: 2
May 11, 2017 17:31:01.443 [0x7f3ef24f4800] DEBUG - The show 'Absolute Genius Monster Builds' needs refreshing, no match yet.
May 11, 2017 17:31:01.454 [0x7f3ef24f4800] DEBUG - The show 'The After Party 2017' needs refreshing, no match yet.
May 11, 2017 17:31:01.551 [0x7f3ef24f4800] DEBUG - The show 'Beachin' Rv's' needs refreshing, no match yet.
May 11, 2017 17:31:01.583 [0x7f3ef24f4800] DEBUG - The show 'Bikinis and Ski Slopes' needs refreshing, no match yet.
May 11, 2017 17:31:01.589 [0x7f3ef24f4800] DEBUG - The show 'Biq Digits' needs refreshing, no match yet.
May 11, 2017 17:31:01.591 [0x7f3ef24f4800] DEBUG - The show 'Bitchin' Rides: High Octane' needs refreshing, no match yet.
May 11, 2017 17:31:01.634 [0x7f3ef24f4800] DEBUG - The show 'Britain at War Imperial War Museum At' needs refreshing, no match yet.
May 11, 2017 17:31:01.672 [0x7f3ef24f4800] DEBUG - The show 'Celebrities: In Therapy' needs refreshing, no match yet.
May 11, 2017 17:31:01.867 [0x7f3ef24f4800] DEBUG - The show 'Everyday Innovators' needs refreshing, no match yet.
May 11, 2017 17:31:01.906 [0x7f3ef24f4800] DEBUG - The show 'Flame Halfworlds' needs refreshing, no match yet.
May 11, 2017 17:31:01.914 [0x7f3ef24f4800] DEBUG - The show 'Flog It Trade Secrets' needs refreshing, no match yet.
May 11, 2017 17:31:01.922 [0x7f3ef24f4800] DEBUG - The show 'Fonejacker Christmas Special' needs refreshing, no match yet.
May 11, 2017 17:31:01.954 [0x7f3ef24f4800] DEBUG - The show 'Gareth Wyn Jones Milk Man' needs refreshing, no match yet.
May 11, 2017 17:31:02.021 [0x7f3ef24f4800] DEBUG - The show 'Gutsful!' needs refreshing, no match yet.

Yes still the same

May 11, 2017 17:31:01.417 [0x7f3ef24f4800] DEBUG - Refreshing section 2 of type: 2
May 11, 2017 17:31:01.443 [0x7f3ef24f4800] DEBUG - The show 'Absolute Genius Monster Builds' needs refreshing, no match yet.
May 11, 2017 17:31:01.454 [0x7f3ef24f4800] DEBUG - The show 'The After Party 2017' needs refreshing, no match yet.

Thanks for trying

@sa2000 No problem, I’m willing to try almost anything to get this nailed down.

@cayars said:

@sa2000 said:

@eldon.mcguinness said:
I can confirm that the file naming I use works just fine for tv and movies. The only issue I’ve encountered so far is this odd refresh issue. The metadata comes in fine for shows that actually are matched.

But would like you to do a test with a structure like this
/mnt/nt/J-M/Jamie's Great Italian Escape/Season 01/Jamie's Great Italian Escape.s01e06.mkv
as opposed to
/mnt/nt/J-M/Jamie's Great Italian Escape/Jamie's Great Italian Escape.s01e06.mkv
and specifying the Season 01 directory path as parameter

I don’t think that’s by spec either UNLESS he adds a new path/folder to the TV show library which would point to /mnt/nm/j-m/

Because from the root it only wants to see Show Name/Season/filename.

Just for clarification here are the sections in my TV library, I am doing exactly as you said. :smile:

/mnt/ct
/mnt/pool0/ct
/mnt/pool0/nt
/mnt/pool0/Kids Series
/mnt/nt/#
/mnt/nt/A-C
/mnt/nt/D-F
/mnt/nt/G-I
/mnt/nt/J-M
/mnt/nt/N-P
/mnt/nt/Q-S
/mnt/nt/T-V
/mnt/nt/W-Z

I’m a bit over organized heh

Thank you for persevering - Whilst I can see where it might be going wrong, what I do not understand now is why it does not go wrong for me in my tests (I am using automatic notification but should be the same as the command line is logged)

Could you get me the output from this and you can send it zipped by private message
ls -laiR /mnt/nt

Sent! Let me know if you need more info.

Perhaps what is happening is the shows you are adding are finding matches? As I mentioned, the only shows that do this are ones that do not have valid entries that the agents can find.

@eldon.mcguinness said:
Perhaps what is happening is the shows you are adding are finding matches? As I mentioned, the only shows that do this are ones that do not have valid entries that the agents can find.

Thanks for the clues.
That might be why it does not happen in my tests. In that I do not have any without valid matches

@eldon.mcguinness said:
Sent! Let me know if you need more info.

You are very organized - you already deleted the Season 01 folder we did the last test for and put back the episodes where you had them before the test - I cannot see in the ls -laiR recursive tree you send me

@sa2000 Did you want the ls -laiR to include the “Season 01” folder? I can create it if you like and do the command again.

@eldon.mcguinness said:
@sa2000 Did you want the ls -laiR to include the “Season 01” folder? I can create it if you like and do the command again.

No need.

I am going to raise this now as an issue with the development team

If it is only picking unmatched items as extra - the impact is not as bad as if it was picking the whole library

I presume the impact could be mitigated further by addressing the mismatches or moving them to a separate library - leaving you with a perfect library

In a perfect situation that would work, but in a case when someone is pairing it with the find command and a time frame of the last 6 hour, it can cause quite a lot of wasted cycles. Unfortunately, this is the only real option to autoscan files that are on a filesystem that does not support polling/notify. Imagine you have 40 new files, that would trigger the refresh command 40 times over and over until the time frame passed.

Again, I understand this is just my use case and I can not say if others will encounter it, but yeah.

Thanks for looking into the issue!

@“eldon.mcguinness” - Ok, I was able to reproduce your issue and have figured out why. It is as you said. When doing a scan, even a partial scan, it checks for other shows that have not been matched and will attempt to match them. @sa2000 and I are checking to see if this is the expected behavior.

To prevent this from happening for you, you will need to get your shows to switch from being unmatched to being marked as a personal show. In Plex Web, at the show level click the ... icon at the top and click match. You get the search window. Click on “Search Options”, change the agent to “Personal Media Show”, then “Search”. You’ll get a result window with the name of that show, select it. Done. Now you’ll need to repeat this for each show that is unmatched.

@“MovieFan.Plex” awesome to hear you’re checking on it. I would think the only time it would I do that is if a directory was not specified.

Thanks for the extra information, but honestly this is only an issue until I have the time to either properly name the shows or enter info into tvdb or tmdb. It would not be worth flagging the shows as personal media as I would be fixing them eventually. I’m just trying to fix the need for redundant api calls trying to identify shows in such a short period of time and the wasted processor time associated with it.