Command Line Scanner and Partial Refresh

@eldon.mcguinness said:
@“MovieFan.Plex” awesome to hear you’re checking on it. I would think the only time it would I do that is if a directory was not specified.

This is where we are investigating. The logic on when it should try.

Thanks for the extra information, but honestly this is only an issue until I have the time to either properly name the shows or enter info into tvdb or tmdb. It would not be worth flagging the shows as personal media as I would be fixing them eventually.

You can always rematch it using the proper agent later.

@sa2000 or @“MovieFan.Plex” I thought I came up with a way to get around this issue in the short term by running the scans on their own and then running a single refresh at the end. However, the refresh does not seem to do anything when triggered in the movie section. Below is an example of what I tried.

Scanned in a new movie
/usr/lib/plexmediaserver/Plex\ Media\ Scanner -v -p -s -c 1 -d /mnt/nm/W-Z/Waterworld.1995/

May 13, 2017 02:20:02.686 [0x7fbd23d27700] INFO - Plex Media Scanner v1.5.3.3580-4b377d295 - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 - GMT -04:00
May 13, 2017 02:20:02.686 [0x7fbd23d27700] INFO - Linux version: 4.4.0-75-generic (#96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017), language: en-US
May 13, 2017 02:20:02.686 [0x7fbd23d27700] INFO - Processor QEMU Virtual CPU version (cpu64-rhel6)
May 13, 2017 02:20:02.686 [0x7fbd23d27700] INFO - /usr/lib/plexmediaserver/Plex Media Scanner -v -p -s -c 1 -d /mnt/nm/W-Z/Waterworld.1995/
May 13, 2017 02:20:02.686 [0x7fbd2f245800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1
May 13, 2017 02:20:02.822 [0x7fbd2f245800] DEBUG - It took 20.000000 ms to retrieve 466 items.
May 13, 2017 02:20:03.271 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 151 items.
May 13, 2017 02:20:03.420 [0x7fbd2f245800] DEBUG - It took 0.000000 ms to retrieve 321 items.
May 13, 2017 02:20:03.731 [0x7fbd2f245800] DEBUG - It took 0.000000 ms to retrieve 196 items.
May 13, 2017 02:20:03.976 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 333 items.
May 13, 2017 02:20:04.301 [0x7fbd2f245800] DEBUG - It took 0.000000 ms to retrieve 156 items.
May 13, 2017 02:20:04.453 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 196 items.
May 13, 2017 02:20:04.812 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 466 items.
May 13, 2017 02:20:05.267 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 151 items.
May 13, 2017 02:20:05.411 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 321 items.
May 13, 2017 02:20:05.715 [0x7fbd2f245800] DEBUG - It took 0.000000 ms to retrieve 196 items.
May 13, 2017 02:20:05.956 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 333 items.
May 13, 2017 02:20:06.272 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 156 items.
May 13, 2017 02:20:06.422 [0x7fbd2f245800] DEBUG - It took 10.000000 ms to retrieve 196 items.
May 13, 2017 02:20:06.777 [0x7fbd2f245800] DEBUG - Scanning Movies using en(Plex Movie Scanner) with 1 current media items in the database...
May 13, 2017 02:20:06.778 [0x7fbd227ff700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=1&sinceTime=1494655206
May 13, 2017 02:20:06.780 [0x7fbd227ff700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/notify/cullTimeline?librarySectionID=1&sinceTime=1494655206
May 13, 2017 02:20:06.857 [0x7fbd2f245800] DEBUG - Performing a scan with 'Plex Movie Scanner' (language: en virtual: 0).
May 13, 2017 02:20:06.857 [0x7fbd2f245800] DEBUG -   * Scanning /mnt/nm/W-Z/Waterworld.1995
May 13, 2017 02:20:06.866 [0x7fbd2f245800] DEBUG -     * Scanning directory /mnt/nm/W-Z/Waterworld.1995 (parent: yes)
May 13, 2017 02:20:06.867 [0x7fbd2f245800] VERBOSE -     * Noting that 'W-Z/Waterworld.1995' was updated at 2017-05-13 01:55:40
May 13, 2017 02:20:06.867 [0x7fbd2f245800] VERBOSE -     * Taking directory 'W-Z/Waterworld.1995' (7175) out of the map (0 left)
May 13, 2017 02:20:06.870 [0x7fbd2f245800] DEBUG - Adding file for scanner: /mnt/nm/W-Z/Waterworld.1995/Waterworld.1995.mkv
May 13, 2017 02:20:06.874 [0x7fbd2f245800] DEBUG - Skipping over directory 'W-Z/Waterworld.1995', as nothing has changed; removing 1 media items from map.
May 13, 2017 02:20:06.875 [0x7fbd227ff700] DEBUG - HTTP requesting GET http://127.0.0.1:32400/:/metadata/updateProgressMessage?message=Scanning%20W-Z%2FWaterworld%2E1995
May 13, 2017 02:20:06.876 [0x7fbd227ff700] DEBUG - HTTP 200 response from GET http://127.0.0.1:32400/:/metadata/updateProgressMessage?message=Scanning%20W-Z%2FWaterworld%2E1995
May 13, 2017 02:20:06.878 [0x7fbd2f245800] DEBUG - Removing 0 media items that were left.
May 13, 2017 02:20:06.878 [0x7fbd2f245800] DEBUG - Removing 0 directories that were left.

Which got the movie to show up in Plex, but without any metadata, as expected. However, running a refresh on the movie section does not seem to trigger an update of the metadata.
/usr/lib/plexmediaserver/Plex Media Scanner -v -r -p -c 1

May 13, 2017 02:24:22.296 [0x7fbf8e527700] INFO - Plex Media Scanner v1.5.3.3580-4b377d295 - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 - GMT -04:00
May 13, 2017 02:24:22.296 [0x7fbf8e527700] INFO - Linux version: 4.4.0-75-generic (#96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017), language: en-US
May 13, 2017 02:24:22.296 [0x7fbf8e527700] INFO - Processor QEMU Virtual CPU version (cpu64-rhel6)
May 13, 2017 02:24:22.296 [0x7fbf8e527700] INFO - /usr/lib/plexmediaserver/Plex Media Scanner -v -r -p -c 1
May 13, 2017 02:24:22.297 [0x7fbf99a4c800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1

Then I tried to direct the refresh to just the folder in question with the same result
/usr/lib/plexmediaserver/Plex Media Scanner -v -r -p -c 1 -d /mnt/nm/W-Z/Waterworld.1995/

May 15, 2017 12:48:43.726 [0x7fcd89638700] INFO - Plex Media Scanner v1.6.1.3722-4955e31cf - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 - GMT -04:00
May 15, 2017 12:48:43.727 [0x7fcd89638700] INFO - Linux version: 4.10.0-20-generic (#22-Ubuntu SMP Thu Apr 20 09:22:42 UTC 2017), language: en-US
May 15, 2017 12:48:43.727 [0x7fcd89638700] INFO - Processor Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz
May 15, 2017 12:48:43.727 [0x7fcd89638700] INFO - /usr/lib/plexmediaserver/Plex Media Scanner -v -r -p -c 1 -d /mnt/nm/W-Z/Waterworld.1995/

However, the kicker is if I pair the two flags -r and -s it acts just fine. And just to be clear, this is ONLY with the movie section. If I do this in the TV section a refresh of the section causes all the unmatched items to be checked. It is also worth noting that if I try the combination of the -r and -s flags AFTER something has been scanned with out the refresh flag it will not refresh the metadata of the movie in question.
/usr/lib/plexmediaserver/Plex\ Media\ Scanner -v -p -r -s -c 1 -d /mnt/nm/W-Z/Waterworld.1995/

May 15, 2017 12:34:43.776 [0x7f8d1e638700] INFO - Plex Media Scanner v1.6.1.3722-4955e31cf - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 - GMT -04:00
May 15, 2017 12:34:43.776 [0x7f8d1e638700] INFO - Linux version: 4.10.0-20-generic (#22-Ubuntu SMP Thu Apr 20 09:22:42 UTC 2017), language: en-US
May 15, 2017 12:34:43.776 [0x7f8d1e638700] INFO - Processor Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz
May 15, 2017 12:34:43.776 [0x7f8d1e638700] INFO - /usr/lib/plexmediaserver/Plex Media Scanner -v -r -s -p -c 1 -d /mnt/nm/W-Z/Waterworld.1995/

Am I missing something or is this actually how this should function?

Any change you make should only be for TV Shows library category

@sa2000 said:
Any change you make should only be for TV Shows library category

I’m not sure what you mean.

@eldon.mcguinness said:

@sa2000 said:
Any change you make should only be for TV Shows library category

I’m not sure what you mean.

You have these library sections

Section 1	Movies
Section 2	TV Shows
Section 3	Music
Section 6	Music
Section 11	TV Shows (using a custom scanner?)

The issue with the extra metadata refresh after the scan, as far as I could see when using the parameters in your scan, only applied to TV Shows. So it is TV Shows Library Category that needed addressing. And any tweaks you make to try circumvent the problem we investigated should be applied to TV Shows Library scans

I think you misunderstood what I meant. I was using a script to find files with a modtime of less than 24 hours, then getting the directory of said files and telling the plex scanner to scan them in. I was doing that in both the TV and Movies libraries.

In the case of TV we have seen what happens so instead of invoking the refresh -r flag I decided to just use scan and then once all the files were scanned run another command to refresh the metadata, which in turn scans the files I want it to scan and then some as we have seen. I can deal with this as instead of rescanning the extra files multiple times it just scans them once.

The issue I’m talking about now is related to how this scan method affect movies. When I try the same process for movies running the refresh on its own in the movie directory scanned does not pick up the unmatched movies. It is almost as if the movies section is doing what one would expect it to do, while the tv section does not.

Due to BOTH of these issues I simply made two different scan starters, one for tv and one for movies. I have included them below as that might be a better explanation.

TVUpdate

su - plex -c /bin/sh << eof
        export LD_LIBRARY_PATH=/usr/lib/plexmediaserver
        find /mnt/nt -type f -mmin -$((60*24)) -print0 | xargs -0 dirname -z | sort -uz | xargs -0 -I {} /usr/lib/plexmediaserver/Plex\ Media\ Scanner -p -s -c 2 -d "{}" 2>&1
        /usr/lib/plexmediaserver/Plex\ Media\ Scanner -r -p -c 2 2>&1
eof

MovieUpdate

su - plex -c /bin/sh << eof
        export LD_LIBRARY_PATH=/usr/lib/plexmediaserver
        find /mnt/nm -type f -mmin -$((60*24)) -print0 | xargs -0 dirname -z | sort -uz | xargs -0 -I {} /usr/lib/plexmediaserver/Plex\ Media\ Scanner -p -s -r -c 1 -d "{}" 2>&1
eof

As you can see in the TVUpdate section I run multiple scans, based on the results of the find command and after all those run refresh on the library to match the tv shows that were just scanned in. In the MovieUpdate section I have to include the refresh flag in the scan command as doing a refresh command like I did in the TVUpdate script after all the finds have been scanned will not actually tag the scanned files as it does with the TVUpdate.

I do hope this makes sense and you can see where the issue is.

The action is different for TV and Movies and what i investigated was specifically relating to TV Shows and any changes you make to how you run the command l,ine scanner jobs to get round the problem would need to be for the TV Shows library only.

I have not looked into issues for Movies partial scanning and any solution you find would be different from that for TV Shows. I think we are in agreement on this fact

any progress on this issue? i have the exact same behavior as @ EldonMcGuinness

pretty annoying so far. seems like an 5 year old issue?

these bugs makes the whole cli-options completely useless

@ChuckPa , this is what i’ve been talking the whole weeks about “metadata not getting loaded”

@sa2000 @ChuckPa

Can we get some info on this? Why is this being ignored? Its so easy to reproduce, surely this can be resolved without years of wait…

@sa2000 @ChuckPa

I’m not sure the Plex-team knows how much this actually impacts some of us. So I’ll chip in and report my frustration as well. Hopefully this will get fixed some time.

I have this working on Windows 7 in a script for both movies and TV shows…
My setup maybe different not sure…

edit; sorry I was in a rush…
I have it set up on the PMS server so I can right click the folder for a movie or TV show and select Run Plex Scanner.

image

Selection of a movie…

image

Selection of a TV series

Both of these folders reside on a NAS…

and if you watch the Alerts on the server…

image

I use it if I make a change to a Movie or Show that way I get to see the results.
Most regular scans of my libraries take hours…

How is this “script” different from envoking the plex cli scanner?

Can we see this script you are using?

I just entered a detailed description… and lost it in this forum, was just finishing up when I got a forum message and this message went blank.

I’ll try again when I some more time…

Thanks, please do - Im very interested in getting this solved, and Plex doesnt seem to care at all about this problem.

I wonder what i pay money for, support definitely aint one of them

giving this a try again…

Please read this entirely before you decide to use either of the scripts below. Test it out on files that are easily recovered if you decide to go forward.

The difference is that my script runs the Plex scanner for just the Movie or Series that I want.
The normal ‘Scan Library Files’ scans the whole library all directory/folders and files associated with it.
In my case that is thousands of files taking hours to complete and it most likely won’t scan the move/series that I am working on first.

So if I am adding a new movie or episode to the library, I set it the way I want it and then have Plex scan it. It only takes a minute or so. and I get to check it and make changes if necessary and re-scan it if I need to.

NOTE:
I HAVE ONLY 2 LIBRARIES I USE THIS WITH AND THAT IS MOVIES AND TV SHOWS. IT IS SET TO WORK WITH THOSE TWO ONLY, IF YOU HAVE MORE THEN THESE TWO, ADDITIONAL CODING WOULD BE REQUIRED. THAT I WON"T GO INTO. I’M NOT A PROGRAMMER.

There are 2 scripts…
One is run to add the right click menu to the windows registry. This is done only once on the PMS server for setup.

It adds the line to the right click menu “Run Plex Scanner” and attaches the script/bat file to it that actually runs the process.
These files need to be located in a specific directory to work.
(if you move it you need to change all the path references in the files)
I have only run this on a Windows 7 systems.
My media is located on NAS units that are mapped by windows as drives and works great.

THERE IS NO ERROR CHECKING IN THESE SCRIPTS USE WITH CAUTION. USE AT YOUR OWN RISK
If you intend to try these, make sure you create and EDIT the files in the directory/folders before you execute anything.

First script; modifies the Windows registry to add “Run Plex Scanner” to the right click menu.
this is the path and file name;
“C:\Users\plex\MyScript1.reg”

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\Run Batch script]
@="&Run Plex Scanner"

[HKEY_CLASSES_ROOT\Directory\shell\Run Batch script\command]
@="“C:\Users\plex\plex_scanner.bat” “%1"”

copy the above 5 lines to a text file and save it as MyScript1.reg in the correct directory/folder
(use notepad, other editors may add hidden characters causing the file to not work correctly)

Second script, the one that does the work…
“C:\Users\plex\plex_scanner.bat”

@ECHO OFF

ECHO “script to start the Plex scanner for a specific TV Show or Movie”
ECHO “Right click on the show or TV folder (top most folder) and click on “Run Plex Scanner””
ECHO " "
ECHO %~n0 was called with the following arguments:
SET args=%*
IF NOT DEFINED args GOTO END
ECHO " "
ECHO Directory/Floder selected %*
ECHO " "

rem set longString=the variable containing the directory/folder selected
set longString=%*

rem ECHO longstring %longString%

rem replace xxxxxx with the string you are looking for in this case it’s movies else it will be shows
set tempStr=%longString:movies=%

rem ECHO tempStr %tempStr%

rem set the section for movies section 3 and tv shows section 4
IF %longString%==%tempStr% (SET section=4) ELSE (SET section=3)

ECHO section %section%
ECHO directory %*

rem this does the work
“C:\Program Files (x86)\Plex\Plex Media Server\Plex Media Scanner.exe” --verbose --scan --refresh --section %section% --directory %*

:END
PAUSE

copy the above lines from @ECHO OFF to and including the PAUSE to a text file and save it as plex_scanner…bat in the same directory as above.
(use notepad, other editors may add hidden characters causing the file to not work correctly)

IMPORTANT:
The second script file needs to be edited to match your movie and TV show section in Plex.
Easiest way is using the web client select the Movie library and in the address bar at the very end you should see the number shown similar to %3D1&context=content.library&source=3, source=3 is what your looking for, that’s the section, make a note of the number. and do the same for the TV Shows. The numbers can be anything not sure what the range is and there is no order.
Now edit the lines;

rem set the section for tv shows section 4 and movies section 3
IF %longString%==%tempStr% (SET section=4) ELSE (SET section=3)

changing the numbers to match yours; the moves are the number 3 for me and TV series is number 4,
also it looks for the word movies in the path, if it’s not found it assumes the folder is a TV show, If your movie paths would be different you need to make the necessary changes.
and save the changes.

When the second script runs it opens a CMD window and shows the progress it makes.
The script is complete when you see the press any key to continue.

NOTE: sometime it can takes minutes to run depending on what is going on with PMS, you can monitor it by watching the Alerts from the Web client. It must be run against the TOP level directory/folder for the single movie, not the file, and the same for the series. Plex Scanner uses the directory/folder name to preform the scan, for shows the alert screen will show each season it scans through. and when it finds the new episode that was added it will show created in the Web client alert screen.

OK you have created the files whats next…
first double click the “C:\Users\plex\MyScript1.reg”
you will get the standard registry warnings…
when it’s completed, takes only seconds, you should be able to right click any directory/folder and see the “Run Plex Scanner” listed,
“DOING SO ON ANY DIRECTORY/FOLDER OTHER THEN A MOVIE OR SERIES THAT IS ATTACHED TO PLEX WILL HAVE UNKNOWN RESULTS”
“DO NOT SELECT MORE THEN ONE DIRECTORY/FOLDER ITS NOT SET TO HANDLE IT”

USE:
Open a Web Client alert window for monitoring the process, and using the Windows Explorer on the PMS system select a movie folder from your movie library directory/folder and right click and click Run Plex Scanner, a CMD window should open and the process should start. It may take a few minutes. its completed the process when press any key message appears. it’s done and Plex should be doing it’s job on the selected directory/folder.

If anyone would like to add, streamline the process etc… feel free…
I hacked this together to fill a need that plex was lacking…

it’s great when that episode your waiting for is finally available, right click the show and get it added to Plex…

Thanks for the detailed reponse, unfortunately it doesnt seem like you do anything we dont already do - So most likely you just dont have any other unmatched items outside the folder you are scanning :frowning:

I just know it’s the easiest way for me to add and update Plex…

I do have the auto scans that Plex kicks off all disabled…

I use my process, I manually start library scans depending how much was added over time and then I clean up the databases.
I have not had it scan anything other than what was selected.

If any progress on this issue?

Any progress on this? Please…?

1 Like

Plex Media Server 1.19.2.2673 is now available to in the Beta update channel.

FIXES:

  • (Scanner) Partial scans would not work correctly on photo and new music libraries (#10877)

Thanks for the fix, but it doesn’t work on my ubuntu plex server.
The command is:
sudo -u plex “/usr/lib/plexmediaserver/Plex Media Scanner” -s -c 4 -d “/volume1/data/Artists/Album”

Plex log add an entry “scanning Artists” but do nothing.