[REL] Extended Personal Media Scanner

Dev Community Dev/API Corner
1

See the documentation here for more details on the scanner and how to install it.


https://bitbucket.org/mjarends/plex-scanners

After you have installed the scanner you will need to install the metadata agent for the scanner to work correctly. See the documentation here for more details on the metadata agent and how to install it.

https://bitbucket.org/mjarends/extendedpersonalmedia-agent.bundle

Change Log:

4/4/2015: Moved the creation of the scanners to within the Scan method

5/29/2015: Added support for episode formatted files that contain episode release dates (ex: s1e1 - 2014-12-13 - Episode Title.mp4)

1/6/2016: Added support for two digit episode index numbers in date-based episodes (ex: \Show Title - s2012e09 - 2015-12-31_25 - Episode Title.mp4. (NOTE: Updating the plug-in and rescanning your library will result in new episode numbers being created)

6/29/2017: Removed functionality that sets the episode release date from the scanner to the metadata agent

8/22/2017: Added support for using the keyword Lesson, lesson, L, or l in directory and file names in place of the word Season, season, S, or s.

9/3/2018: Added support for the following features:

  • Added support for episode file names that contain date and time. See the readme here for more details on file naming: https://bitbucket.org/mjarends/extendedpersonalmedia-agent.bundle/src/master/

9/9/2020: Added support for the following features:

  • Added support for date and times that do not have spaces between the date and time parts (ex: YYYYMMDD or HHMMSS)

3/7/2021:

  • Fixed issue where the SeriesDateBasedMediaParser failed to parse media that did not use spaces between dates for the following pattern: `\Show.Title\20120919 - Episode Title.mp4`
2 Likes
2

I'd like to cast a vote for mp4 metadata reading. At the moment I use standard Plex Media Scanner but I hit the bug " multiple shows on the same date " every now and then.

I've reported the bug but was told that it wont be fixed, so I'd like to get rid of it ASAP. ( I wonder what people are doing with repeating daily shows which air more than once each day )

3

Hi Mitch!

First of all thanks for the early Christmas present!

Could not wait to test this scanner but had a hardware failure on my server :(

All good now, installed the new agent and scanner after I upgraded the PMS. Deleted my old section, created a new one with your scanner and agent but it would not find any media at all.

When I switched back to the series scanner they all appeared again...

Am I doing something wrong or has the new PMS version broken something?

Cheers

Franz

4

I've had this installed in the past, with no issues. I just re-installed it and restarted my Plex Server. For some reason it is no longer showing up within my Scanner sections. :( Should I rename it to just Extended Personal Media? 

5

Hi!

I'm having trouble with the agent using the episode name from other files. 

Here is the log for one example.

The two different files (from different shows) should be named 

Svennes stuga i Sandselet

and

Kreta

but they both get the name Kreta in Plex. 

2014-11-12 23:09:36,266 (f10) :  INFO (logkit:16) - update :: episode metadata: 
2014-11-12 23:09:36,267 (1dc0) :  INFO (logkit:16) - update :: episode: 1621
2014-11-12 23:09:36,269 (f10) :  INFO (logkit:16) - update :: episode file path: D:\hemmedia\video\Elisabeth och Bert-Görans videor\1990\1990-07-21 Svennes stuga i Sandselet.mp4
2014-11-12 23:09:36,269 (1dc0) :  INFO (logkit:16) - update :: episode metadata: 
2014-11-12 23:09:36,269 (f10) :  INFO (logkit:16) - unicodize :: before unicodizing: D:\hemmedia\video\Elisabeth och Bert-Görans videor\1990\1990-07-21 Svennes stuga i Sandselet.mp4
2014-11-12 23:09:36,269 (1dc0) :  INFO (logkit:16) - update :: episode file path: D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.mp4
2014-11-12 23:09:36,270 (f10) :  INFO (logkit:16) - unicodize :: after unicodizing: D:\hemmedia\video\Elisabeth och Bert-Görans videor\1990\1990-07-21 Svennes stuga i Sandselet.mp4
2014-11-12 23:09:36,272 (1dc0) :  INFO (logkit:16) - unicodize :: before unicodizing: D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.mp4
2014-11-12 23:09:36,273 (f10) :  INFO (logkit:16) - update :: absolute file path: D:\hemmedia\video\Elisabeth och Bert-Görans videor\1990\1990-07-21 Svennes stuga i Sandselet.mp4
2014-11-12 23:09:36,273 (1dc0) :  INFO (logkit:16) - unicodize :: after unicodizing: D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.mp4
2014-11-12 23:09:36,273 (f10) :  INFO (logkit:16) - update :: parser <__code__.SeriesDateBasedMediaParser object at 0x0337AF30> contains match - parsing file path
2014-11-12 23:09:36,275 (1dc0) :  INFO (logkit:16) - update :: absolute file path: D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.mp4
2014-11-12 23:09:36,276 (f10) :  INFO (logkit:16) - parse :: regex [\\/](?P[^\\/]+?)[ ]*[-\.]{0,1}[ ]*(?P[0-9]{4})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{2})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: None
2014-11-12 23:09:36,276 (1dc0) :  INFO (logkit:16) - update :: parser <__code__.SeriesDateBasedMediaParser object at 0x0337AF30> contains match - parsing file path
2014-11-12 23:09:36,276 (f10) :  INFO (logkit:16) - parse :: regex [\\/](?P[^\\/]+?)[ ]*[-\.]{0,1}[ ]*(?P[0-9]{2})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{4})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: None
2014-11-12 23:09:36,278 (1dc0) :  INFO (logkit:16) - parse :: regex [\\/](?P[^\\/]+?)[ ]*[-\.]{0,1}[ ]*(?P[0-9]{4})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{2})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: None
2014-11-12 23:09:36,278 (f10) :  INFO (logkit:16) - parse :: regex (?P[^\\/]+)[\\/](?P[0-9]{4})([-\. ]+(?P[^\\/]+)){0,1}[\\/][^\\/]*?(?P[0-9]{4})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{2})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: <_sre.SRE_Match object at 0x04FDD140>
2014-11-12 23:09:36,279 (1dc0) :  INFO (logkit:16) - parse :: regex [\\/](?P[^\\/]+?)[ ]*[-\.]{0,1}[ ]*(?P[0-9]{2})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{4})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: None
2014-11-12 23:09:36,279 (f10) :  INFO (logkit:16) - parse :: found matches
2014-11-12 23:09:36,279 (1dc0) :  INFO (logkit:16) - parse :: regex (?P[^\\/]+)[\\/](?P[0-9]{4})([-\. ]+(?P[^\\/]+)){0,1}[\\/][^\\/]*?(?P[0-9]{4})[-\. ](?P[0-9]{2})[-\. ](?P[0-9]{2})(_(?P[0-9]+)){0,1}[ ]*[-\.]{0,1}[ ]*(?P.*)\.(?P.+)$ - matches: <_sre.SRE_Match object at 0x04FDD260>
2014-11-12 23:09:36,280 (f10) :  INFO (logkit:16) - scrubString :: original: [Svennes stuga i Sandselet] scrubbed: [Svennes stuga i Sandselet]
2014-11-12 23:09:36,282 (1dc0) :  INFO (logkit:16) - parse :: found matches
2014-11-12 23:09:36,282 (f10) :  INFO (logkit:16) - setValues :: file name without extension D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta
2014-11-12 23:09:36,282 (1dc0) :  INFO (logkit:16) - scrubString :: original: [Kreta] scrubbed: [Kreta]
2014-11-12 23:09:36,283 (f10) :  INFO (logkit:16) - setValues :: looking for summary file D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.summary
2014-11-12 23:09:36,283 (1dc0) :  INFO (logkit:16) - setValues :: file name without extension D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta
2014-11-12 23:09:36,285 (f10) :  INFO (logkit:16) - setValues :: summary file does not exist
2014-11-12 23:09:36,286 (1dc0) :  INFO (logkit:16) - setValues :: looking for summary file D:\hemmedia\video\Inga och Arnes kamera\1997\1997-06-11 - Kreta.summary
2014-11-12 23:09:36,286 (f10) :  INFO (logkit:16) - update :: before setting season.summary: None
2014-11-12 23:09:36,288 (1dc0) :  INFO (logkit:16) - setValues :: summary file does not exist
2014-11-12 23:09:36,288 (f10) :  INFO (logkit:16) - update :: season.summary: None
2014-11-12 23:09:36,289 (1dc0) :  INFO (logkit:16) - update :: before setting season.summary: None
2014-11-12 23:09:36,289 (f10) :  INFO (logkit:16) - update :: episode.title: Kreta
2014-11-12 23:09:36,289 (1dc0) :  INFO (logkit:16) - update :: season.summary: None
2014-11-12 23:09:36,290 (f10) :  INFO (logkit:16) - update :: episode.summary: None
2014-11-12 23:09:36,292 (1dc0) :  INFO (logkit:16) - update :: episode.title: Kreta
2014-11-12 23:09:36,292 (f10) :  INFO (logkit:16) - update :: episode: 2101
2014-11-12 23:09:36,292 (1dc0) :  INFO (logkit:16) - update :: episode.summary: None
2014-11-12 23:09:36,293 (f10) :  INFO (logkit:16) - update :: episode metadata:  

From what I can tell from the log, they seem to run through the algorithm simultaneously which I believe is what causes the issue. 

This is just one example, there are many more in my library. 

6

schteff - Would it be possible to see the entire log? Based on the information above I can't determine the problem. As a side note Plex actually controls the number of threads that get called to read and parse media. The scanner plug-in is just parsing the file names and providing the information back to Plex. It's possible that there is another video outside of the details logged above that may be causing this issue.

7

I have temporarily "solved" it by creating a separate library per tv show. And i'm currently kinda busy looking through them all for a high priority project. But later in december I will try putting all the "shows" in the same library again. I will post the result then. Thanks for the reply!

8

Hi,

I just discovered this Scanner + Agent.

I am using Ubuntu 14.04 and Plex 0.9.12.0.

I have an issue with the Scanner. I followed the installation instructions, all permissions are plex:plex and 770. Agent seems to be ok but scanner does not seem to be installed properly.

When I choose the Extended Personal Media Scanner Plex says "Scanner A Scanner is  required" and does not allow to create or edit the Library.

9

Make sure under the Scanners folder you see the path Series/Extended Personal Media Scanner.py


You might also need to restart your server

10

Has anyone noticed you can't use this scanner with the newest Plex server build?

When I now try and use it to create a new library I get "Scanner A scanner is required"

Has anyone else tried to use this with the Version 0.9.12.1 version of the server?

Carlo

11

Carlo - I'm also running 0.9.12.1. I just verified on my server and the scanner is still showing up correctly under the sections where I'm using it. Did you recently re-install it?

12

It shows up but you can't use it.  Try creating a new test library pointed to something you already have setup and know to be working (a duplicate library).

You should see it won't work.

If you want a hurting then also try this.  Remove a file from an existing library managed by this scanner and delete trash and clean bundles.  Then add the file back and do a refresh.

You'll see that even on existing libraries previously setup under an older version of Plex it will now NOT use the scanner correctly and will try and load the file with a standard TV Show scanner.

This sort of problem has also been mentioned here: https://forums.plex.tv/topic/160013-pms-09120-doesnt-let-me-create-libraries-with-alternate-agent-or-scanner/

and I opened this: https://forums.plex.tv/topic/161021-quick-fix-needed-customer-scanners-broke-in-new-version-of-plex/

Carlo

PS I hadn't noticed this other thread before posting here. So it's not a problem with this scanner in itself but something in Plex "core".

13

Hi, does this read embedded MP4 metadata yet?

14

Hi, does this read embedded MP4 metadata yet?

There is already a local media metadata agent that Plex provides and can be used in tandom with the Extended Personal Media scanner/metadata agent. The local media metadata agent reads the tag data from the local files and sets them in the metadata within Plex. The extended persona media scanner/metadata agent adds the ability to use the file name to set metadata in Plex. Let me know if you have any questions.

15

Ah, ok. I have a recurrent issue with the existing Local Media Assets Agent and Personal Media Agent not scraping Release Year, full Genre listings, Long Description and full Cast Lists. Sometimes, it skips the artwork, too. Sometimes, it skips the entire cast. Most often, it ignores that Personal Media and Local Media Assets are set as the primary and secondary agents and that the only additional agents checked are Freebase, TMDB and TVDB and only so because they are force checked and cannot be unchecked. Removing the bundles for those items works until the server is updated or restarted, when they are recreated. So, then it gives me marketing log lines, horribly written plot summaries, bit players instead of the starring and supporting casts, partial genre listings, other people's ratings and everything in my library, including the silent films, were all apparently made in the year they were added to my server. Fixing all this by hand is annoying and a waste of time, especially, given that I already spent the time to fully and correctly add and embed all the metadata I want. It, also, refuses to acknowledge 2/3rds of the Twin Peaks Gold Box Set because my ordering is not the same as some online database or another that I don't want to use at all. So, I was hoping the Extended Personal Media scanner/agent was something that would more fully and correctly scrape user created MP4 metadata and, in so doing, help override the forced defaults to bad databases.

16

Hi guys,
I have two strange issues:
1/. I have a folder with mp4 files named like so: genre, NameOfTheShow - SnnEnn - title.mp4. None of these files appears in the Plex library

2/. I have named my shows with a prefix to have them sorted properly in the Plex Library. They are named like above with “genre,” at the start of each name. When I import those files, they are all added to a single show rather than creating separate ones. For example:
/ Files
    / Music
        / Show 1
                Music, Show 1 nice - S01E01 - my Tuto.mp4
                Music, Show 1 nice - S01E02 - my other Tuto.mp4
        / Show 2
                Music, Show 2 great - S01E01 - my song.mp4
                Music, Show 2 great - S01E02 - my other other song.mp4

These are merged in a single how and I don’t know why…

Any help would me much welcomed!

Thanks in advance guys!

17

@mrkaymrkay I tested the directory structure that you posted on my test system and I am seeing separate shows and episodes being created by the plug-in. Can you ensure that you have done the following:

  • Installed the Extended Personal Media Scanner - details are available at the top of this thread
  • Installed the Extended Personal Media Metadata Agent - details are available here: https://forums.plex.tv/discussion/83440/rel-extended-personal-media-shows-agent/p1
  • Create/modify your library to use the Extended Personal Media Scanner and Metadata agent by clicking on the library Edit button and then on the Advanced link and changing the two settings indicated by the screen capture below.
    1rwaprnhkdbi.png735×469 27.2 KB
  • Update your library in Plex using the Update Library button (shown in the screen capture below).
18

Hi mjarends and thanks for the feedback!

Both agent and scanner are installed and did work for another Library I made yesterday.

That’s very odd indeed…

19

I’m assuming that you are intending the show name to be “Music, Show name” and the title to be the part after the sXXeXx part. That’s the outcome that I got.

You can enable debug logging and I could take a look at the logs if you want.

20

Thanks,

I’ll do it in the morning and shall post it here
This is the cas where one of the show does not appear at all in the library.

I wil reproduce the other case with a smaller amount of files before posting the log

Thanks in advance for your help.

EDIT: Log added