Metadata agents for exported XBMC library

I found the bug causing the missing fanart for TV-Shows and sent Guillaume a pull request that will fix it.

Just out of curiosity, why would you want to use the nfo based meta data for tv-series? The TVDB based built-in TV-series agent does it's job very well and returns perfect meta data at least all my tv-series. If you had something missing from the TVDB you could always contribute it there. If you use Filebot to rename your episodes to match Plex's recommendations you will get 100% sure result since Filebot also supports TVDB as the metadata source.

The same cannot be said for the movie agent for which reason I built my own DVDProfiler based agent and fully understand why you would want to manage your own movie meta data. Just saying that for tv-series you will get perfect meta data with the built-in agent with zero effort ;)

This is to SirG regarding my post a couple of posts back.

The reply box just ate my reply, which I had interlaced with a quote of your post.  So here goes in order of your post.

I was in plexweb.  Since tested with Toshiba thrive android tablet and did not see any placeholders for missing art.  Have not tested on my Logitech Revue, may be a few days.

Backgrounds would be nice for TV Shows, but not a deal breaker. [edit] Great news in your last post.

Brain fart on the fanart naming convention.

I am seeing screencaps from my tbns at the episode level.

Since my previous post I scanned the tv-shows on my slow NAS,  41 series, between 3 and 136 episodes per series, so probably at least 1000 episodes, and it only took a little over an hour.  That is tremendously faster than the first time I scanned them with all info being fetched from the internet.

Thanks to you and Guillaume for the good work you have done.

Just out of curiosity, why would you want to use the nfo based meta data for tv-series? The TVDB based built-in TV-series agent does it's job very well and returns perfect meta data at least all my tv-series. If you had something missing from the TVDB you could always contribute it there. If you use Filebot to rename your episodes to match Plex's recommendations you will get 100% sure result since Filebot also supports TVDB as the metadata source.

The same cannot be said for the movie agent for which reason I built my own DVDProfiler based agent and fully understand why you would want to manage your own movie meta data. Just saying that for tv-series you will get perfect meta data with the built-in agent with zero effort ;)

TVDB is too rigid for all my series.  TVDB does not have all my series, not even by a longshot.  Although I have created entries on TVDB for a couple of dozen series where their rules fit the reality of the series in question, some of their rules are unacceptable to me. The worst has to do with getting rid of PBS Masterpiece and all of its sub-series.  They only allow a single series to be indexed to its first broadcast.  I would rather have my series that appeared on Masterpiece indexed under Masterpiece rather than have to index each series that showed in Masterpiece in the US under its unique series name.  Another problem, sometimes also showing up with Masterpiece series, is that the show may have a very different number of episodes in the TVDB than were actually broadcast in the US.     In addition to numerous Masterpiece series this applies to are all sorts of documentary series like "Life", "Earth", "Frontier House" and dozens of others.  They also sometimes arbitrarily change the season and episode numbers for a series that is many years old.  Some of my series they will not accept because there is no authoritative source of data except in the tivo file that I recorded years ago.     I could keep on going.

Amazing plug-in! Thanks.

@all: The fix for TV-Show fanart display and another one for episode duration is now pulled into Guillaumes repo. Have fun. :) And please report problems if there are any.

@henris:

My first reason is, that I want full control over what poster, fanart, ... is used. I do scrape TheTVDB data, but I do it using Ember Media Manager and manually overide what I don't like. Everything is then saved locally. Scraping local data is much faster than pulling everything from the web. The second reason is, that from time to time one will have to do a full rescan for whatever reason. (I have to do at least). Doing that without the local data will generate unnecessary traffic to the TheTVDB again, will take longer and one will have to do all manual overriedes for posters, ... again and again.

@greybeard:

I'm glad you like the agents. I did only some small tweaks here and there, the most work was done by Guillaume. :) If your problem with the placeholders in PlexWeb still exists, please post a screenshot. I still can't imagine where/what the problem is and this might help to locate it.

Thanks for the responses. Sounds like the main reasons for not using the built-in agent for tv-series are the same as I have for movies; lacking meta data in the online databases, need to have full control on the used meta data and media center software independent way of storing the meta data locally. I've been collecting movies for over 10 years but tv-series only for the last year. Since we are living in non-English speaking country we need good subtitle support. And this guides us to watch only mainstream shows which have also good meta data support in online databases.

But I'm also concerned about the availability and integrity of the meta data in the long run (>5-10 years); for movies I've all the data in DVD Profiler database from which I can always produce an xml-export with all the relevant content. But DVDP does not handle tv-series at all.

I took a look on the mentioned Ember Media Manager and it seems to have a very nice support for the initial scraping and further modifications. But I don't like the idea of distributing my meta data to hundreds of directories using the XBMC nfo-concept. If I manage the meta data centrally I would also like to keep the meta data centralised. It would be extremely easy to access this centralised but local database with an agent, either directly or through some export mechanism.

This leads to actual question: Is there a media manager with tv-series support (theTVDB, RageTV, etc.) and with a centralised database? Sqlite is mentioned with Ember Media Manager but I couldn't find any details for what it is used for.

To SirG,

The latest rev does display fanart.jpg as background on the main series info page in plexweb, but not in android tablet client.  Still have not reconnected my Logitech Revue or my Roku to check them.  However, of the 5 series in my test, only three have fanart, but only 2 show the right fanart, while 1 shows the fanart for another series.

Also, Image placeholders are still showing up.  I am posting 2 screenshots, 1 with right fanart another with wrong fanart, but both with Image PH.

I do not have time this AM to go through the logfile to see where the wrong fanart may have been added, but I am attaching the logfile.

@greybeard:

Thanks for posting the screenshots. The image tag below the year is for the rating the series received. E.g. TV14 or TVPG. The image tag below the plot is for the logo of the channel the series premiered on. Will later post a screenshot of how this looks. If the .nfo does not contain this info the tags should not be shown. So it seams that, while "Heat Seekers" has only channel info, "A Cook's tour" has both, but the icons won't display. I'll check your logfile and report back later, if I get a clue whats going wrong.

The same goes for the fanart.

Also please do recreate the test section and try again. You'r runtime info in the screenshots still says 0 min. This should be fixed in the latest version of the agent, but it may be, that a deep scan is not sufficient to populate this info.

SlrG

I was wondering about the 0 minutes on the main show page.   I just checked Heat Seakers and tvshow.nfo has 

30
 

Does that need to be in ms?

.  As far as rescanning the test library, I am deleting it everytime and then re-adding it.  If I do not do a deep-scan does that leave old cruft in the sqlite db?

@greybeard

Attached is the promised screenshot with rating and studio images.

To check if fanart is attached to a wrong series for me too, I deleted my full library and did a rescan. All my fanart (74 different series) is assigned correctly. I had not yet the time to check your logfile.

Deleting and re-adding should be the best way to avoid any old cruft. :)

Reading the wiki, XBMC expects the runtime in the nfo to be in minutes. So I am responsible for the agent not correctly recognizing your runtime. I create my nfo files using Ember Media Manager and for TV Shows it saves the runtime in seconds and so I made the agent work with seconds. As the agent is named XBMC importer, this is wrong and I'll correct it asap.

IMHO the tag should support something like 1h40m, 30m, 1800s, ... Having only the number in there makes it difficult to know the real meaning.

But as this is not defined I'll try to improve the recognition anyway.

@all

What do you say about the following pattern?:

values below 2 - should be time in hours (I don't know a TV Show with a larger runtime, do you?)

values larger but below 120 - should be time in minutes

values larger but below 7200 - should be time in seconds

all larger values will be time in ms.

At the moment the runtime of the last scanned episode decides what is displayed as episode duration by plex. I guess it would be better to calculate an average value and store this instead, wouldn't it?

More on the image placeholders.  A Cook's Tour, which shows 2 of them, the tvshow.nfo has a field TV-14 and a field Food Network . The scanner is creating a place for an image for each of them, but the correct  images did not get linked in.  Is that what looks like is happening?  Since plexweb is running in opera and I mostly use firefox I am fumbling around trying to find the source for the screen elements, but I tried right clicking on the lower-right emplty image and got a screen of source for that.  Here is the critical part of what I got:

 

When I paste the above URL into a firefox tab I got a black page with a grey squate in the middle of the screen, probably about 40x40px.

Here is the same info for the rating image:

 

Since I am an old C programmer I have not been able to wrap my head around objects  but it looks like plexweb is trying to fetch the image from the database that is being managed by PMS but for some reason the correct image did not get stuffed into the structure.  That could be a Plexweb issue only, or even a problem unique to Linux or to Opera, etc..

I have looked at the database with my Android tablet, and there is not much data being displayed by it.  It doesn't do background, rating nor studio.  The Roku interface is horrible.  It shows less onscreen than the Android tablet app.  I cannot find the power supply for my Logitech Revue so have not been able to check with it yet.

As far as duration goes,  most of my American made tv shows have picked up a from TVDB, but these are usually the time with commercials, whereas the British series are frequently actual program time.  I think the best way to handle tv show duration is to let the person who created the tvshow.nfo set that.  IFF is a non-zero value, then use that.  If it is missing, blank or zero, then either do not display the field, or if that is too difficult, use a value of "varies" if it can be a string and either zero or best guess if it must be an integer.  I think trying to calculate it from the actual episodes only makes sense if all episodes are present and you could quickly scan for the mode of the duration.

@greybeard

All the agent does, is take the values of rating and studio tags from the nfo and puts it into the respective fields of the plex database. Plex then does assign the images for ratings and shows it recognizes. No modifaction in the agent will change that. The only thing that could be done would be to assign an empty value if Food Network is recognized. Could you try changing the studio to an empty value by editing the show details in Plexweb? Does it still show an empty frame afterwards or no frame anymore? If a frame still is shown it would be meaningless to let the agent assign an empty value.

I put Food Network into the studio field of one of my shows. For me Plexweb doesn't show an image (and no empty frame) for this studio. Did you try different browsers for browsing the show? Is it maybe only opera which shows the empty frame?

Sadly I can't do anything about the other clients not showing all info Plexweb does. If it's possible, the developers of those clients will have to. :)

Runtime seems to always be displayed and needs a number value. Base for the average value I'm talking above will be the runtime provided in the nfo files. I'll run some more tests and then submit a Pull Request to Guillaume to have a look at it, too.

Plex expects/supports a runtime value in the tvshow.nfo and runtime values in the episode .nfo files. I'll change the agent to support this. XBMC theoretically should export these values. For EMM I'll ask the devs to support this.

Haven't rescanned the db yet, but I did try a bunch of different browsers.

linux ubuntu 12.04 32-bit, chromium: Cook's Tour only shows one image and it correctly displays as TV-14 graphic.  None of the other shows shows anything for rating or studio, except Money Vacation shows two graphical placeholders.

Win 7 Ult 64-bit, chrome:  exactly like lx Chromium.

Lx firefox:  TV-14 on CT, nothing on the rest.

Win 7 ff:  ditto.

I do not have opera on win or safari.  I will install them tonight o tomorrow and test.

I do have the Logitech Revue which runs chrome.  I finally found out how to expose the omnibox and type in a url.  The displays of the info pages are as with the other instances of chrome.  However I just tried play an episode of HS and it display on the root window or something behind a decorated plexweb window.  That is too bad, b/c of all the plex clients I have seen I like plexweb the best.  Google TV Plex does not scale videos to fit the window but seems to display at absolute px size.

I read a post somewhere saying that plex downloads this rating and studio/network images. Sadly it doesn't mention where this download happens and from where the images are pulled from. So no way to check whats wrong. :(

Please check your nfo files when viewing a show. If there is no rating tag or the rating is not recognized plex should show nothing. So if it is showing nothing for you, this could be the correct behaviour.

The same goes for the studio/network. If there is no or empty tag or if it isn't recognized, nothing schould show.

So reading your above post the placeholders now show only for Money Vacation? (In the screenshots it was for Cooks Tour, too.)

I read a post somewhere saying that plex downloads this rating and studio/network images. Sadly it doesn't mention where this download happens and from where the images are pulled from. So no way to check whats wrong. :(

Please check your nfo files when viewing a show. If there is no rating tag or the rating is not recognized plex should show nothing. So if it is showing nothing for you, this could be the correct behaviour.

The same goes for the studio/network. If there is no or empty tag or if it isn't recognized, nothing schould show.

So reading your above post the placeholders now show only for Money Vacation? (In the screenshots it was for Cooks Tour, too.)

The Studio .png images are found in the Media-Flags.bundle in your plugin folder. The images for content ratings are there too. To add a new studio logo find the logo for your missing studio on google make it white with a image editing program like paint.net and place it in the studios folder. Than go to the substitutions.xml and add in lines for your new studio

 

match name is the name of your .png file and expression is your studio tag in plex. make sure to add a few more lines with different expressions for any variation that may come through the scanners eg. bbc_three. you only have to add lines to the substitutions.xml when adding new studios. you will also need to do this to ensure changes aren't overwritten.

Great to get a full explanation of the studio and rating images.  I found two copies of the images on Linux, one descended from /var/lib/plexmediaserver and another descended from /usr/lib/plexmediaserver.  there was an image for foodnetwork, but I did not not discover substitutions.xml, so I was going to symlink foodnetwork.jpg the alternate stringnames.jpg, and add images for missing networks directly into the media folders.

I installed safari and opera on Win 7.  Opera behaved better there on CT.  For Money Vacation I added a random studio name and re-added the folders and it still showed a placeholder,  I fixed it by moving the closing tag to the same line as the tag and re-adding.

@jkmcn

Thank you very much for your help. :) Very valuable information.

@greybeard

I'm glad you were able to fix your problems.

I'm still working on the corrected runtime recognition. Nearly done, though. :)

This is to jkmcn if you are still following this.

When I change a distributed file in an app or os I like to save a copy of the original with "-DIST" or "-ORIG" as a backup in case I break something  if I do that in the Media-Flags.bundle will files whose names end with either of those strings be ignored by PMS or will PMS try to do something with them thus messing up the program?

New version with fixed runtime recognition is in Guillaumes repository.

If a runtime is given in the tvshow.nfo, it is used to set the runtime on the series page. If not, the median value of the episodes runtimes is used. Runtimes for the episodes are taken from the respective episodes .nfo file.