[Rel] HTTP Anidb Metadata Agent (HAMA)

Dev Community Dev/API Corner
121

Yeah you would think a refresh with no posters would attempt to look for posters though? Just had fun getting some unmatched series to go through hate series naming conventions

Alternative universes really cause chaos lol

Classic series Tenchi Muyo! Ryo-Ohki http://thetvdb.com/?tab=series&id=80009&lid=7 1 series

vs

anidb listing as 3 series (nothing unusual just naming convention of having (airdate year)

http://anidb.net/perl-bin/animedb.pl?show=anime&aid=205 Tenchi Muyou! Ryououki

http://anidb.net/perl-bin/animedb.pl?show=anime&aid=2509 Tenchi Muyou! Ryououki (1994)

http://anidb.net/perl-bin/animedb.pl?show=anime&aid=2510 Tenchi Muyou! Ryououki (2003)

Scan went through it finds the folders find just doesn't like matching them and leaves me with just the first series http://anidb.net/perl-bin/animedb.pl?show=anime&aid=205 Tenchi Muyo! Ryo-Ohki. To get round it had to take away the brackets on the folders so they read Tenchi Muyou! Ryououki, Tenchi Muyo! Ryo-Ohki 1994 and Tenchi Muyo! Ryo-Ohki 2003 as it didn't like looking inside the brackets with the folder names

122

@shikaiwraith: thanks for the descriptive answer as always.

Naming convention with years

=======================

manually searching for the serie name with "title (year)" should work, i just receive the title without from plex...

you can custom rename with aid:2509 without renaming or custom search the title with the year, will pick up the right one...

maybe we can modify the scanner to not remove the year...

i could use the manual search variable to use the folder name to check for (year): path1 = os.path.dirname(media.items[0].parts[0].file). if year is stripped and put in year variable, i could reinject it... trying that on git at the moment

http://dev.plexapp.com/docs/agents/search.html#Media

 

Check update behaviour:

=====================

http://dev.plexapp.com/docs/agents/update.html

It look like first scan or refreshes are calling update functions, so it's up to my agent.

The agent update if it's any different from what's already there and could not spot anything wrong with the poster part, if an issue, it seem it lies with Plex framework handling posters.

I didn't pass along the "force" bolean variable so am not refreshing when i should. passed the string, need to implement forced refresh now as i always updated but... i update any different from the XMLs so actually wasn't relevant

 

Left to do:

========

   . remove poster caching since plex won't cache it [DONE on git]

   . fix the "title (year)" series: test if i can re-inject the year stripped by Plex

   . fix the error you were getting (need new log with it appearing with the added log in git, just update __init__.py)

   . implement local xml cache and refresh if missing eps titles.

   . make it compatible with unofficial app store https://github.com/mikedm139/UnSupportedAppstore.bundle

123

Okay, so the steps that I understand them:

Download and install the .bundle directory into AppData\Local\Plex Media Server\Plug-ins

Add the directories to AppData\Local\Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama

If needed, update the __init__.py file in AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code

Is there any step I'm missing?  Plex doesn't seem to recognize that the agent exists.

124

The agent should be copied and moved in such a way the core python file is located here:

   . C:\Users\XXXXX\AppData\Local\\Plex Media Server\Plug-Ins\Hama.bundle\Contents\Code\__init__.py

There should be other bundles in the "Plug-Ins" folder already there.

App data is a hidden folder.

Make sure the .bundle include contents folder and not another .bundle folder

The agent detected after restarting plex

The data folders are optional (now).
125

@ZeroQI: That's where the file was in the first place and I just copied the linked file over on the off chance that's what I needed to do and it's still not working.  I've restarted my Plex a few times and it's not finding anything.

126
Uploaded on Git new __code__.py:
   . Download series without crashing
   . solved TypeError: not enough arguments for format string (Log.Debug missing argument when no poster present)
   . solved 404 error when mapping file point to outdated thetvdb
   . renamed data folders to Plex, AniDB and TVDB with the default xml at root, and anidb/TVDB xmls for series in their respective folders. TVDB was missing a seasonswide folder.
 
tried to handle series with year, but i received the serie name without year even if present in the folder name, and media.filename is equal to None, so cannot fix on agent side.
I could modify BABS to handle "anidbid" files to correct, while series can be corrected with aid:xxxxx as title name during manual search... Best way would be to see if scanner cause this and fix it if so...
 
[EDIT]: Force refresh on 394 anime series: no error. Code is stable and on github currently
Please let me know ANY crash experienced in logs as we need stability above all.
 
I changed the data folders (optional, cannot be created by the agent)
I have updated the git readme with the folders, and corrected a mistake, i was missing hte DataItems folder...
/volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems
       
  • "AniDB"
    •    
  • "Plex"
    •    
  • "TVDB/_cache/fanart/original"
    •    
  • "TVDB/fanart/original"
    •    
  • "TVDB/fanart/vignette"
    •    
  • "TVDB/graphical"
    •    
  • "TVDB/posters"
    •    
  • "TVDB/seasons"
    •    
  • "TVDB/seasonswide"
    •    
  • "TVDB/text"
    •  
    @Zandroid: i have no clue why only you (to my knowledge) is not working with my agent (other agents ok?)
    I would have liked more information, so it could benefit more people. enable debug logging in Plex settings, restart plex, then:
       . attach logs: PMS Plugin Logs/com.plexapp.system.log & com.plexapp.agents.hama.log [attached as a file, so the thread is readable]
       . Plex Server version
       . OS version
     
     The only way to fix if i cannot find the reason after you did provide the above would be remote control (can use teamviewer after hours or web tool called WEBEX during office hours[dell-emeaeec.webex.com]). If you solve while gathering, please post anyway
    127

    I recently went through and pushed my anime movies into their own library section; however, I've had some issues with HamaMovies.  I updated to the latest version, created the proper folders, and I've been having issues with it properly parsing and retrieving metadata for said movies.  Here's the error that it runs into for Initial D Battle Stage:

    2014-05-31 22:19:24,591 (eec) :  CRITICAL (core:561) - Exception in the update function of agent named 'HamaMovies', called with guid 'com.plexapp.agents.hama://11?lang=en' (most recent call last):
  File "C:\Users\Will\AppData\Local\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 970, in _update
    agent.update(obj, media, lang, force)
  File "C:\Users\Will\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 991, in update
    def update(self, metadata, media, lang, force ): self.parseAniDBXml(metadata, media,   lang,      True      )
  File "C:\Users\Will\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 617, in parseAniDBXml
    elif "s"+defaulttvdbseason+"e"+str(epNumVal) in tvdbSummary:    tvdb_ep = "s"+defaulttvdbseason+"e"+str(epNumVal)
  File "C:\Users\Will\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 844, in getImagesFromTVDB
    Data.Save(filename, poster)
  File "C:\Users\Will\AppData\Local\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\attributes.py", line 17, in __getattribute__
    return object.__getattribute__(self, name)
AttributeError: 'Movie' object has no attribute 'seasons'

    I'm not sure if I'm messing something up, but I get this problem regardless of what movie that I attempt to scrape.

    128

    @aikouka: I worked mainly on the TV part to do both series and movies. This error is due to the fact it is a movie but a serie special on thetvdb side so have season posters while the metadata for movies doesn't allow it. have corrected code in getImagesFromTVDB, it is on Git right now. [EDIT: just uploaded a new version with correct posters, didn't add them previously is cached already]

    Initial D Battle Stage however show only the AniDB ID poster, have to check why...

    Also tvdb removed duplicate and scudlee corrected mapping file for Groove Adventure Rave

    129

    What should i add?

        . XBMC AniDB mod has movie poster and fanart retrieval from themoviedb.org. Movies linked to TV shows could fetch both movie and TV fanart, prioritizing movie.

        . Shows distinguished by a year in brackets (e.g. Bakuman. (2012)): i need info, can we leave the year in the title in the scanner ?

         . Local cache: manually refresh tvdb xml when episode title is missing...

         . Complete the Collection feature? i can load the relation graph to add

         . Any other idea ? (actor name in kanji was requested, but would have raised the number of xml to download per serie dramatically) 

    Side post about Anime download

    ==========================

    Sickbeard for tv series is great, but Sickbeard anime branch is not updated to the latest but works...

    http://sickbeard.com/forums/viewtopic.php?f=4&t=1452

    Sickbeard custom (from synology package tracker url http://packages.synocommunity.com?beta=1)

      . Fork URL: git://github.com/lad1337/Sick-Beard.git

       . Fork Branch: anime

    For windows use http://sickbeard.lad1337.de/

    the developer for the anime branch did a plugin for XDM but can't get it to work...

    I was thinking, would you guys like a RSS link in missing ep htm log ?

    It would look like:

       . NZB: http://fanzub.com/rss?q=one+piece+1080p+horriblesubs

       . Torrent: Nyaa.se: http://www.nyaa.se/?page=search&cats=0_0&filter=0&term=one+piece+horriblesubs+720

       . Torrent animesuki: http://animesuki.rut.org/rss.php?style=alt (can't give proper example, blocked at work)
     

    I could use a local mapping file for res+team per serie like "anidbid,resolution,release_group" or "anidbid,searchstring". you can add rss manually into Sabnzbd making sure to select prevent duplicate downloads to have them processed

     
     

    http://forum.xbmc.org/showthread.php?tid=142835 show link to AniDB rename utility [anidb.net/perl-bin/animedb.pl?show=applet]

    http://thexem.de/xem/show/1245 allow to see mapping between the tvdb and AniDB

    130

    Side thread: testing SickRage (thanks to TrazgodX for the tip) on my fake synology box (XPEnology) using "sickbead custom" from SynoCommunity package

    SickRage

    ==========

    I had some issues with anidb main serie names, using rage indexer (supports tvdb and rage)

    However it support sports (which tvdb does not do) and tv series in one, and has nyatorrent downloads

      . GitHub URL:    https://github.com/echel0n/SickRage

       . Report issues: https://sickrage.tv/forums/forum/help-support/bug-issue-reports

       . Forums:           https://www.sickrage.tv/forums/

       . Fork URL:        git://github.com/echel0n/SickBeard-TVRage.git

       . Fork branch:   master

    Uses tvdb indexer (so no animation movies like akira) so doesn't find stuff, and filename recognition is low (need to rename all eps)

       . GitHub URL:    http://github.com/lad1337/Sick-Beard/tree/myAnime
       . Report issues: http://code.google.com/p/sickbeard-myanime/
       . Forum:         http://sickbeard.com/forums/viewtopic.php?f=4&t=1452&p=6377
       . Fork URL:      git://github.com/lad1337/Sick-Beard.git
       . Fork Branch:   anime
     
    Doesn't work with sickbear custom, but uses AniDB indexer
       . GitHub URL:   https://github.com/GrandpaTetrapod/Anorak
       . Forum:           http://forums.sabnzbd.org/viewtopic.php?f=6&t=17162
       . Fork Branch: master
       . Review:          http://usenetreviewz.com/anorak-review/
     
    You can follow anime using rss:
       . [torrent] http://www.shanaproject.com/ to create custom rss to follow each series, that your favorite torrent client can follow
       . [torrent] http://www.animesuki.com/doc.php/rss.html see http://www.animesuki.com/doc.php/rss.html
       . [torrent] nyaatorrent.com see http://anilist.co/forum_post?id=432 on how to create rss feed
       . [usenet] http://www.fanzub.com/
     
    Should i include in the episode missing html log links to the nzbs or rss for that serie ?
    131

    hiya guys (and girls). I am currently working on series with a year in the title:

       . Tenchi Muyou! Ryououki (2003)

       . Tenchi Muyou! Ryououki (1994)

       . Tenchi Muyou! Ryououki

    Plex give me the following title:

       . Unmatched yet: no year in the title (ie "Tenchi Muyou! Ryououki" in all 3 cases), and only a filename non empty string when i automatch. While i could solve for automatch, I could not for "Fix incorrect Match" first prompt

       . the matched serie name, so when checking and it's mapped it looks normal...

    In babs there is code to remove or clean the year:

            if cleanYear != None:
                file = file.replace(str(cleanYear), 'XXXX')
                print "BABS: replaced year, file = |", file, "|"
     
    I tried TheTVDB agent and similar thing happens:
       . The Outer Limits
       . The Outer Limits (1995)
     

    Can somebody with scanner knowledge see if we can leave the year in the title or when a year is present inject it back at the end?

    Both BABS and common scanner are impacted (ie can't match a serie with year at the end when a similarily named serie exists)

    132

    Coded a bit since last week and yesterday:

       . partial titles with year fix (Works in Auto-Match mode)

       . Increased movie poster source: TMDB posters (using both tmdbid {rare} and imdbid {more common} in mapping file)

       . Increased movie poster source: IMDB poster (through OMDB API)

       . Posters for series considered as seasons in TVDB should have different posters now (sort_order needed to be a string)

    To Do:

        . i want to clean the code and do a proper stable release. most functionalities already there, shouldn't change much

       . improve collections, might involve forking the movie collection to add series... your ideas? do you use collections? would you use it if it worked decently ?

    Need feedback on:

       . a report of ANY issue. my only feedback is this form, period. The source code was made hightly readable (because i couldn't read other metadata agents too well). please feedback any mods you do as i would be interested

       . any combination of agent and scanner that can recognise a serie or movie (anime or not, movies or series, doesn't matter) with a year in parenthesis when an identical named serie exist with no year ("The Outer Limits (1995)" and "The Outer Limits" for example) that you ever witnessed so i replicate for HAMA

       . anime studio icon pack (can be edited in Plex source directly)

       . functionality improvements (People names in Kanjis is noted, but on hold for the moment)

    133

    I was trying to solve the year in the title recognition issue fully using Tenchi Muyou! Ryououki 1992 1994 and 2003

    by default, all are grouped into one so i split the series and all share the 1992 version info

    if i unmatch a serie, and modify the title to include the year, i click "Fix incorrect match" and it show automatically the original title 

    matched (not the one with the year) as a result (had issues with the first screeen being fixed, and could not change it, seem to be a cache issue, resolved now). if i go to custom, it shows the year in the title

    I have modified my agent to use match if the string is contained in the search, not only the exact match, so the sequels with years show in the custom search, looks great and make it very easy to correct issues

    Also, movies in series get info pulled ok (although the filename seem to match sereis with 01 afterwards FOR NOW)

    To DO (not sure i can, it involves scanners, help welcome)

    ======

       . Allow movies (single OAVs) in the series. the code works, pull a poster from anidb, tmdb, imdb for princess mononoke, BUT needs an "01" after the name so the scanner accepts it. definitely in the scanner code, we need to make single movies episode 01 (or not ?).

     
       . Allow the use of AniDB membering for specials in the scanner by translating to the s00exxx range concerned since the agent already supports it (s00e101 is opening 1 if i recall well)
     

       . Allow year in title, as Plex remove the year when handing it to the agent, and if series have the same name without the year (tenchi muyo Ryouoky) it group them in one. possibly in the VideoFiles.py > VideoFiles.CleanName() function here:

     https://github.com/plexinc-plugins/Scanners.bundle/blob/master/Contents/Resources/Common/VideoFiles.py#L164

    It seems we can create a folder Scanners > Common >CustomVideoFiles.py and modify in the scanner "import Media, VideoFiles, Utils"  to add "CustomVideoFiles, CustomStack" along with replacing "VideoFiles.CleanName" with "CustomVideoFiles.CleanName"

    Source: https://forums.plex.tv/topic/58512-how-to-overrideupdate-videofilespy/?hl=videofiles

    Source: https://forums.plex.tv/topic/53368-custom-plex-scanners/

    I speak and write french english and spanish in that order. i will work on hama and look into the scanner, but it is my first python project (which i forked from Atomic Strawberry, thanks again dude), so compliments, critics welcome, and if you know the best way to select 

    134

    Can you please tell me why it matches but does not yield any info, posters, etc.? log attached

    135

    it crashes miserably as in urlLoadXml it cannot save the cache due to missing folder...

    I have updated the code on git but am unsure if it will solve the issue.

    Please go in "C:\Users\Ofek\AppData\Local\Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama\DataItems" and add the folders indicated the readme for HAMA whose link is in my signature: (/ indicate a subfolder is inside)

    • "AniDB"
    • "Plex"
    • "TVDB/_cache/fanart/original"
    • "TVDB/fanart/original"
    • "TVDB/fanart/vignette"
    • "TVDB/graphical"
    • "TVDB/posters"
    • "TVDB/seasons"
    • "TVDB/seasonswide"
    • "TVDB/text"
    136

    I missed a few folders... thank you!

    137

    I just installed this and BABS. When I'm using it however, it defaults to the Main Title instead of the official English title. I would prefer to see the English title if at all possible. I've tried Fix Incorrect Match and set it to HamaTV with English Language and still get Hagure Yuusha no Aesthetica instead of Aesthetica of a Rogue Hero as an example. Please help. :)

    138

    I just installed this and BABS. When I'm using it however, it defaults to the Main Title instead of the official English title. I would prefer to see the English title if at all possible. I've tried Fix Incorrect Match and set it to HamaTV with English Language and still get Hagure Yuusha no Aesthetica instead of Aesthetica of a Rogue Hero as an example. Please help. :)

    In the agent settings you can alter "First Language preference for the Serie title" from main to en to use english for the titles by default.

    Normally though I just edit the show and set the title to whatever I want it to be and it will lock that value for any refreshes.

    139

    ZeroQI, I have been trying to resolve an issue in the original thread but thought it would be best to continue on it here.

    Any ideas on why new episodes added to a "fixed match" folder keep showing up as the original auto matched show?

    140

    No, i don’t but i can search. I would originally think you are mistaken, but it could be a cache issue with plex solved by recreating the library. …


    I would need hama and scanner logs attached as file to this thread please and I will search. I need the Serie name and episode recently added


    I am also writing HAMA Scanner currently that has a better recognition rate, solve the year in serue name issue and semicolon in titles, supports dual digits épisodes without ep in front… and logging capabilities, with list of skipped files.


    I say that because if the Serie is recognised as the right one straight away, there is no more issue. It currently only work for synology due to absolute link used for access, I wish to address that by using relative path or a selection of fixed path to edit according to your is and pkex location.


    So attach the files and I will get crackin’