[Rel] HTTP Anidb Metadata Agent (HAMA)

Dev Community Dev/API Corner
1581

Using the [anidb-xxxx] tag in the directory name should override basically anything you use for the folder name and force the match, though make sure the episode files then also use that name as well as it uses the folder name to eliminate some matching in the file names, if I remember correctly. I’m not at home so I can’t be sure but I think I have them named “Aldnoah.Zero” and “Aldnoah.Zero (2015)” with anidb2 tagging and they worked fine for me.

Plex Dance between renaming also is a good idea, as just changing directory name it still seems like it can sometimes recognize and hold on to the file between scans.

1582

@YamiTamashi i made a list of them in the readme. Hama agent logs. It’s good you have them but if you do not attach them i cannot troubleshoot.
@ma3ay “As I understand, agent tries to works with them as if they are the TV-series - with a logical CRITICAL result.”. You say that as it i a logical outcome when you are the one that put movies in your series folder and installed a scanner that allows it… IAt least you attached logs that shows it crashed but it’s a bug already corrected in the beta branch and you use normal branch

@OxymoRonald you should agent logs like the person before you. it should have work in both instances. Probably a crash. Are you using beta branch?

Status update

  • Beta branch is the only branch being worked on, will replace normal branch at some point. It allows to select the metadata source per field and have very readable logs and a source for full music themes. Also quite few bugs are resolved
  • Need to implement language priority across sources as currently it picks the source then pick based on language priority and for titles line chinese doesn’t work that well as it ends up in english despite a chinese title in the second metadata source for exemple
  • you can specify a different metadata source order for series and episodes by separating the common field like title in agent settings with pipe “|”
  • Stuck on scanner development now, doing an hybrid between legacy scanner and normal one so you get plex caching unless a grouping folder without the need to flag it as grouping folder

I can no longer access my Synology NAS at work as they tightened the security some time ago so it slowed development immensely. I am also left with little free time so create bug reports on github after having tried the beta branch and attaching agent logs and i will get to it at some point

1583

@ZeroQI said:
You say that as it i a logical outcome when you are the one that put movies in your series folder and installed a scanner that allows it… IAt least you attached logs that shows it crashed but it’s a bug already corrected in the beta branch and you use normal branch

No, they aer not in the series folder. Library type - Movies, scanner - Plex Movie Scanner. All other 70 movies in this folder were handled correctly.

1584

Maybe error is in anime-list-master.xml?
They are not marked as movies here - and have “tvdbid” set to corresponding TV-series instead:
<anime anidbid="9293" tvdbid="254663" defaulttvdbseason="0" episodeoffset="" tmdbid="" imdbid=""> <name>Mouretsu Pirates: Abyss of Hyperspace - Akuu no Shin'en</name> </anime>

<anime anidbid="11680" tvdbid="257516" defaulttvdbseason="0" episodeoffset="10" tmdbid="" imdbid=""> <name>Accel World: Infinite Burst</name>

And beta not workin for me completely - I don’t even have posters. Even for other movies like Blame! (2017) who worked well before. Maybe problem with settings?

1585

@ma3ay i only maintain beta now, can’t switch between two versions… It will replace main branch really soon
I will have more bugs in movies libraries since i don’t use it…

Thanks for the precise logs, could solve quickly thanks to that.

in AnimeLists.py line 56 it seems “if movie: folder = os.path.foldername(media.items[0].parts[0].file)” gives and error “os.path.foldername” doesn’t exist it’s “os.path.dirname” the right command…

2017-11-22 20:33:37,839 (1f78) :  INFO (logkit:16) - AnimeLists.GetMetadata() - source: anidb, id: 1050, AniDB_id: 1050, TVDB_id: 
2017-11-22 20:33:37,842 (1f78) :  CRITICAL (core:574) - Exception in the update function of agent named 'HamaMovies', called with guid 'com.plexapp.agents.hama://anidb-1050?lang=en' (most recent call last):
  File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-23901a099\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 1076, in _update
    agent.update(obj, media, lang, **kwargs)
  File "C:\Users\deadma3ay75\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 117, in update
    def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  True)
  File "C:\Users\deadma3ay75\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 88, in Update
    dict_AnimeLists, AniDBid, TVDBid, TMDbid, IMDbid, mappingList =  AnimeLists.GetMetadata(media, movie, error_log, metadata.id,                   AniDBMovieSets) #, AniDBTVDBMap
  File "C:\Users\deadma3ay75\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\AnimeLists.py", line 56, in GetMetadata
    if movie: folder = os.path.foldername(media.items[0].parts[0].file)
AttributeError: 'module' object has no attribute 'foldername'

if you can correct and try, it should go further, any issue, post logs like you just did as it correct bugs for everybody that way and will post an updated version afterwards

1586

Hey, I’m having some issues setting up dragon ball super with seasons but keeping absolute episode number, I have all the files in the root folder of the show and they all have absolute episode number in the file name, I tried using tvdb2 and tvdb3 to organize them in seasons and it did work but it doesn’t update the episodes titles to those on either tvdb or anidb.
My folder structure looks like this

/Anime  
    /Dragon Ball Super  
        /Dragon Ball Super - XXX - Title.ext
        /tvdb3.id

Then on plex it does sort by the tvdb2/3 standard, so it adds seasons and sets up each episode correctly to each keeping absolute number or season number (depending of tvdb2 or 3) but the episodes names are fetched from the file name and no episode description gets added. So it seems it manages to get the season/episodes list just fine, but it isn’t able to import the episode metadata?

If I just use the default anidb.id it fetches the titles and descriptions from anidb but then they are all one season and it gets a bit cumbersome to navigate

1587

Still don’t work. Even with Blame! (2017) - agent considers this an old series from year 2003 with the same name. And it can’t fetch nor poster, nor description.

Found in my archives old version of HAMA - from 2015-04-11. Works perfectly with all problem movies: Blame!, Accel World: Infinite Burst, Fate/Stay Night: Unlimited Blade Works, Mouretsu Pirates - Abyss of Hyperspace

1588

@ma3ay the more i improve series the more bugs pop up for movies since i am not using movie libraries…
i believe you miss write access to this file on your windows machine
IOError: [Errno 2] No such file or directory: u'\\\\?\\C:\\Users\\deadma3ay75\\AppData\\Local\\Plex Media Server\\Plug-in Support\\Data\\com.plexapp.agents.hama\\DataItems\\_Logs\\._Plex themes missing.htm'

1589

No, all directories are write-enabled. Create folders “Logs” and “_Logs” in “DataItems” just in case. Both are empty. All logs in DataItems are from stable version. Tryed to handle other movies with beta - no logs in Logs folders, no posters, no info. Completely broken for movies.

1590

I have a question: Could you consider enabling mixing data from both tvdb and anidb (or is it already possible)?
I have everything mapped according to tvdb and don’t want to change it, however I would like to get the japanese title and genres from anidb. My ideal case would be metadata from tvdb and genres + title from anidb.

1591

@arddor it is already done. Go to the beta branch and select the meta source for each field. for titles you can split between series and episodes with ‘|’ like: AniDB, TheTVDB | TheTVDB, AniDB
@ma3ay No logging line in code for movies, corrected that and uploaded to github beta branch, please test
Solved AniDB episode titles missing and handling languages better now

1592

@ZeroQI Awesome. It is working almost perfect, the only problem I have is with multi-season anime where the title is missing on episodes.

gpzii4qkej3x.png1248×696 601 KB
.

1593

@arddor can you give me the specifics so i can fix it? serie name, agent logs, and ep title priority string will sort the rest

1594

Just to reiterate, it isn’t possible to group multiple seasons by parts like I have now?

Since the Monogatari series structure is so confusing, it would be nice to group the series the way I have it now. Since the watch order isn’t actually S1 -> S1 -> S3, etc. I have attached a chart of the watch order and I set up my files accordingly. Is there any way to set it up like this?

1595

the only ways i see are:

  • use tvdb numbering
  • use anidb numbering
  • use grouping folder and clever anidb numbering using brackets to sort chronologically
  • use a season per series but wrong metadata

You can sort locally but they will be separate series in plex, possibly in Monogatari collection…

Monogatari series [grouping]

  • [01] bakemonogatari [anidb-xxxxx]
  • [02] kizumonogatari [anidb-xxxxx]
  • [03] nisemonogarari [anidb-xxxxx]
  • [04] nisemonogatari kuro [anidb-xxxxx]
  • [05] monogatari second season [anidb-xxxxx]
  • [06] hanamonogatari [anidb-xxxxx]
  • [07] tsukimonogatari [anidb-xxxxx]
  • [08] owarimonogatari [anidb-xxxxx]
  • [09] koyomimonogatari [anidb-xxxxx]
  • [10] owarimonogatari s2 [anidb-xxxxx]
  • [11] zokuowarimonogatari [anidb-xxxxx]

OR you can sort in plex but the agent cannot sort the mess (say goodbye to correct metadata unless you craft your own custom entry locally of scudlee mapping file to map anidb to fake tvdb xml and matching mapping)

  • Season 1 [Bakemonogatari]
  • Season 2 [kizumonogatari]
  • …
1596

Hello zeroQI,

Your plugin are realy good. I have lot of suggestions

  • First, I am french. And we have named your files with “vostfr”. This word it’s for find our translation. I think it’s just one expression to add. Like : “Danganronpa 3 Désespoir Episode 10 - Vostfr” or “[Tyrannosaure] Super Danganronpa 2.5 OAV - Komaeda Nagito to Sekai no Hakaisha - VOSTFR (BD 1920x1080)”
  • The second point are for file download on a web site. We have for exemple : “Grimoire.of.Zero.S01E05.VOSTFR.1080p.WEB-DL.AAC.x264-WKN-Shanks@Zone-Telechargement.Ws” For find the seanson are after “S” and episod after “E”. WKN are the name of first source. A another exemple : “Vandread 1st stage - 01.avi” and “Vandread 2nd stage - 01.avi” where season are the 1th and 2nd stage. And finaly a very problematic exemple : DefinitelyNotMe Mahou Tsukai no Yome - 01 VOSTFR 720p-Zone-Telechargement.ws-.mkv and Danganronpa 3 Désespoir Episode 3 - Vostfr.mp4
  • A another point are for a group of season. For exemple Danganronpa 3 contain 3 arc and one special episod. It’s the same problem with monogatari. If you can, your plugin take the folder structure. First are the name of the series. The second are for the arc or season. You can take name of the sub folder, and search a match with the directly or indirectly related. For exemple : Danganronpa: Kibou no Gakuen to Zetsubou no Koukousei The Animation - Anime - AniDB or Bakemonogatari - Anime - AniDB
    And after if you have the time, the sub folder can be a sub serie folder. Exemple : /Danganronpa/The animation /Danganronpa/Danganronpa/Despair Arc . This exemption need resolved.
  • This point are a problem : the first episod on each danganronpa are bugged, and take the place of the episode 3.


  • The final point are the subfolder name. For monogatari I have a big problem. The sub serie name are classed with “(Number)” And if you consider each sub folder for a season the bug not going. But actualy with your plugin I have no monogatari referenced.

-So this is a suggestion. myanimlist have good database. And can you add the posibility to download background, image, information from this source. (Just add informations)

Thank you,
Best regrads,
studyfranco

1597

@ZeroQI said:
the only ways i see are:

  • use tvdb numbering
  • use anidb numbering
  • use grouping folder and clever anidb numbering using brackets to sort chronologically
  • use a season per series but wrong metadata

You can sort locally but they will be separate series in plex, possibly in Monogatari collection…

Monogatari series [grouping]

  • [01] bakemonogatari [anidb-xxxxx]
  • [02] kizumonogatari [anidb-xxxxx]
  • [03] nisemonogarari [anidb-xxxxx]
  • [04] nisemonogatari kuro [anidb-xxxxx]
  • [05] monogatari second season [anidb-xxxxx]
  • [06] hanamonogatari [anidb-xxxxx]
  • [07] tsukimonogatari [anidb-xxxxx]
  • [08] owarimonogatari [anidb-xxxxx]
  • [09] koyomimonogatari [anidb-xxxxx]
  • [10] owarimonogatari s2 [anidb-xxxxx]
  • [11] zokuowarimonogatari [anidb-xxxxx]

OR you can sort in plex but the agent cannot sort the mess (say goodbye to correct metadata unless you craft your own custom entry locally of scudlee mapping file to map anidb to fake tvdb xml and matching mapping)

  • Season 1 [Bakemonogatari]
  • Season 2 [kizumonogatari]
  • …

I went ahead and tried the naming convention you suggested. However, it looks like the scanner is still skipping over the data.

Please see the attached logs/screenshots.

1598

@clowncracker You are not using beta branch scanner seemingly… You can add the Monogatari folder as library additional root folder and the folders inside won’t be skipped anymore

@studyfranco This uses Python which doesn’t require compilation and can be viewed with notepad++.
That’s one messy post. i will treat each point in the same order

  • will add VOSTFR to WHACK_PRE_CLEAN list.
  • SxxExx already taken as long as the file is not in a season folder
  • si ton repertoire est “Story Arc” ou “Arc xxxx” ou Ark xxxx" le folder est transparent. Will add “xxxx Arc” to transparent folders. If you use tvdb4 force id and label arcs as seasons like “Season 01 Despair Arc”
  • You folder naming is non-existant. your arc folder doesn’t follow conventions and you should feel bad (insert Zoidberg meme)
  • Exchange the ‘()’ with ‘[]’ and add “…\download\Plex\Anim\monogatari” on top of “…\download\Plex\Anim” as library root folder while i do another agent that will handle grouping folders seamlessly
  • myanimelist info IS taken in the beta branch as is anidb, thetvdb, themoviedb(movies and series), OMDb, FanartTV, TelevisionTunes. Please read the readme and try Beta branch AND then change agent settings, you are able to select the metadata source for each field.
1599

@ZeroQI said:
@clowncracker You are not using beta branch scanner seemingly… You can add the Monogatari folder as library additional root folder and the folders inside won’t be skipped anymore

@studyfranco This uses Python which doesn’t require compilation and can be viewed with notepad++.
That’s one messy post. i will treat each point in the same order

  • will add VOSTFR to WHACK_PRE_CLEAN list.
  • SxxExx already taken as long as the file is not in a season folder
  • si ton repertoire est “Story Arc” ou “Arc xxxx” ou Ark xxxx" le folder est transparent. Will add “xxxx Arc” to transparent folders. If you use tvdb4 force id and label arcs as seasons like “Season 01 Despair Arc”
  • You folder naming is non-existant. your arc folder doesn’t follow conventions and you should feel bad (insert Zoidberg meme)
  • Exchange the ‘()’ with ‘’ and add “…\download\Plex\Anim\monogatari” on top of “…\download\Plex\Anim” as library root folder while i do another agent that will handle grouping folders seamlessly
  • myanimelist info IS taken in the beta branch as is anidb, thetvdb, themoviedb(movies and series), OMDb, FanartTV, TelevisionTunes. Please read the readme and try Beta branch AND then change agent settings, you are able to select the metadata source for each field.

I managed to get it working with the beta client, so thank you for your help!

I was wondering where episode specific posters are pulling in from? I checked anidb and I couldn’t find out where this episode poster came from.

Reference: Kizumonogatari; anidb show 8357; episode 1; Iron-Blooded Arc

762×305 98.4 KB

I wanted to manually copy the metadata of this episode to a different version of this show in my library. However, I am unable to find episode specific posters on anidb. Where exactly do these pull from?

1600

Thank for your reply @ZeroQI .

So I test your solution. I will treat each point in the same order.

  • I add “vostfr”, “HEVC”, “VOSTFR”, “(Bonus inclus)”, “(BD 1920x1080)”, “10Bits-WKN”, “WKN”, “(Complet)”, “Zone-Telechargement”, “Despair-Paradise”, “Shanks@”, “Vostfr”, “[720p]”, “10Bits”, “(TV)” on WHACK_PRE_CLEAN
    All are good. If you want add this term on your code you are welcome (With the time I add more I think)

  • Ok, I check this point and I have a big BUG. The scanner not recognize the OAV,OAD and S02.

  • I follow your instruction. But metadata are not update. I think the problem are on anidb who considert one serie by one. (Same problem for the OAV) Have you an idea ?


And on your code can you add the match for saison (Ignore case)
### Extract season folder to reduce complexity and use folder as serie name ### folder_season = None for last_folder in reverse_path[:-1]: # remove root folder from test, [:-1] Doesn't thow errors but gives an empty list if items don't exist, might not be what you want in other cases for rx in SEASON_RX : # in anime, more specials folders than season folders, so doing it first match = re.match(rx, last_folder, re.IGNORECASE) # if match: # get season number but Skip last entry in seasons (skipped folders) if rx!=SEASON_RX[-1]: folder_season = int( match.group('season')) if match.groupdict().has_key('season') and match.group('season') else 0 reverse_path.remove(last_folder); break # All ways to remove: reverse_path.pop(-1), reverse_path.remove(thing|array[0]) if match and rx!=SEASON_RX[-1]: break # cascade break if not skipped folder since season number found if len(reverse_path)>1 and path.count(os.sep): #if grouping folders, skip and add them as additionnal folders Log.warning("Grouping folder: '%s' skipped, need to be added as root folder if needed" % path) Log.info("".ljust(157, '-')) return folder_show = reverse_path[0] if reverse_path else ""

  • The problem are resolved when I remove vostfr on the file name. I have no test with my modification on the script. (for the next post I test)
  • Ok, I test this one point later. (Or in january I read correctly your code for understand)
  • Cool ! I will test that.

New point :

  • A more complex series are Code geas. This series have film, OAV, Bonus, and two season. The problem is the same than Danganronpa for season. But the big problem are film and OAV. All OAV are in bazard.

    Folder OAV and film are no detected. Same like be special folder.

Have you an idea for fix that ?