[Rel] HTTP Anidb Metadata Agent (HAMA)

ZeroQI · May 10, 2018, 2:15pm

@iball as explained in both agent and scanner read-me, if you see the file in plex at the right season and ep number, the scanner did its job. Hama once working (using beta branch ?) allow you to follow either AniDB or TheTVDB numbering.
You are free to show the two seasons as series or 1 show with 2 seasons. metadata will work in both cases.

Anderstn · May 13, 2018, 10:38am

Just a quick question here, and I may not be using the latest Scanner and Agent so that might fix this, but why am I getting some shows listed in collections as well as series and some shows listed normally.

Here is a quick example with Gintama vs some other shows appearing both as a single show and in a collection:

ZeroQI · May 13, 2018, 11:25am

Because you let a metadata source in collection fields that provided it and you didn’t read Plex announcement at the beginning of the year and didn’t went into your library settings and set to ignore collections or to only display items if not in a collection…

xeroiv · May 28, 2018, 8:01pm

Is there a way to change the default title search provider of HamaTV? I have my library named mostly to thetvdb naming conversion and would like plex to show them as such rather than AniDB which typically uses the Japanese titles.

ZeroQI · May 28, 2018, 8:24pm

@xeroiv The search() function assign a unique id, update() find the metadata for the title for that series id, according to the source and language priority specified, you do not search for a title per say:

Series with seasons are matched with TheTVDB by default, single season series with AniDB.
Regardless of an AniDB id or TVDB id being used, the title is selected as per the language priority then metadata source order (in its beta branch HAMA)
You might want to set ‘title’ agent setting to “TheTVDB, AniDB|TheTVDB, AniDB” (serie priority|episode_priority) and the serie priority set to “en, x-jat, ja” possibly and same for episode language priority.
Why not use TheTVDB agent?
Please consider a donation if you are satisfied with the level of customisation. It took forever to manage a proper priority for languages and multiple anidb xml file download for multiple seasons/tvdb series. it is possibly the most complex agent, but features live tvdb3 mode, anidb2, scanner recognizing op/eds, etc… are only found here, like movies in series folder and together with recent plex library settings allowing to hide seasons for single season series ad items when in collection, works like a dream…

Vaylan · June 6, 2018, 11:49am

Hello. I’m new to HAMA so i’m probably missing something. I want to anime titles to be in romaji but they are still being displayed in english for some reason. Do i need to change some settings?

I’m using this settings. I tried to change main to x-jat and same result.

PS: I’m using seasons folders. does it matter?

ZeroQI · June 6, 2018, 12:12pm

Use beta branch, priority ‘x-jat, en’ and source ‘AniDB, TheTVDB’ in agent settings

Vaylan · June 6, 2018, 4:32pm

@ZeroQI said:
Use beta branch, priority ‘x-jat, en’ and source ‘AniDB, TheTVDB’ in agent settings

Thanks for your reply. I changed to beta and the titles now are fine but some shows like for example Higurashi no Naku Koro ni have all episode names as dates for some reason like this:

Which settings i need to change to fix this?

Edit: Added logs.
Edit2: Found new broken episode titles. I added another log. but still have more show with “dates” as titles like all the “monogatari” and others.
Edit3: Its using the “firstAired” value instead of “EpisodeTitle” for some reason

<EpisodeName>The Promise</EpisodeName>

<EpisodeNumber>1</EpisodeNumber>

<FirstAired>2014-01-11</FirstAired>

<GuestStars/>

<IMDB_ID>tt3461860</IMDB_ID>

ZeroQI · June 7, 2018, 8:01am

Plex use the date if no episode title is present. my scanner gave the ep title but was overwritten with the date…
Thanks for report, fixed major ep title issue i missed
Please update to latest Hama beta code, test again and report

Vaylan · June 7, 2018, 4:48pm

@ZeroQI said:
Plex use the date if no episode title is present. my scanner gave the ep title but was overwritten with the date…
Thanks for report, fixed major ep title issue i missed
Please update to latest Hama beta code, test again and report

Thanks for the quick update. Everything is fine so far with names with the new version.
I have found a small issue but its not urgent. the first episode of Steins;Gate 2011 keep getting identified as the first special episode for some reason. I switched for this show to the tvdb and its fine. just thought i’d tell you about it.

ZeroQI · June 7, 2018, 8:26pm

i would need the filelist or scanner log in the agent data folders to see why but the filename decide the season and episode number… Seem like it is scrubbed as season 0… Changing the agent shouldn’t change that…
Unless i have the scanner log in Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama\DataItems_Logs i cannot see what causes it…

Vaylan · June 8, 2018, 11:04am

@ZeroQI said:
i would need the filelist or scanner log in the agent data folders to see why but the filename decide the season and episode number… Seem like it is scrubbed as season 0… Changing the agent shouldn’t change that…
Unless i have the scanner log in Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama\DataItems_Logs i cannot see what causes it…

Hello, I attached the logs for the series. hope they help.

Edit: both the special 1 and episode 1 are getting the info of Egoistic Poriomania

ZeroQI · June 8, 2018, 12:40pm

metadata.seasons[ 0].episodes[ 1]
[x] title Sources: TheTVDB, AniDB| (TheTVDB), AniDB Inside: ‘[‘TheTVDB’]’ Value: ‘Egoistic Poriomania’
metadata.seasons[ 1].episodes[ 1]
[x] title Sources: TheTVDB, AniDB| TheTVDB, (AniDB) Inside: ‘[‘TheTVDB’, ‘AniDB’]’ Value: ‘Egoistic Poriomania’

TheTVDB have the right info: https://www.thetvdb.com/series/steins-gate
you assigned the series to tvdb-244061 BUT s01e01 title comes from AniDB (notice the parenthesis)

It should have taken the titles in order of language ([‘en’, ‘’, ‘ja’]) and if equal TheTVDB before AniDB, the empty string is possibly AniDB X-jat (also called main title as it always is x-jat/romaji in AniDB)

Will investigate this evening. To do:

need to check mappings between thetvdb and AniDB for episode numbers
need to check the language priority
need to see why TheTVDB title is missing/not taken first
need to see why AniDB assigned the special ep to s01e01

EDIT:

reproduced issue, will be able to fix in one update
forgot to check language index before updating title, behave properly

This allow me to see while tvdb s00e01 cannot be mapped back to tvdb s00e02 could but wasn’t working after above fix

no anidb season 0 ep mapping
single anidbid detected
bad defaulttvdbseason (‘a’ instead of ‘1’)
episode listing per anidbid

Please update to latest and report.
This is the new output in logs, thanks to your feedback

2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - === Update ==================================================================================================================================================
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - id: tvdb-244061, title: Steins;Gate, lang: en, force: True, movie: False
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - -------------------------------------------------------------------------------------------------------------------------------------------------------------
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - AnimeLists.GetMetadata() - tvdb_numbering: True
2018-06-09 12:23:21,780 (2de0) :  INFO (logkit:16) - [+] AniDBid:  7729, TVDBid: 244061, defaulttvdbseason:  1, offset:   0, name: Steins;Gate
2018-06-09 12:23:21,782 (2de0) :  INFO (logkit:16) -     - season: [ 0],           [ 0], range:      [000-000], offset:   0, text: 2-7
2018-06-09 12:23:21,782 (2de0) :  INFO (logkit:16) - [+] AniDBid:  8655, TVDBid: 244061, defaulttvdbseason:  0, offset:   0, name: Steins;Gate: Fuka Ryouiki no Deja vu
2018-06-09 12:23:21,783 (2de0) :  INFO (logkit:16) -     - season: [ 1],           [ 0], range:      [000-000], offset:   0, text: 1-2;2-0;3-0
2018-06-09 12:23:21,786 (2de0) :  INFO (logkit:16) - [+] AniDBid: 10887, TVDBid: 244061, defaulttvdbseason:  0, offset:   0, name: Steins;Gate: Soumei Eichi no Cognitive Computing
2018-06-09 12:23:21,786 (2de0) :  INFO (logkit:16) -     - season: [ 1],           [ 0], range:      [000-000], offset:   0, text: 1-3;2-4;3-5;4-6
2018-06-09 12:23:21,790 (2de0) :  INFO (logkit:16) -              -----          ------
2018-06-09 12:23:21,792 (2de0) :  INFO (logkit:16) -               7729          244061
2018-06-09 12:23:21,792 (2de0) :  INFO (logkit:16) - [=] mappingList: {'TVDB': {'s0e5': ('1', '3', '10887'), 's0e6': ('1', '4', '10887'), 's0e7': ('0', '2', '7729'), 's1': {'7729': '0'}, 's0': {'10887': '0', '8655': '0'}, 's0e2': ('1', '1', '8655'), 's0e3': ('1', '1', '10887'), 's0e0': ('1', '3', '8655'), 's0e4': ('1', '2', '10887')}, 'defaulttvdbseason': '1'}
2018-06-09 12:23:21,803 (2de0) :  INFO (logkit:16) - -------------------------------------------------------------------------------------------------------------------------------------------------------------
2018-06-09 12:23:21,803 (2de0) :  INFO (logkit:16) - AniDB.GetMetadata() - AniDBid: 7729, AniDBids list: ['7729', '10887', '8655']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  7729: ['s1e1', 's1e2', 's1e3']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+] 10887: ['s0e1']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  8655: ['s0e2']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - AniDBid: 7729, url: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7729
018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  8655: ['s0e2']
                        2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
                        2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - AniDBid: 7729, url: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7729

Vaylan · June 10, 2018, 8:12am

@ZeroQI said:
metadata.seasons[ 0].episodes[ 1]
title Sources: TheTVDB, AniDB| (TheTVDB), AniDB Inside: ‘[‘TheTVDB’]’ Value: ‘Egoistic Poriomania’
metadata.seasons[ 1].episodes[ 1]
title Sources: TheTVDB, AniDB| TheTVDB, (AniDB) Inside: ‘[‘TheTVDB’, ‘AniDB’]’ Value: ‘Egoistic Poriomania’

TheTVDB have the right info: STEINS;GATE - TheTVDB.com
you assigned the series to tvdb-244061 BUT s01e01 title comes from AniDB (notice the parenthesis)

It should have taken the titles in order of language ([‘en’, ‘’, ‘ja’]) and if equal TheTVDB before AniDB, the empty string is possibly AniDB X-jat (also called main title as it always is x-jat/romaji in AniDB)

Will investigate this evening. To do:

need to check mappings between thetvdb and AniDB for episode numbers

need to check the language priority

need to see why TheTVDB title is missing/not taken first

need to see why AniDB assigned the special ep to s01e01

EDIT:

reproduced issue, will be able to fix in one update

forgot to check language index before updating title, behave properly

This allow me to see while tvdb s00e01 cannot be mapped back to tvdb s00e02 could but wasn’t working after above fix

no anidb season 0 ep mapping

single anidbid detected

bad defaulttvdbseason (‘a’ instead of ‘1’)

episode listing per anidbid

Please update to latest and report.
This is the new output in logs, thanks to your feedback
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - === Update ==================================================================================================================================================
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - id: tvdb-244061, title: Steins;Gate, lang: en, force: True, movie: False
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - -------------------------------------------------------------------------------------------------------------------------------------------------------------
2018-06-09 12:23:21,766 (2de0) :  INFO (logkit:16) - AnimeLists.GetMetadata() - tvdb_numbering: True
2018-06-09 12:23:21,780 (2de0) :  INFO (logkit:16) - [+] AniDBid:  7729, TVDBid: 244061, defaulttvdbseason:  1, offset:   0, name: Steins;Gate
2018-06-09 12:23:21,782 (2de0) :  INFO (logkit:16) -     - season: [ 0],           [ 0], range:      [000-000], offset:   0, text: 2-7
2018-06-09 12:23:21,782 (2de0) :  INFO (logkit:16) - [+] AniDBid:  8655, TVDBid: 244061, defaulttvdbseason:  0, offset:   0, name: Steins;Gate: Fuka Ryouiki no Deja vu
2018-06-09 12:23:21,783 (2de0) :  INFO (logkit:16) -     - season: [ 1],           [ 0], range:      [000-000], offset:   0, text: 1-2;2-0;3-0
2018-06-09 12:23:21,786 (2de0) :  INFO (logkit:16) - [+] AniDBid: 10887, TVDBid: 244061, defaulttvdbseason:  0, offset:   0, name: Steins;Gate: Soumei Eichi no Cognitive Computing
2018-06-09 12:23:21,786 (2de0) :  INFO (logkit:16) -     - season: [ 1],           [ 0], range:      [000-000], offset:   0, text: 1-3;2-4;3-5;4-6
2018-06-09 12:23:21,790 (2de0) :  INFO (logkit:16) -              -----          ------
2018-06-09 12:23:21,792 (2de0) :  INFO (logkit:16) -               7729          244061
2018-06-09 12:23:21,792 (2de0) :  INFO (logkit:16) - [=] mappingList: {'TVDB': {'s0e5': ('1', '3', '10887'), 's0e6': ('1', '4', '10887'), 's0e7': ('0', '2', '7729'), 's1': {'7729': '0'}, 's0': {'10887': '0', '8655': '0'}, 's0e2': ('1', '1', '8655'), 's0e3': ('1', '1', '10887'), 's0e0': ('1', '3', '8655'), 's0e4': ('1', '2', '10887')}, 'defaulttvdbseason': '1'}
2018-06-09 12:23:21,803 (2de0) :  INFO (logkit:16) - -------------------------------------------------------------------------------------------------------------------------------------------------------------
2018-06-09 12:23:21,803 (2de0) :  INFO (logkit:16) - AniDB.GetMetadata() - AniDBid: 7729, AniDBids list: ['7729', '10887', '8655']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  7729: ['s1e1', 's1e2', 's1e3']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+] 10887: ['s0e1']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  8655: ['s0e2']
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - AniDBid: 7729, url: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7729
018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - [+]  8655: ['s0e2']
                        2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
                        2018-06-09 12:23:21,805 (2de0) :  INFO (logkit:16) - AniDBid: 7729, url: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7729

Thanks for the quick fix. Everything looks fine at the moment.

ZeroQI · June 10, 2018, 2:36pm

Major bugfix for ep titles and also multi season series with anidb metadata

language priority works then takes in metadata source order.
If title for series and episodes differ, use | as here in agent settings title: 'AniDB, TheTVDB|'TheTVDB, AniDB and language ‘ja, x-jat, en’ for series and episodes for example. x-jat only exist in anidb though…
Asian users should have better experience now… Japanese ep titles pulled properly now, other languages should work too…

Edit: and search now scrub titles to remove bracketed words

mcrommert · June 16, 2018, 11:46pm

Okay hoping for some advice here

I have dragon ball kai 1-167 - anidb considers it two different animes (up to cell and then the rest of kai) - and tvdb has some insane numbering which randomly takes episodes from season two (around the mid 100’s) and makes them specials for a reason i still don’t understand. I had setup it as tvdb4 with hama…but the major issue i have run into is that trakt stops being able to track the show at a certain point

So your advice…would probably like to keep absolute numbering, wouldn’t mind the separation between the sagas…but i absolutely need mapping to tvdb

Any advice would be appreciate -have wasted hours on this so far and nothing seems to be a good solution

ZeroQI · June 17, 2018, 8:31am

TheTVDB is a database about US/English TV series and episodes. They will
follow the series provider provided season splitting and episode order

No duplicate series especially for different contries airing, almost no
re-cut series.
Sources officially supported http://forums.thetvdb.com/
viewtopic.php?f=7&t=9271
-original provider is gospel and released first a cut-down version in japan
of a serie that is a re-cut already so no other entry tolerated and episode
aired internationally are specials

TheTVDB has a very precise thread about the stance and the very active
presence of oneofourdead and active Plex member: Sven-7,hi Sven-7!
https://forums.thetvdb.com/viewtopic.php?f=18&t=34812&start=50
TheTVDB absolute numbering and dvd order seem accurate, due to the
extensive thought processand rules followed it is not insane,

You have to follow either thetvdb or anidb numbering on disk, however when
you edit a series settings you can select the Episode Ordering (how the
episodes are stored on disk)

Airing order (default)
DVD order
Absolute numbering (should include the specials in the right order
withTVDBagent, i could add support forTheTVDB absolute numbering if
required)

You could use TheTVDB agent instead of HAMA in that instance and select to
have absolute numbering on disk: https://forums.plex.tv/
discussion/288407/pulling-dvd-order-vs-aired-order-from-thetvdb
Season 1 - Episodes 1-26 (as shown on the Blu-Ray box art
https://www.amazon.ca/Dragon-Ball-Kai-Season-Blu-Ray/dp/B0079KGANU)
Season 2 - Episodes 27-52 (as shown on the Blu-Ray box art
https://www.amazon.ca/Dragon-Ball-Kai-Season-Blu-Ray/dp/B007BMIGZG/)
Season 3 - Episodes 53-77 (as shown on the Blu-Ray box art
https://www.amazon.ca/dp/B008D1RDJO/)
Season 4 - Episodes 78-98 (as shown on the Blu-Ray box art
https://www.amazon.ca/Dragon-Ball-Kai-Season-Blu-Ray/dp/B00AUJH3GU/)
Season 5 - Episodes 99~

HAMA use a mappinglist which can use AniDB absolut numbering if
defaulttvdbseason=‘a’
I use thetvdb made up absolute numbering by taking eps sequentially since
TheTVDB absolute numbering add back these specials back in the middle…
i skipped TheTVDB numbering for anidb to tbdb mapping reasons

@sven-7 How do you havethesestored on your disk ?
@mrcommert so in short, you want Trakt to be accurate and Plex therefore to
show accurate TheTVDB information and if possible keep absolute numbering?
What happens when you tell the series it is in absolute numbering on disk
and select TheTVDB agent ?

zanejchua · June 17, 2018, 5:41pm

I am not sure if I am facing an issue with ASS or Hama.. Weirdly enough I have Bleach in a folder and in the respective Season folders e.g.

Bleach

Season 1
- - Bleach.S01E01
- - Bleach.S01E02

and it does not get picked up by Plex at all. Only Season 1 is this way, Season 2 onwards is using Bleach - 21 [480p].mkv, Bleach - 22 [480p].mkv.

The other issue is with Death Note. I still haven’t been able to get Death Note to match correctly with different naming schemes. Right now the scheme I am using is the following:

Death Note

Season 1
- - Death Note S01E01

which gets all 37 episodes to finally appear (Previously there were matches but only up to 24 episodes as some episodes were merged into different episodes incorrectly).

However now I have the issue where it’s merging episodes wrongly also.

ZeroQI · June 17, 2018, 5:49pm

@zanejchua

github.com

ZeroQI/Absolute-Series-Scanner/blob/master/README.md

## Plex scanner responsibilities
A Plex Series Scanner makes the video files showing in Plex and populate the following for the video files:
- Series name (including forced id passed to agent search() function)
- Series year (optional)
- Season number
- Episode number
- Episode title (not filled by plex default series scanner, but ASS fills it, but this will be overwritten by the metadata agent)

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

If you post anything on the scanner github or thread for any issue like poster missing/wrong, episode thumbnail/screenshot/title/summary missing/wrong, then you clearly haven't read this and need to pay the RTFM tax by donating (or you just find this the best anime scanner and/or agent, much apreciated average is 5 euros). Also applies if you want assistance but didn't follow troubleshooting steps or include logs...

## Plex Agent responsibilities
A Plex metadata agent will:
- Search the metadata source using the Series name given by the scanner and assign the series a guid, a unique identifier used to download the metadataa informations
- Update all the metadata information ((Series Title, summary year, episode title, summary, posters, fanart, tags, ...) for series and files showing in Plex thanks to the scanner. 
- Full list of fields here:  see https://github.com/ZeroQI/Hama.bundle/wiki/Plex-Metadata-Fields-Available for full list of fields)
Any information missing or wrong inthere in Plex is an Agent issue, refer to the Agent readme here: https://github.com/ZeroQI/Hama.bundle/blob/master/README.md

## Which Metadata/Title source to select?

This file has been truncated. show original

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

I do not think the read-me needs rewriting at this point…
So both issues ARE scanner related issue, so please attach series specific filelist and scanner logs:

[…]/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/Library_name/Logs/root_folder_name.filelist.log
[…]/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/Library_name/Logs/root_folder_name.scanner.log

I did try and Bleach.S01E01 is seen in Plex… Does your library have “Absolute Series Scanner” selected?

ZeroQI · June 18, 2018, 7:59am

Have migrated beta branch to master branch. Windows update script will now work
Need to add Hama to unsupported appstore and YouTube series an Movies Agent too. somebody knows how to do ?

Topic		Replies	Views
[Release] HTTP Anidb Metadata Agent (HAMA) Dev/API Corner scanner-agent-dev	63	2076	December 20, 2019
Absolute Series Scanner (for anime mainly) Dev/API Corner scanner-agent-dev	844	37556	October 19, 2023
[Release] [Alpha] [v0.1.3][v0.1.4 demo-7] Plex Anime Multi Source Agent (AMSA) Dev/API Corner scanner-agent-dev	54	3336	January 8, 2020
[Release] MyAnimeList.net Metadata Agent Dev/API Corner scanner-agent-dev	578	58432	August 4, 2021
MyAnimeList.net Metadata Agent Dev/API Corner scanner-agent-dev	101	1494	December 20, 2019

[Rel] HTTP Anidb Metadata Agent (HAMA)

Related topics