[Rel] HTTP Anidb Metadata Agent (HAMA)

HAMA will use thetvdb search for multiple season and AniDB search for single seasons, but you can force it with a tvdb.id file with the tvdbid inside in the series folder or ’ [tvdb-xxxxx]’ in the series foldername. That impact the numbering used, but you can select the metadata source per field on every field in the agent. Fields shared by both series and episodes can use the same or different ones by using pipe character: ‘|’

You can select the title from any source regardless the numbering, for example title agent setting “AniDB, TheTVDB|TheTVDB, AniDB” woud do AniDB first then TVDB for series but the other way around for episodes. That said, language order take precedence, so if you have language priority “ja, en”, if ja title is in TVDB and not AniDB, it will take the language in the order provided

Do you have a need to select the source for the metadata (agent settings) OR the numbering for single season series ?

I only want to know to know if there is an automatic way with me trying fix this

If i add this [tvdb-xxxxx] in the title it will work fine and metadata will be pulled. But i dont want to go through each show and do this manually i dont like the library to be inconsistent “shows use anidb and others use tvdb”.

The way Hama works is good for people who have their specials folder as a separate show, but I like mine inside the main series folder. I thought that there might be an option to make Hama use the tvdb by DEFAULT for both 1 season and multi seasons shows so i can avoid having shows like the one in the image.

EDIT:
This is just a small suggestion not sure if you will think its good or not:
For the show Relife from the image above in the anime-list-full file there are 2 listings

<anime anidbid="11074" tvdbid="299508" defaulttvdbseason="1">
  <name>Relife</name>
</anime>

and

<anime anidbid="13123" tvdbid="299508" defaulttvdbseason="0">
   <name>Relife (2018)</name>
</anime>

As you can see both of them shares the same tvdbId and both of them tells you which season is the right one. What im suggestion is maybe you can use this information to make the agent gets the specials automatically. What do you think?

Thanks

You prefer tvdb numbering so all movies, oavs and specials are considered specials in the series, i can understand…

Multiple seasons automatically use thetvdb numbering convention (and specials would be mapped propertly for you) but single season use AniDB as it is an AniDB (and tvdb) agent and quite some anime fans use aniDB as they have opening, endings, etc…

May i ask, if you want only thetvdb numbering and information, wouldn’t thetvdb agent work better for you?
I could make it an agent option to use thetvdb matching only. for single seasons if Hama has functions/data TheTVDB does not have… Others can weight in if this would be useful

Well, they are specials, aren’t they?

Not really, because I wont get the anidb titles this way. What about my suggestion? You can get the specials data also from anidb because they will have the same tvdbid.

TBH i don’t care where the data is coming from as long as it’s correct and complete. But IMO the agent is not working perfectly because it’s missing a whole season data. so if you manage to make this work even from the anidb my problem would be solved :). Like i said i only care about the data being complete and correct.

Only specials from TheTVDB point of vue, from a japanese/AniDB standpoint they are theatrical releases (movies), direct to video (OAVs), and seasons are separate series. Torrent downloads seem to favor AniDB numbering, which is why i allow to display as tvdb files following AniDB numbering (tvdb2 and 3 modes).

The Agent title says AniDB isn’t it ?

I already get specials info from both anidb and tvdb and cross reference them already through an XML mapping file. The issue is the numbering convention…

How can the agent know you are using TheTVDB by opposition to AniDB (which doesn’t have specials for that season, hence the lack of data)…

  • multiple seasons are always THeTVDB so it’s easy
  • single files are movies so always AniDB
  • single seasons are tricky but i understand your plight but cannot impact other users using AniDB as numbering convention…

I will add a drop-down option to select numbering for single seasons series with more than 1 files when i can, this way specials from TVDB will gather metadata in your case.

However, i am currently writing another agent for metadata export/restoration in local files, and Hama is currently working as designed, so it is a feature request, so it won’t be done right away but soon enough.
I will update this thread when done

This is good enough for me. Thanks :slight_smile:

I have a problem with the series “Amagami SS” (AniDBid: 7506, TVDBid: 172921) and the way specials are matched. I have one special (out of the 2 in total as per AniDB) that on AniDB is listed as a single episode, not handled as multiep, but contains 3 stories. On TheTVDB these special eps are handled as 3-3 separate episodes, 6 in total. On my disk I also have all the OP/ED clips that are present on AniDB as separate files, and HAMA is matching one of my OP/ED files with special ep. 4 on TheTVDB, which is wrong.

As I have several similar arrangements, I’d like to ask for suggestions on how to handle this. With these specials I could be fine with the title embedded in the filenames, or somehow coerce HAMA into cooperating. I can provide log files or other data if needed. Thanks in advance.

@ZeroQI: maybe you can help with some hints?

@nls42
I would need the scanner logs to see why the op/ed is mapped to an ep number that is mapped to TheTVDB. Regarding anidb specials mapped to multiple tvdb eps, i cannot help much as the mapping file doesn’t support this scenario [https://github.com/ScudLee/anime-lists]

using tvdb forced it and using tvdb specials numbering would help but op/ed would not be recognised but title in the file should remain.

Animelist <mapping anidbseason=“0” tvdbseason=“0”>;1-7;2-1;3-2;4-3;5-4;6-5;7-6;</mapping>

  • anidb s00e01 = tvdb s00e07
  • anidb s00e02 = tvdb s00e01
  • anidb s00e03 = tvdb s00e02

In reality mapping is:

  • anidb s00e01 = tvdb s00e07
  • anidb s00e02 = tvdb s00e01-03
  • anidb s00e03 = tvdb s00e04-06 (mapping file should point to s00e04)

|S1|Miya Tachibana: Little Sister|25m|29.04.2011||
|S2|Morishima Haruka Chapter / Tanamachi Kaoru Chapter / Nakata Sae Chapter|15m|11.06.2011||
|S3|Nanasaki Ai Chapter / Sakurai Rihoko Chapter / Ayatsuji Tsukasa Chapter|15m|05.08.2011||

https://www.thetvdb.com/series/amagami-ss/seasons/0

Thanks for the reply. I’m attaching the scanner log (I can attach the medatata agent log if needed). There’s one peculiarity with how I name specials. I’m using my own scanner to produce a directory of symlinked files (I don’t want to rename the originals). The file names used in the output of the scanner are determined by exact matches on AniDB (by ed2k hash + size), but the specials naming is linear WRT how you see them on the AniDB episode list for an anime (that’s my goal anyway but it seems to work as expected), because scanners can’t do anything wtih ep. numbers like C1, T5, etc. I’ve tried several combinations for naming files, the one you see in the log makes Plex see all the files separately, but the scanner connects the wrong dots for the specials.

Amagami SS.scanner.zip (1.5 KB)

http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7506
Type 3 specials starts at C1 and i assign openings from 101 and endings at 151.
You assign the opening/ending sequentially and specials 4 to 19 numbering is not endorsed.
Can you not use: the opening/ending supported scanner format:

  • AniDB numbering OP1a to OP1d, OP2a to OP2e, ED1 to ED8
  • Opening Xx
  • Ending Xx

Actually OP1a, OP1b, … are C1, C2, … when queried from the AniDB UDP API. That’s what I use to base my linear numbering on. If you could provide some docs on how to use these ranges, I could implement them easily. Normally specials are only a few dozen max. However AniDB has more special categories:

  1. Sxx - special episodes (unaired, BD-only, etc.)
  2. Cxx - credits or “clean” OP/ED footage
  3. Txx - trailers
  4. Pxx - parodies
  5. Oxx - misc. other

This is the order I’m numbering them in currently, considering the highest episode count for each respective category (the API provides those), and would need to be fixed to use the ranges your scanner uses. Though the multiple stories in a single ep. vs. multiple separated episodes per story issue remains, as outlined earlier…

EDIT: actually, renaming to the ranges your scanner produces might not work as that is the end result and would need code changes in your scanner, but anything that makes a reasonable mapping possible and already works with it is fine with me.

ANIDB_OFFSET = [ 0, 100, 150, 200, 400, 0, 0] ANIDB_TYPE = [‘Special’, ‘Opening’, ‘Ending’, ‘Trailer’, ‘Other’, ‘Episode’, ‘Episode’]
“Op1a” type numbering is directly supported by the scanner.
Here are the ranges: (same an xbmc anidb mod agent)

  • 100 opening
  • 150 ending
  • 200 trailer
  • 300 parodies
  • 400 other

The mapping file doesn’t support split eps mapping. My other project (lambda) may help once NFO support is completed for updating settings from NFO to add metadata to Plex possibly but it will take time

Thanks again for the quick help. So what I would need to do is name files linearly for each category, but also restarting numbering for each category, using the correct tagging, like Opening xx, Trailer xx, etc, right? Actually the regex noise you removed from your post was useful for getting hints for naming my files, but I guess I can find it again on GitHub :slight_smile:

That NFO support would be awesome. When implemented, it will be the most effective and accurate way for renaming, especially for custom solutions like mine. Split eps mapping is less important, though one wonders where TheTVDB got the idea of splitting apart those episodes, as they’re a single clip on the BD IIRC.

“opening 1a” in title without “specials 4” beforehand would work too…
The source code is open source on github so all regex will be at the top

Once NFO support for uploading works you generate an nfo, edit it and it will add back metadata back but might take some time to make this happen.

i don’t blame thetvdb, it’s a database of tv episodes going by the producer description, which is clear, but at times at odds with both bluray and torrent releases…

I’ll use your naming scheme, it’s easy. You might be right about TheTVDB.

Normally I’d open a ticket on GitHub but while we’re at it. I think I spotted a mistake in “Absolute Series Scanner.py” line 63 where the parodies regex seems to have become a victim of comment at the end of the line…

EDIT: HM, the ANIDB_OFFSET array is also missing the range for parodies in that file, maybe it’s intentional? Your list in your above post contains the parody bit, though.

i am not handling parodies yet, never seen anybody storing them…
How many parody files recognised in AniDB do you have?

Currently none, but I think it’s prudent to handle them when AniDB is providing info on them. I can download a few though to test. The code is there, why disable it?

I applied the discussed naming scheme to the Amagami SS series and it works, sort of. But there’s a slight discrepancy in how you handle OP/ED specials and the way AniDB presents them - on AniDB they’re just Cxx and OP/ED/clean insert song/etc is not distinguished - they’re all just “credits”. Which I think is good and since I don’t want to fiddle around with a heap of regexes (that’s the job of the agent) I just name the Cxx specials like “Series - OP 5 - Something opening.mkv”. Now the agent just slaps them in the OP category (1xx ep. numbers), understandably. Would it be possible to refine the agent to handle “Credits” in place of OP/ED/etc tags and do a second regex pass on the title itself to determine the correct category?

EDIT: BTW, the series mix-up is gone, thanks. All eps are in the correct season this way, at least.

The scanner (ASS) manage the episode and season numbers.
Only AniDB XML show Cxx to my knowledge. Please indicate where you see the Cxx reference and which release group or torrent release include this format as you are the only person mentioning it and no tool support it I believe…

As I’ve mentioned, I’m using the UDP API. The episode number returned is Cxx there. No more detailed designation (except for the title). I’m unaware/don’t care about release group file naming as I’m only using info gathered from AniDB for my hashed files. I’m querying files using ed2k hash+filename (which I cache and only query by file ID for subsequent requests).

EDIT: the HTTP API uses the same episode numbering format.