AniDB metadata agent

Dev Community Dev/API Corner
241

Sometimes this plugin gets the order of the episodes wrong. When it is merging the episodes it sometimes starts with episode 5 or sets episode 9 at second place. Any ideas how to resolve this?

What do you mean by "starts with" and "sets"? Do you mean that after scanning a tv show the episodes are listed in incorrect order in the Plex GUI? Episode ordering has nothing to do with the agent, the agent merely supplies metadata for any episode that is identified by the Plex scanner. It is the scanner that finds episode 1,2 etc. based on file naming.

242

What do you mean by "starts with" and "sets"? Do you mean that after scanning a tv show the episodes are listed in incorrect order in the Plex GUI? Episode ordering has nothing to do with the agent, the agent merely supplies metadata for any episode that is identified by the Plex scanner. It is the scanner that finds episode 1,2 etc. based on file naming.

After importing the episodes, they get analysed and then they are put together in one entry with different streams (I don't know if this is supposed to happen). In the list where I can choose wich episode I want to stream, the episodes are in the wrong order (Additionally the list does not show the name of the episode, so it is unconfortable to find the right episode).

243

After importing the episodes, they get analysed and then they are put together in one entry with different streams (I don't know if this is supposed to happen). In the list where I can choose wich episode I want to stream, the episodes are in the wrong order (Additionally the list does not show the name of the episode, so it is unconfortable to find the right episode).

Well, it sound like you have some issues with the scanner. Could you post some examples of filenames, the outcome of the scan and the log file for the agent?

244

Hi Tuffo,

Thanks for all the updates along the way to what is a very awesome little agent, however I've noticed that this scanner seems to have some pretty serious issues I've run into with two particular types of Anime:

1. Those with funky characters: Anime such as [Steins;Gate], [Say 'I Love You'], and even [code Geass: Hangyaku no Lelouch R2] all have issues with both manual and auto searching.

2. Those with craploads of episodes (mainly those with over 100 episodes): Anime such as [Bleach], [InuYasha], [Naruto], and [Naruto Shippuuden].

The problem these have fall into two categories:

1. Ban by "too many faults" - Basically, all the failed search attempts, and then (in the case of Bleach, Naturo, etc..) when it does find it - all the episode requests... I get banned time and time and time again. I've been speaking to one of the admins on anidb.net, and he's said the following:


The ban is coming from what I've happily termed "too many faults". This is almost always caused by the client asking for data that doesn't exist -- deleted records or the like. Typically these bans last for a few hours and drop, but they keep the count just at the threshold level, so if the client's behaviour reoccurs, it gets nabbed much faster than it would have originally. This is almost certainly what you're seeing happening now.
 
If I had to make guesses (and the logs do back this up, to a point), it would be that the plex client authors are making a number of calls for episode details, based on the number of episodes in the title. This, being a bit of a shotgun approach, is what is requesting the missing data.

2. It doesn't like 'hundredth' episodes (100, 200, 300, etc.): If it manages to get past the searching and the episode requests and the half-dozen bans along the way, things like InuYasha's episode 100 get mistaken as episode '0'.

In the end, I gave up fighting the bans and failed searches with [Steins;Gate] and [Say 'I Love You'], and managed to get them to work with the TVDB agent, and most episodes of InuYasha are sorted (hasn't seemed to have detected beyond episode 100, or 0 as it refers to it, but that may be a scanning issue). As for the others... I'm just hoping that by reducing the number of attempts spent on the two I've now got sorted, there'll be more room left in the 'keep trying until you get banned again' line-up to get through the rest, lol.

For reference, I'm using 'v12' of the agent, and PMS 0.9.8.3 running on a Synology DS2413+ NAS.

Any suggestions? :-/

Cheers,

~Ithiel

Edit: Just as a side note, here's a bit from the log of my most recent ban-hammer experience, lol

2013-08-04 21:22:04,526 (3800b) :  INFO (__init__:71) - Auth ok!
2013-08-04 21:22:04,544 (3800b) :  INFO (__init__:183) - Trying to lookup /volume1/media/Anime/Naruto (2002 TV-PG)/Naruto - 220 - Going On A Journey [480p XviD MP3 2ch].avi by file on anidb
2013-08-04 21:22:45,082 (3800b) :  INFO (__init__:186) - Could not load file data, msg: Command has timed out
2013-08-04 21:22:45,097 (3800b) :  INFO (__init__:195) - Trying to lookup Naruto 2002 Tv Pg by name on anidb
2013-08-04 21:22:45,112 (3800b) :  INFO (__init__:198) - Could not load anime data, msg: Not sending, banned
2013-08-04 21:22:45,133 (3800b) :  CRITICAL (agentkit:930) - Exception in the search function of agent named 'AniDB', called with keyword arguments {'openSubtitlesHash': '6868ecd59b72c24e', 'episode': '20', 'episodic': '1', 'show': 'Naruto 2002 Tv Pg', 'season': '2', 'filename': '%2Fvolume1%2Fmedia%2FAnime%2FNaruto%20(2002%20TV-PG)%2FNaruto%20-%20220%20-%20Going%20On%20A%20Journey%20[480p%20XviD%20MP3%202ch]%2Eavi', 'plexHash': '403bb7f7d723af53cbc1879a45233c1eac5cda8a', 'duration': '1384824', 'id': '51108'} (most recent call last):
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 923, in _search
    agent.search(results, media, lang)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 283, in search
    self.doSearch(results, media, lang)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 226, in doSearch
    fileInfo = self.doNameSearch(results, metaName, connection)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 199, in doNameSearch
    raise e
AniDBError: Not sending, banned
 

Thanks ^.^

245

Hi Ithiel,

Glad you like the agent, but there are a few quirks left, some of the that I honestly don't know how to solve given the somewhat harsh rules enforced by the AnidDB UDP API.

1. Not sure what you mean by "failed" attempts, but the agent only requests info for the episodes that the scanner (which is separate from the agent and owned by the Plex team) has picked up in the file system, it does NOT try to load info for all episodes listed on AniDB. Not sure what in the logs at AniDB that suggests that behaviour...

2. I think the 0 error is the scanner, have you tried the BABS scanner? That one is supposedly better at shows with a lot of episodes. I'll try to see if I can reproduce this, but again the agent only uses whatever episode number that the scanner has found.

About the naming issues with non-alphanumeric characters I'll see what can be done. I mostly use the hash-code lookup (by file) so I don't really see those kind of errors. If you want to you can alwasy try the new HAMA agent which is based on AniDB's HTTP API and only does name lookups, not hash lookups. It is faster/better on shows with a lot of episodes as it loads a complete show with just one call, so the ban issue will be at least less frequent depending on your usage (I think there are some rules regarding the HTTP API as well, not sure).

Hi Tuffo,

Thanks for all the updates along the way to what is a very awesome little agent, however I've noticed that this scanner seems to have some pretty serious issues I've run into with two particular types of Anime:

1. Those with funky characters: Anime such as [Steins;Gate], [Say 'I Love You'], and even [code Geass: Hangyaku no Lelouch R2] all have issues with both manual and auto searching.2. Those with craploads of episodes (mainly those with over 100 episodes): Anime such as [Bleach], [InuYasha], [Naruto], and [Naruto Shippuuden]. The problem these have fall into two categories: 1. Ban by "too many faults" - Basically, all the failed search attempts, and then (in the case of Bleach, Naturo, etc..) when it does find it - all the episode requests... I get banned time and time and time again. I've been speaking to one of the admins on anidb.net, and he's said the following: 
2. It doesn't like 'hundredth' episodes (100, 200, 300, etc.): If it manages to get past the searching and the episode requests and the half-dozen bans along the way, things like InuYasha's episode 100 get mistaken as episode '0'. In the end, I gave up fighting the bans and failed searches with [Steins;Gate] and [Say 'I Love You'], and managed to get them to work with the TVDB agent, and most episodes of InuYasha are sorted (hasn't seemed to have detected beyond episode 100, or 0 as it refers to it, but that may be a scanning issue). As for the others... I'm just hoping that by reducing the number of attempts spent on the two I've now got sorted, there'll be more room left in the 'keep trying until you get banned again' line-up to get through the rest, lol. For reference, I'm using 'v12' of the agent, and PMS 0.9.8.3 running on a Synology DS2413+ NAS. Any suggestions? :-/ Cheers, ~Ithiel  Edit: Just as a side note, here's a bit from the log of my most recent ban-hammer experience, lol

2013-08-04 21:22:04,526 (3800b) :  INFO (__init__:71) - Auth ok!
2013-08-04 21:22:04,544 (3800b) :  INFO (__init__:183) - Trying to lookup /volume1/media/Anime/Naruto (2002 TV-PG)/Naruto - 220 - Going On A Journey [480p XviD MP3 2ch].avi by file on anidb
2013-08-04 21:22:45,082 (3800b) :  INFO (__init__:186) - Could not load file data, msg: Command has timed out
2013-08-04 21:22:45,097 (3800b) :  INFO (__init__:195) - Trying to lookup Naruto 2002 Tv Pg by name on anidb
2013-08-04 21:22:45,112 (3800b) :  INFO (__init__:198) - Could not load anime data, msg: Not sending, banned
2013-08-04 21:22:45,133 (3800b) :  CRITICAL (agentkit:930) - Exception in the search function of agent named 'AniDB', called with keyword arguments {'openSubtitlesHash': '6868ecd59b72c24e', 'episode': '20', 'episodic': '1', 'show': 'Naruto 2002 Tv Pg', 'season': '2', 'filename': '%2Fvolume1%2Fmedia%2FAnime%2FNaruto%20(2002%20TV-PG)%2FNaruto%20-%20220%20-%20Going%20On%20A%20Journey%20[480p%20XviD%20MP3%202ch]%2Eavi', 'plexHash': '403bb7f7d723af53cbc1879a45233c1eac5cda8a', 'duration': '1384824', 'id': '51108'} (most recent call last):
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 923, in _search
    agent.search(results, media, lang)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 283, in search
    self.doSearch(results, media, lang)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 226, in doSearch
    fileInfo = self.doNameSearch(results, metaName, connection)
  File "/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/AniDB.bundle/Contents/Code/__init__.py", line 199, in doNameSearch
    raise e
AniDBError: Not sending, banned
 

Thanks ^.^

246

How do I get this to work properly with the Japanese numbering system? I've loaded it, shows up in the agents list, I added my credentials to the agent.

Once I scan a folder (in this case One Piece), it just says scan complete and doesn't list any files or filenames in it at all. I'm thinking this is due to the numbering scheme not being compatible with English vs Japanese version of the show. The episodes are titled in this way:

One Piece - 484 - The Navy Headquarters Falls! Whitebeard`s Unspeakable Wrath!

So I've tried changing it to:

One Piece - 01x484 - The Navy Headquarters Falls! Whitebeard`s Unspeakable Wrath!

Which works, but then it doesn't get the episode titles, just names them Episode 484.

I don't want to use the English naming scheme because it incorrectly things some of the newer episodes are part of season 6 (rather than episode 600 or I guess season 16)

247

How do I get this to work properly with the Japanese numbering system? I've loaded it, shows up in the agents list, I added my credentials to the agent.

Once I scan a folder (in this case One Piece), it just says scan complete and doesn't list any files or filenames in it at all. I'm thinking this is due to the numbering scheme not being compatible with English vs Japanese version of the show. The episodes are titled in this way:

One Piece - 484 - The Navy Headquarters Falls! Whitebeard`s Unspeakable Wrath!

So I've tried changing it to:

One Piece - 01x484 - The Navy Headquarters Falls! Whitebeard`s Unspeakable Wrath!

Which works, but then it doesn't get the episode titles, just names them Episode 484.

I don't want to use the English naming scheme because it incorrectly things some of the newer episodes are part of season 6 (rather than episode 600 or I guess season 16)

Well, did you set the agent to be the default for your tv show section? If now, did you do a manual match using the AniDB agent? What does the log file from the agent say?

248

[...]

Regarding this i may be able to help.

Some files do have their CRC-Checksum attached to their filenames, usually ending in something like (23AF37...).mkv . This is especially the case when using the AniDB Client to rename and add files to your MyList. Removing the CRC Checksum and rescanning the file solved all my problems concerning the bad numbering.

"Tales of the Abyss - 09 - The Usurped One - [Coalgirls](c9dd3021)" -> works like a charm and results in "The Usurped One"

"Tales of the Abyss - 10 - Atonement`s Return - [Coalgirls](f11aea49)" -> results in "Episode 49"

"Tales of the Abyss - 10 - Atonement`s Return - [Coalgirls]" -> results in "Atonement`s Return"

"Tales of the Abyss - 11 - City of Falling Snow - [Coalgirls](e028b47b)" -> results in Episode 28

"Tales of the Abyss - 11 - City of Falling Snow - [Coalgirls] -> results in "City of Falling Snow"

As far as I can tell there's no telling when or why the plugin mistakes the checksum for an episode number.

249

Regarding this i may be able to help.

Some files do have their CRC-Checksum attached to their filenames, usually ending in something like (23AF37...).mkv . This is especially the case when using the AniDB Client to rename and add files to your MyList. Removing the CRC Checksum and rescanning the file solved all my problems concerning the bad numbering.

"Tales of the Abyss - 09 - The Usurped One - [Coalgirls](c9dd3021)" -> works like a charm and results in "The Usurped One"

"Tales of the Abyss - 10 - Atonement`s Return - [Coalgirls](f11aea49)" -> results in "Episode 49"

"Tales of the Abyss - 10 - Atonement`s Return - [Coalgirls]" -> results in "Atonement`s Return"

"Tales of the Abyss - 11 - City of Falling Snow - [Coalgirls](e028b47b)" -> results in Episode 28

"Tales of the Abyss - 11 - City of Falling Snow - [Coalgirls] -> results in "City of Falling Snow"

As far as I can tell there's no telling when or why the plugin mistakes the checksum for an episode number.

Just to clarify (again), it's the scanner that parses the files and tries to make sense of name and episode numbering, not the agent. Ask the Plex devs about this or if you're using BABS then jmjf (the author of BABS).

250

Just to clarify (again), it's the scanner that parses the files and tries to make sense of name and episode numbering, not the agent. Ask the Plex devs about this or if you're using BABS then jmjf (the author of BABS).

Alright, sorry for misunderstanding your former post. Either way, hopefully my suggestion solves caffeinepills' problem.

251

i just need an agent that can recognise anime movies (all my anime tv show are done nicely), but not my movies.

example, i have Bleach - Memories of Nobody (2006).mkv

i tried this anidb meta agent, but no success (copied to plugin folder, register and put my login id, stop and re run plex media server, choose anidb as agent, update the anime group)

252

i just need an agent that can recognise anime movies (all my anime tv show are done nicely), but not my movies.

example, i have Bleach - Memories of Nobody (2006).mkv

i tried this anidb meta agent, but no success (copied to plugin folder, register and put my login id, stop and re run plex media server, choose anidb as agent, update the anime group)

Any output in the logs? Under PMS Plugins Logs there should be a file for the anidb agent.

253

Hi, thanks for your work! 

Probably I'm stating the obvious but I can suggest you to take a look to the "AnidbMOD" scraper for XBMC, wich manages to join anidb and tvdb worlds thanks to a unique ids list :)

I don't know if the same would be possible with Plax, but it would be useful!

Sorry for the bother :)

254

Hi, thanks for your work! 

Probably I'm stating the obvious but I can suggest you to take a look to the "AnidbMOD" scraper for XBMC, wich manages to join anidb and tvdb worlds thanks to a unique ids list :)

I don't know if the same would be possible with Plax, but it would be useful!

Sorry for the bother :)

If you want that integration you can have a look at HAMA agent which is based on the HTTP API at AniDB. The limitation of that API is that it only supports name lookups though. In the ideal world the agents should be combined, although someone has to do the work :)

255

Any output in the logs? Under PMS Plugins Logs there should be a file for the anidb agent.

yes there are files .log .log.1 .log.2, etc.

sadly i dont know how to read them. do you want me to attach it here?

or maybe the movies need to be inside separate individual folders (just like anime shows)

im trying again, this time do deep scan, and maybe later try to move some movie out and back in.

256

Erwinsie, yes, attachvthe logs here and I’ll have a look.

257

Hi Tuffo,

cant find how to attach, and text is long and messy so i have uploaded it here.

https://dl.dropboxusercontent.com/u/75355035/com.plexapp.agents.anidb.log

Thanks so much.

Erwinsie, yes, attachvthe logs here and I'll have a look.

258

Hi Tuffo,
cant find how to attach, and text is long and messy so i have uploaded it here.

https://dl.dropboxusercontent.com/u/75355035/com.plexapp.agents.anidb.log

Thanks so much.


The logs are just standard text files so they're easy to look at yourself, but going through them I find

2013-09-07 19:24:08,349 (6400d) :  INFO (__init__:130) - Loading metadata for anime aid 6751
2013-09-07 19:24:08,819 (6400d) :  DEBUG (networking:172) - Requesting 'http://img7.anidb.net/pics/anime/32901.jpg'
2013-09-07 19:24:11,110 (6400d) :  DEBUG (networking:172) - Requesting 'http://img7.anidb.net/pics/anime/32901.jpg'
2013-09-07 19:24:14,402 (6400d) :  INFO (__init__:313) - Loading metadata for '11 Eyes', season 1 episode 11

so it seems that the agent correctly identifies the show and tries to load information for episode 11 which apparently is the only episode that the scanner has picked up. So not sure what the problem is here?

259

The logs are just standard text files so they're easy to look at yourself, but going through them I find

so it seems that the agent correctly identifies the show and tries to load information for episode 11 which apparently is the only episode that the scanner has picked up. So not sure what the problem is here?

yes i realised there was an issue with anime series called "11 eyes" where the agent only can see 1 episode, episode 11.. almost gave up fixing it but strangely today all sorted out, i have all 12 episodes with correct names.

but.

my main concern is, the agent cannot see my anime movies collections (movies means single file)

for example, i have these in my anime folder and they dont show up on plex:

Afro Samurai (2007) BRRip.avi

Bleach - Hell Verse (2010).mkv

Bleach - The DiamondDust Rebellion (2007).mkv

Sword of the Stranger (2007).avi

Toaru Majutsu no Index Movie Miracle of Endymion (2013) FULLHD.mkv

Yu-Gi-Oh! The Movie (2004).avi

Afro Samurai Resurrection (2009) DC 720p.avi

260

yes i realised there was an issue with anime series called "11 eyes" where the agent only can see 1 episode, episode 11.. almost gave up fixing it but strangely today all sorted out, i have all 12 episodes with correct names.

but.

my main concern is, the agent cannot see my anime movies collections (movies means single file)

for example, i have these in my anime folder and they dont show up on plex:

Afro Samurai (2007) BRRip.avi

Bleach - Hell Verse (2010).mkv

Bleach - The DiamondDust Rebellion (2007).mkv

Sword of the Stranger (2007).avi

Toaru Majutsu no Index Movie Miracle of Endymion (2013) FULLHD.mkv

Yu-Gi-Oh! The Movie (2004).avi

Afro Samurai Resurrection (2009) DC 720p.avi

Ok, once again, it's not the agents responsibility to find and parse the video files, that's up to the scanner. The agent merely supplies metadata for any show or movie identified.