Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

AniDB metadata agent

TuffoTuffo Posts: 151Members ✭✭
Anime tv-show and movie metadata agent

So after lots of hassle and suffering I've finished my first metadata agent - in this case for AniDB.net and specifically it's UDP API. The API uses a hash function to uniquely identify every video file that it has information about, and the hashing function needs to read the file fully so it's not very fast. Also, AniDB has a limitation on the number of API calls per time unit, so everything is done sequentially. This means that if you have a large set of movies/tv-shows it will take some time to get all the metadata.

- Uses AniDB.net and it's UDP API (read more at http://wiki.anidb.net/w/UDP_API_Definition)
- Based on an existing client called adba (read more at https://github.com/lad1337/adba), that has been heavily modified to work in Plex's restricted runtime and with some additional functions
- Requires an AniDB.net account - make sure you set your AniDB username and password in the agent preferences and then restart the server
- Supports both movies and tv-shows
- Tries to locate titles in the following language order [English, Romaji, Kanji] (this is currently not configurable)
- OpenSubtitles can be enabled as secondary agent

A few gotcha's:
- Search is done firstly by file hash (file must match the released files that AniDB lists) and secondly by name (name must exactly match that which is registered on AniDB). You can also search by aid (AniDB id) by specifying "aid:XXX" as name in the custom search dialog under "Fix incorrect match".
- If you have a multi-season tv-show, you should either keep them as separate shows or use absolute numbering, as AniDB has no notion of seasons, just shows. For example, you should avoid a season 2 folder with episode no 1 - that will be overwritten with metadata for season 1 ep 1
- Slow


Installation options:

1. Install Unsupported Appstore from http://forums.plexapp.com/index.php/topic/25523-unsupported-appstore/ and download AniDB agent through there.

or

2. Download and unzip the attached file in your Plug-ins folder and you should be good to go.

I've tested it successfully on Windows 7, OSX Lion and Linux (Debian Wheezy), but if you have any problems let me know.

Cheers,
Fredrik

 

Version 13:

- Added title language selection in preferences. Default is English.

AniDB.bundle_v13.zip

 

Version 12 (only as a fallback):

- Added handling of specials. See http://wiki.plexapp.com/index.php/PlexNine_PMS_Naming_Guide#Specials for info on file structure.

AniDB.bundle_v12.zip

«13456716

Comments

  • TazgodXTazgodX Posts: 30Members, Plex Pass Plex Pass
    Excellent work! been waiting/hoping for this.

    a couple quick notes
    -Does not get banners/fanart
    (expected as AniDB doesn't have them)
    -Does not work with files you modified.(ex. changing the audio language label in MKVs or subtitle label)
    (expected as AniDB uses CRC)
    -Does not give Episode descriptions.(i don't mind not having them anyways)
    (expected as AniDB doesn't have episode descrptions)

    definitely not deal killers, just more of a heads up to others.
    again, excellent work!
  • BGSBGS Posts: 22Members, Plex Pass Plex Pass
    First off, thank you very much for your hard work and effort on this. I'm very grateful.

    I've tried this on OSX 10.7.2 and Windows 7 machines with similar results. On the Mac most files are not recognized at all, while on the PC many are recognized and depicted but when you click on the icon, only one episode is detected. I'm aware of the season limitation and all of the files are using absolute numbers so that's not the issue. I've played around with it for hours trying SHIFT-Click refreshing and every other thing I can think of, but no joy. The other thing I noticed was that in the Metadata Agent Settings, while AniDB shows up in the list, I can't select the checkbox in the list of agents in the sublist after choosing it. I'm not sure why that is, or if it has anything to do with my issue.

    I'm hopeful I can solve this, because I've been waiting patiently for someone like yourself to delve into an AniDB meta agent for some time.

    Thanks again, and if you have any suggestions, I'd be happy to do some testing!
  • TuffoTuffo Posts: 151Members ✭✭
    BGS wrote:

    First off, thank you very much for your hard work and effort on this. I'm very grateful.

    I've tried this on OSX 10.7.2 and Windows 7 machines with similar results. On the Mac most files are not recognized at all, while on the PC many are recognized and depicted but when you click on the icon, only one episode is detected. I'm aware of the season limitation and all of the files are using absolute numbers so that's not the issue. I've played around with it for hours trying SHIFT-Click refreshing and every other thing I can think of, but no joy. The other thing I noticed was that in the Metadata Agent Settings, while AniDB shows up in the list, I can't select the checkbox in the list of agents in the sublist after choosing it. I'm not sure why that is, or if it has anything to do with my issue.

    I'm hopeful I can solve this, because I've been waiting patiently for someone like yourself to delve into an AniDB meta agent for some time.

    Thanks again, and if you have any suggestions, I'd be happy to do some testing!


    I'm not sure what's going on here, so the best way is of you could send me the log file for the agent - both for Windows and OSX. You'll find it in Logs/PMS Plugin logs/com.plexapp.agents.anidb.log. The only thing I can think of at the top of head is that you have a lot of files that aren't the original release files that AniDB lists (perhaps some has been remuxed or similar). As AniDB relies on a hashing function the files must be identical with those released by a specific group that AniDB knows of.
  • TuffoTuffo Posts: 151Members ✭✭
    edited February 2012
    TazgodX wrote:

    Excellent work! been waiting/hoping for this.

    a couple quick notes
    -Does not get banners/fanart
    (expected as AniDB doesn't have them)
    -Does not work with files you modified.(ex. changing the audio language label in MKVs or subtitle label)
    (expected as AniDB uses CRC)
    -Does not give Episode descriptions.(i don't mind not having them anyways)
    (expected as AniDB doesn't have episode descrptions)

    definitely not deal killers, just more of a heads up to others.
    again, excellent work!


    Well, the agent can't really find things that aren't there, so I didn't think that had to be mentioned :-) I've added a gotcha about the modified files though, thanks!
  • TazgodXTazgodX Posts: 30Members, Plex Pass Plex Pass
    I was just pointing it out for others.
    Just a thought, is it possible to get a small TheTVDB part added to this agent to just get banner and fanart? Thats the only thing that would make this better.

    And i had a quick question, would Movies and OVA's pull up from the TV Shows Metadata agent? I would prefer 1 TV Show section with all my anime, including OVA and Shows.
  • TuffoTuffo Posts: 151Members ✭✭
    TazgodX wrote:

    I was just pointing it out for others.
    Just a thought, is it possible to get a small TheTVDB part added to this agent to just get banner and fanart? Thats the only thing that would make this better.

    And i had a quick question, would Movies and OVA's pull up from the TV Shows Metadata agent? I would prefer 1 TV Show section with all my anime, including OVA and Shows.


    I'll look into adding TheTVDB and perhaps also OpenSubtitles as secondary agents - although if the show can be found on TheTVDB why not just use that agent for the specific show?

    As for the movies and OVA's, the quick answer is no. Some brief testing gave that Plex's scanner doesn't pick them up properly in a tv-show section, so the agent cannot set metadata for them.
  • TazgodXTazgodX Posts: 30Members, Plex Pass Plex Pass
    well, TheTVDB does a lot that is bad for anime. it has episodes listed wrong, ex. Panty & stocking with garterbelt breaks down shows they played back to back, where AniDB lists them as one show. They also put new anime with different names as a season 2 and those are impossible to scrape, ex Amagami SS (season 1) Amagami SS+ (Season 2). A lot of anime will have a following season, but name it differently but TheTVDB just throws it in with the first season. And in a case of Hellsing Ultimate they threw it in under Hellsing as a special, and its not. its a completely new series standalone from hellsing.

    There is a lot they do that makes them a bad metadata agent for anime. but they do have banners/fanart. I used TheTVDB absolute ordering agent for a while and did an OK job, just get annoyed at many little things.
  • TuffoTuffo Posts: 151Members ✭✭
    TazgodX wrote:

    well, TheTVDB does a lot that is bad for anime. it has episodes listed wrong, ex. Panty & stocking with garterbelt breaks down shows they played back to back, where AniDB lists them as one show. They also put new anime with different names as a season 2 and those are impossible to scrape, ex Amagami SS (season 1) Amagami SS+ (Season 2). A lot of anime will have a following season, but name it differently but TheTVDB just throws it in with the first season. And in a case of Hellsing Ultimate they threw it in under Hellsing as a special, and its not. its a completely new series standalone from hellsing.

    There is a lot they do that makes them a bad metadata agent for anime. but they do have banners/fanart. I used TheTVDB absolute ordering agent for a while and did an OK job, just get annoyed at many little things.


    Ok, if you're only interested in the fanart it will work ok, but you'll have the same problem with episode descriptions. You might end up with descriptions for some of your episode. Like I said, I'm adding it as a secondary agent.

    Also, I'll probably update the search function to use the name of the movie/series as presented by Plex's scanner, in case the file hash lookup fails (for modified files). Note though that AniDB's API search is only for exact name matches, so naming the files in that case are essential.
  • TazgodXTazgodX Posts: 30Members, Plex Pass Plex Pass
    Yeah, I would mainly want just fanart and banners from TheTVDB. I don't care for episode descriptions or episode lists from TheTVDB.
  • TuffoTuffo Posts: 151Members ✭✭
    Ok, so after some testing and development it seems as if it's not that easy to just add TheTVDB as a secondary agent. It turns out that it's not really written to be used as a secondary agent, because it will overwrite any metadat already set by the AniDB agent. In my case, Saiyuki Reload from 2003 turned into Melrose Place from 1992 - not my cup of tea :-) For now, this is not an option, and I'm not that interested in writing a new scraper for TheTVDB just for fanart, so you'll just have to live without it (or write it yourself).

    I've added OpenSubtitles as a secondary agent and also added a fallback that will try to text search AniDB for the name that the Plex scanner presents to the agent, in case the file lookup fails (because of modified files). The new version should be up tonight both here and on the Unsupported AppStore.
  • pikohnpikohn Posts: 2Members
    Is there a certain setting in PMS that allows you to enter your anidb username and password? I tried hard coding it in the defaultPrefs file, but it does not work. Can someone please clarify?

    Mike
  • box1813box1813 Posts: 9Members
    edited April 2012
    pikohn wrote:

    Is there a certain setting in PMS that allows you to enter your anidb username and password? I tried hard coding it in the defaultPrefs file, but it does not work. Can someone please clarify?

    Mike


    I'm having the same problem. I've been all through there and can't figure out where to put my info.
  • TuffoTuffo Posts: 151Members ✭✭
    edited April 2012
    box1813 wrote:

    I'm having the same problem. I've been all through there and can't figure out where to put my info.


    There should be a custom preferences files created in your <Plex-dir>/Plug-in Support/Preferences/ directory (<Plex-dir> depends on your platform). Look for a file named com.plexapp.agents.anidb.xml - try updating that with username and password and then restart the server.
  • box1813box1813 Posts: 9Members
    I used an XML editor to drop my username and pw in. I checked the file with a different programs and was able to verify the changes had taken effect.

    Unfortunately, the files still won't update. I'm using individual folders for each show and season (no stacking), and leaving the files named as they were when I downloaded them. Any idea where I should look for the issue?

    (log attached)
  • TuffoTuffo Posts: 151Members ✭✭
    edited April 2012
    box1813 wrote:

    I used an XML editor to drop my username and pw in. I checked the file with a different programs and was able to verify the changes had taken effect.

    Unfortunately, the files still won't update. I'm using individual folders for each show and season (no stacking), and leaving the files named as they were when I downloaded them. Any idea where I should look for the issue?

    (log attached)


    I don't see anything in the log stating that the agent has actually run? How do you match the shows, manually or by agent priority? Is it set to be primary agent for the section? Try matching a show manually in the Media Manager UI, right click on the show and select "Match Using" and then "AniDB". After that you should see log statements like "Trying to lookup <file-path> by file on anidb". And be patient - it is a bit slow.
  • box1813box1813 Posts: 9Members
    I added the folder as a new section, selecting AniDB as the metadata source. The automatic search cycles through slowly but never returns any information. A manual update on a recent show (Nisemonogatari, subbed by CMS w/CRCs that match AniDB) returns "There was no acceptable match using the specified agent."

    The log refreshes after each time I start PMS, but never logs anything different (aside from times) than what was on the one I already submitted. I think you are right about it not running, but have no idea what could be blocking it. I'm using Windows 7 64bit with a standard install of PMS/PMC. The plugin was install through the Unsupported Plugins application in PMC.
  • KetchmanKetchman Posts: 11Members
    I've been trying to get this to work with no success. I have put in my username and password and even verified that the program is taking it in by printing it in the log file.

    It fails with trying to authenticate with a "Command has timed out" exception message.

    In __init__.py, line 22 the connect.auth is attempted and each time it fails and throws an exception for me. The exception is generated from abda.py on line 121 when command.resp is called and the exception is raised "raise AniDBCommandTimeoutError, "Command has timed out"".

    So I think this agent is not working at all. When I try to click on it within Preferences, it is greyed out and only gives me the opensubtitles.org or local options.

    Has anyone gotten this to work? Seems like an issue connected to anidb.

    The only thing I can think is with ANIDB's profile settings there is a place within the Account tab for "UDP API Key:". Are we required to put something there to get this working?
  • box1813box1813 Posts: 9Members
    @Pallimud

    I'm doing the UDP API tab thing right now. I've got a lot of shows so I'll post the results when I get up tomorrow. I'm willing to try anything up to and including live sacrifice to get this to work, lol. The stock metadata agent has a habit of removing shows: no me gusta.
  • TuffoTuffo Posts: 151Members ✭✭
    Pallimud wrote:

    I've been trying to get this to work with no success. I have put in my username and password and even verified that the program is taking it in by printing it in the log file.

    It fails with trying to authenticate with a "Command has timed out" exception message.

    In __init__.py, line 22 the connect.auth is attempted and each time it fails and throws an exception for me. The exception is generated from abda.py on line 121 when command.resp is called and the exception is raised "raise AniDBCommandTimeoutError, "Command has timed out"".

    So I think this agent is not working at all. When I try to click on it within Preferences, it is greyed out and only gives me the opensubtitles.org or local options.

    Has anyone gotten this to work? Seems like an issue connected to anidb.

    The only thing I can think is with ANIDB's profile settings there is a place within the Account tab for "UDP API Key:". Are we required to put something there to get this working?


    No, you're not supposed to put anything into the UDP API settings in your account. All you (should) need is a username and a password. I'm using the agent almost every week and I've had no problems, so I know the agent is working, we just need to figure out why it's not working for you. I'll do a fresh install of PMS in my Virtualbox Windows 7 when I get home tonight and see if I can find anything. Have you tried downloading the agent manually from the first post? Perhaps it has something to do with the Unsupported Appstore installation.
  • box1813box1813 Posts: 9Members
    Tuffo wrote:

    No, you're not supposed to put anything into the UDP API settings in your account. All you (should) need is a username and a password. I'm using the agent almost every week and I've had no problems, so I know the agent is working, we just need to figure out why it's not working for you. I'll do a fresh install of PMS in my Virtualbox Windows 7 when I get home tonight and see if I can find anything. Have you tried downloading the agent manually from the first post? Perhaps it has something to do with the Unsupported Appstore installation.


    Version 1 is working for me /woot. Installed, restarted the server a few times (sacrificed that goat I've been saving) and it started working.
«13456716
Sign In or Register to comment.