[Release] [Alpha] [v0.1.3][v0.1.4 demo-7] Plex Anime Multi Source Agent (AMSA)

Dev Community Dev/API Corner
1

Hi everyone,

I’d like to introduce what has been a pet project of mine, it’s a drastically modified version of ZeroQI’s agent (Hama) which I’ve been using on my personal server for some time now.

The general idea behind this agent is to use metadata supplied from a number of different sources and cherry pick which details you’d like to include in your library, eventually this will be configurable through the agent’s options but for the time being it simply gathers metadata from AniDB (Posters, Series Info, Ratings & Tags), TvDB (Posters, Background, Banner, Series Info, Episode Info) and Plex (Themes songs) then uses them based on a order of preference (1. AniDB, 2. TVDB and 3. Plex).

Basically it uses AniDB metadata then falls back to TVDB is any of that data’s missing then finally falls back to Plex; the result is a library which will have (for example) its series information & ratings from AniDB, its episode information and ratings from TVDB and its theme music from Plex.

I’m expanding its functionality and squashing any bugs I find but I thought it was about time I offered it up as an alpha for anyone who’d like to give it a go, please note that I’m using this agent exclusively from my own anime library so I’m pretty confident in its abilities; please try it out if your interested and any feedback is welcome.

ChangeLog:
v0.1.4 (demo 7)

  • Updated proxy code (Issue github.com/Dingmatt/AMSA/issues/4)
  • Added a few episode titles to the AniDB bad title list (designed to help reduce the likelihood of using lazy AniDB titles)
    v0.1.4 (demo 6)
  • Fixed an issue which was slowing series matching.
  • Tweaked the new AniDB AntiBan support behaviour (was on by default in demo 5, now only used when required).
  • Fixed an error caused by missing metadata.
  • Added a AniDB ‘Bad Episode Name’ list (the agent will skip AniDB episode title metadata if it matches any pattern in the new list).
    v0.1.4 (demo 5)
  • Tweaked series matching scores (previously some scores were lower than expected).
  • Switched to using image thumbnails for TvDB & AniDB sources (this reduces the agents bandwidth usage and speeds up initial metadata download times; side effect is reduced quality of images in the Plex edit menu’s though if selected then the full quality images are retrieved).
  • Added AniDB AntiBan support for various scenarios (including but not limited to multiple servers sharing the same external IP; this will significantly reduce the likelihood of an AniDB ban for large libraries).
    v0.1.4 (demo 4)
  • Added support non-windows OS’s.
  • Reworked TheTVDB series/season poster ordering (will now default to highest rated).
  • Added ordering by name for roles.
    v0.1.4 (demo 3)
  • Added support for Plex theme music.
  • Removed restrictions to improve filename support.
  • Corrected a v0.1.4 branch folder name which was causing clones of that branch to fail.
  • Added http exception handling.
    v0.1.4 (demo 2)
  • Improved series matching results.
  • Improved series search speed.
  • Partial support for Hama’s custom folders (i.e. tvdb2, anidb2, etc).
  • Reworked absolute numbering episode detection.
  • Slight increase in the AniDB Anti-Ban timer.
  • A few minor tweaks to the metadata formatting.
    v0.1.4 (demo 1)
  • A complete re-code of the agent.
    v0.1.3
  • Updated the agents AniDB token (was previously using the Hama agents token).
  • Updated the base TVDB - AniDB custom mapping file (automatically updates on all prior versions upon next Plex restart).
    v0.1.2
  • Hotfix to the custom Absolute Series Scanner to apply a naming convention to undocumented opening / ending specials.
    v0.1.1
  • Initial Alpha release.

Download: github.com/Dingmatt/AMSA/releases

2

Hiya, could you expand a bit on the added functions or bugs resolved? I can only see missing bits in code…
If they are meaningful i would like to include them if any…

You can test the scanner against the test library (worst possible namings that use to trigger bugs) here:
https://github.com/ZeroQI/Hama.bundle/releases/download/v1.0/ZeroQI.test.Library.7z

3

Hi ZeroQI, good to see you. In regards to the files names and scanning the AMSA agent pretty much uses your Absolute Series Scanner (ASS) with a minor tweak which assigns a sequential name to any unidentified opening or ending.

The AMSA agent itself has been designed with different priorities to Hama, its not specifically designed to be an AniDB metadata agent (in contrast to Plex’s default TVDB agent), its designed to accept the metadata from a number of sources and build the Plex entry based on a combination of this data; currently only AniDB (Posters, Series Info, Ratings & Tags), TvDB (Posters, Background, Banner, Series Info, Episode Info) and Plex (Themes songs) have been integrated but my plans are to add more as they become available (and feasible).

You can see an example of this process below, where I’ve listed the data scraped from both AniDB and TVDB for the series Hibike! Euphonium; first thing the agent does is parse the data into a common format broken down into series, seasons and episodes (i.e. for AniDB it scrapes information from Hibike! Euphonium and its sequel Hibike! Euphonium 2 whilst TVDB it simply returns season 1 & 2 of the same series). After that is goes ahead and combines the information based on the servers preference (by default its AniDB, TVDB, Plex in that order); any fields which are blank end up getting filled by the next source.

Now in the case of Hibike! Euphonium all the metadata got populated by AniDB except the episode descriptions which were taken from TVDB but this ratio will change depending on the series, eventually I’m planning on making each field configurable (i.e. ‘name’ from AniDB but ‘ratings’ from TVDB, etc); please be aware that my examples only list a few of the common fields that anyone can recognise and the agent actually processes far more (as does HAMA).

The Scraped Data:

Combined:

590×724 130 KB

That the main key difference between HAMA and AMSA. As for other functionality unlike HAMA, AMSA doesn’t currently handle Movies (except ones considered to be specials) or the ability to specify how to Plex displays the series (i.e. AniDB split series or TVDB’s combined seasons); I plan on adding the former once I’ve decided on a convention but I’m not sure if I’ll add the latter unless its requested. AMSA is first and foremost a metadata agent and I believe its up to the user to determine how they want their shows displayed by following one convention or another (the agent can then detect the convention and display accordingly).

Our code bases have diverged quite a bit so it’s tricky to offer any bug fixes however when I notice issues in common code I’ll submit them as issues on the HAMA Git like I have in the past.

Feel free to plug into the AMSA ScudLee corrections list if you like which I perodically update based on my own experiances.

Though AMSA is listed as an Alpha I personally use it as my main agent so I’m pretty confident in its abilities, that aside if you have any specific questions then please ask away.

4

I use the my anime list metadata agent for series without a content rating on thetvdb. I added a bit of code to convert the rating on my anime list to a standard american tv rating.
#G - All Ages #PG - Children #PG-13 - Teens 13 or older #R - 17+ recommended (violence & profanity) #R+ - Mild Nudity (may also contain violence & profanity) #Rx - Hentai (extreme sexual content/nudity) temp = metadata.content_rating.split(" - ",1) front = temp[0] options = {"G" : "TV-Y7", "PG" : "TV-G", "PG-13" : "TV-PG", "R" : "TV-14", "R+" : "TV-MA", "Rx" : "NC-17"} #animeRating = str(options[animeRating]) metadata.content_rating = str(options[front])

I donno, if you would want to use something like that in your plug in. I like using the content rating to filter what gets shared to my family.

5

I donno, if you would want to use something like that in your plug in. I like using the content rating to filter what gets shared to my family.

That will be helpful, thanks for the contribution.

6

Hi.

Tried your agent, somehow he is not loading release year metadata.

Logs + screen:


Thx.

7

@IBNB said:
Hi.

Tried your agent, somehow he is not loading release year metadata.

Logs + screen:

Dropbox - File Deleted - Simplify your life
Dropbox - File Deleted - Simplify your life

Thx.

Thanks for the logs, I’ll take a look and figure out what’s going on.

8

Furthermore it loads the same poster for shows with same starting word regardless of the following words of show example:

xxx 123; xxx 456 = same posters

Logs: https://www.dropbox.com/s/7js6husgdchoj8f/AMSA%20Logs.zip?dl=0
Screen: https://www.dropbox.com/s/pp87x2kmlc72l5w/same%20poster.jpg?dl=0

9

@IBNB said:
Furthermore it loads the same poster for shows with same starting word regardless of the following words of show example:

xxx 123; xxx 456 = same posters

Logs: Dropbox - AMSA Logs.zip - Simplify your life
Screen: Dropbox - same poster.jpg - Simplify your life

This is a known limitation when using a non-tvdb file structure, it’ll be fixed in version 0.1.4 where I’m working on making things compatible with the AniDB structure (and absolute numbering); cheers for the feedback.

10

Okay looking forward to new version. :stuck_out_tongue:

Thank you.

11

Hi DingMatt. I just found and tried out this plug-in and it’s not doing so well with InuYasha season 4. Eps 8 and 9 have the same title, but different plot summaries. I mean… I could manually go in and copy paste all of the metadata, but with 167 ep shows like the original Inuyasha? I’d rather not. lol. Also, besides MAL what sources does your plug-in use? Does it touch AnimeNewsNetwork’s vast encyclopedia at all?

Thanks,
ShiggitayHTPC

12

@shiggitayhtpc said:
Hi DingMatt. I just found and tried out this plug-in and it’s not doing so well with InuYasha season 4. Eps 8 and 9 have the same title, but different plot summaries. I mean… I could manually go in and copy paste all of the metadata, but with 167 ep shows like the original Inuyasha? I’d rather not. lol. Also, besides MAL what sources does your plug-in use? Does it touch AnimeNewsNetwork’s vast encyclopedia at all?

Thanks,
ShiggitayHTPC

@shiggitayhtpc Thanks for the feedback, I’m currently working on a re-code of the agent which should expand its capabilities and correct issues like this (though it’s taking a while).

At the moment the agent uses both TheTvDB and AniDB as source but that’s getting expanded to include MAL for images; I’ve taken a look at ANN’s available information and I can add it however I can’t really see any information there that isn’t available from other sources so it’s a lower priority at the moment.

13

I’ve gone ahead and released a demo for the next version of AMSA, this demo will allow users to see experience the next version of the agent during its development but lists as a separate agent to ensure it won’t disrupt existing libraries.

The demo is meant for people who’d like to help me test the new functionality and i’d advise people don’t use this as their primary agent but instead create a new Plex library in order to demo it.

So here’s what to consider if you’d like to give it a try:

  • The demo’s metadata will likely not be supported by the next release
  • It’s not feature complete (I’ll be adding more sources)
  • It’s unoptimized in places (it’ll be slow in places especially during the first refresh, testing will help me fix this)
  • Support will be limited (I’ll use feedback to fix the agent but I can only offer limited individual library support if you have a unique issue)
  • It will be short lived (demo 2’s on the horizon)

Installations is simple:

  1. Place the content of the zip (plug-ins, plug-in support, scanners, etc) into your Plex Media Server directory
  2. Create a new library
  3. Set its scanner to “Absolute Series Scanner”
  4. Set its agent to “Anime Multi Source Agent (Test)”
  5. Access the Amsa Test settings from your server library admin tab

Thanks to anyone who gives it a try and let me know what you think, you can find the demo here.

14

I’ve gone ahead and released the second demo of the newly re-coded AMSA.

The demo is meant for people who’d like to help me test the new functionality and i’d advise people don’t use this as their primary agent but instead create a new Plex library in order to demo it.

Changes since demo 1:

  • Improved series matching results
  • Improved series search speed
  • Partial support for Hama’s custom folders (i.e. tvdb2, anidb2, etc)
  • Reworked absolute numbering episode detection
  • Slight increase in the AniDB Anti-Ban timer
  • A few minor tweaks to the metadata formatting

All other notes from Demo 1 are still applicable.

15

I’ve gone ahead and released the third demo of the newly re-coded AMSA.

The demo is meant for people who’d like to help me test the new functionality and i’d advise people don’t use this as their primary agent but instead create a new Plex library in order to demo it.

Changes since demo 2:

  • Added support for Plex theme music.
  • Removed restrictions to improve filename support.
  • Corrected a v0.1.4 branch folder name which was causing clones of that branch to fail.
  • Added http exception handling

All other notes from Demo 2 are still applicable.

16

I’ve gone ahead and released the forth demo of the newly re-coded AMSA.

The demo is meant for people who’d like to help me test the new functionality and i’d advise people don’t use this as their primary agent but instead create a new Plex library in order to demo it.

Changes since demo 3:

  • Added support non-windows OS’s.
  • Reworked TheTVDB series/season poster ordering (will now default to highest rated).
  • Added ordering by name for roles.

All other notes from Demo 3 are still applicable.

17

I have been testing your agent (beta 4) and i love it. At the moment its outperforming HAMA with metadata and poster.
as you pointed out. movies don’t work atm and i hope these will be added in the future.

But I have found 2 problems so far.
OVA anime are bid of “hit or miss”. some work. most don’t (probably the same reason as the movie thing.)
the second is more of a problem… the first 200 anime work great with getting the metadata etc. but anime 201 gives an error.
as soon as i ask to link an anime or ask for a re scan it starts scanning but instantly says it finished without downloading the metadata. the general log says something with Cache overflow.
I can still refresh or change posters or metadata from the first 200 anime… just the 201 and up stop working.
let me know if you need more data.

18

@Jackolas126 Thanks for the feedback, it’s good to know that someones finding the agent useful.
Would you mind giving me an example of an OVA which isn’t working along with how it’s stored in the filesystem (filename, directory it’s in, etc), with that I should be able to recreate the problem and fix it.

As for the 200 issue I’m pretty sure this is likely due to anidb, they have a number of anti-flooding safeguards in place which temporarily ban IP from accessing their system if too many requests are made within a certain timeframe; publically they’ve stated that the limit is a request every two seconds (AMSA waits three seconds to be sure) but the consistency you’ve mentioned might mean there’s another hidden limit I wasn’t aware of. I’ll attempt contacting their dev team to see if they can help, in the meantime you’ll likely see the remaining series correct there metadata within 24 hours (the standard time for an anidb ban).

If you have the time could you send me over a copy of your AMSA log files which are available at “Plex Media Server\Logs\PMS Plugin Logs” (com.plexapp.agents.amsa_test.log, one through to five) so I can confirm its an anidb issue.

19

line 14 and 17 are duplicated
https://github.com/Dingmatt/AMSA/blob/v0.1.4-Demo4/Plug-ins/Amsa_Test.bundle/Contents/Code/functions.py#L14

20

@ZeroQI Nicely spotted, thanks.