If you have not already, we suggest setting your Plex username to something else rather than email which is displayed on your posts in forum. You can change the username at https://app.plex.tv/desktop#!/account
Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

[Release] HTTP Anidb Metadata Agent (HAMA)

13468981

Comments

  • sparanoidsparanoid Posts: 7Members, Plex Pass Plex Pass

    @ZeroQI: Studio icons support is awesome! I think I can help you with that. just downloaded from https://sites.google.com/site/anidblist/AnimeStudios.zip%5D and I can rearrange them to the proper size. However their are about 120 studio images so I can't provide an estimate date.

  • TazgodXTazgodX Posts: 30Members, Plex Pass Plex Pass

    Thanks ZeroQI, i just replaced all parentheses with brackets and it works now. I didn't know if you had that coded in, but thanks!

    I was thinking about the Studio images, and I think if you made a HTM log file like you do for missing theme songs we could all contribute and find a couple that we need. I don't want to find 100+ studio graphics, but i don't mind trying to find 10 or 20 im missing.

    Just like the themes file just link a place to send it, and the restrictions on what you want it to be.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    I couldn't possibly have that coded in... To be precise it is the scanner (BABS or other) that translate filenames and folders into serie name, season, episodes, or more precisely the regular expressions (called regex) at the beginning of the scanner code. It is known that CRCs cause scanning problems and i noticed brackets at the end don't bother generally the detection... i generally use brackets for movies: "title (year) [special edition, crc, etc...].ext"

    The agent is to take the file showing and finding the serie accuratly, and downloading metadata (serie summary, poster, season posters, episode summaries, studio, producers, etc...). for a good understanding read the framework PDF

    The following Agent shows how to access files in local folder: https://forums.plex.tv/topic/38402-metadata-agents-for-exported-xbmc-library/. I am trying to find out how to access plex folders for the agent to check if the file exists then create a log

    @sparanoid&TazgodX: If you redo logos, you might want to modify directly Plex database so it gets included in the next Plex release for everybody https://github.com/plexinc-plugins/Media-Flags.bundle . Since an xml file lists all studio icons, i would like to avoid an issue where two package exists (movies, documentaries, channels) and people have to manually mix the two xml together...

    Also thanks for actually looking at the logs and giving feedback...

    I took me some time to do and i was under the impression nobody really used the agent since i had only 68 downloads on it...

    For logs, i wanted to allow common users to help the databases in small ways for everybody's benefits...

    BTW, if you enable web links, you will have links to anidb, tvdb and relation graphs straight from the serie summary.

    For the bible black hentai, naming works for me so advise you reinstall hama from the zip and update to latest code file on gihub and try again. try to use latest babs with tvdbid support even though you won't use that part

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • dirkheidenreichdirkheidenreich Posts: 7Members, Plex Pass Plex Pass
    ZeroQI wrote on May 16 2014, 7:46 AM: »

    Logs seem to point at my code crashing. sure you have the data folders created ?

    Yes, all folders in ./Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems

    I will try a reinstall and give feedback if it works.

    Thanks for your work.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    For studio icons, for a comparison, XBMC uses png file, white-on-clear, sized 161px x 109px, and are saved in 'skin.aeon.nox"/media/flags/studios/' for example. Already created ones: https://github.com/B...a/flags/studios. (http://forum.xbmc.org/showthread.php?tid=70398 show two sources for icons)

    On Plex however, it uses 512x288px .png located in '/volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Media-Flags.bundle/Contents/Resources/Studio/'. substitutions.xml file in parent folder contain the mappings and needs amendind

    <match name="Animal_Planet" expression="animalplanet" />
    <match name="Animal_Planet" expression="Animal_Planet" />
    <match name="Animal_Planet" expression="animal planet" />
     
       . name is the name of the pnd, spaces and hyphens are mostly converted to underscores (remember reading about spaces not being allowed for filename but could spot exceptions in the xml)
       . expression is the name in the databases (since logos are used by plex, so movies and series agents will also use them) [one line per expression, can point to the same name]
    Video and audio codecs also have logos (Hi10p and FLAC logos would be cool)
     
    Didn't had time to improve logging but it's on the to do list

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • Mord_SithMord_Sith Posts: 2Members, Plex Pass Plex Pass

    Excellent, that guide helped me get it rolling even with my FreeNAS plex version going ploink on me after the weekend!

    I would strongly encourage that guide be made a little more visible, might help folks other than myself and prevent them from asking the same question later. (not the how to plug in from Plex mind you, though that link is handy)

    Great work, my library's spooling up and getting all the right imagery now!

  • erwinsieerwinsie Posts: 56Members ✭✭

    is 0.5 alpha still the latest release? just want to make sure since the date is 2013...

  • arithinearithine Posts: 32Members
    edited May 2014

    I am trying to add, http://anidb.net/perl-bin/animedb.pl?show=anime&aid=5868 and I just can't get it to recognize no matter what I do, I tried searching the main title, I tried searching the official title, I tried searching the aid(using aid:5868 aid=5868 5868 a5868), I tried removing the - from the title, I tried removing both the - and the ! from the title, I tried simply searching negima, I tried all combinations with and without dates (both 2008 and 2009) and all I get is "No matches found"

    Edit: Actually it looks like I can't get any searches to go through, period(the data for most of my shows was downloaded by thetvdb), I had to resort to using myanimelist

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014
    Well i didn't really increment it when coding so i would go by the date at the beginning of the __code__.py page... The latest code is always on Github.
    I did gather all process and info in the first post to increase the adoption rate and reduce the amount of useless questions, not that there have been that many but still... 
     
    @arithine: "Mahou Sensei Negima Shiroki Tsubasa Ala Alba" is how i would label it
    Went into the test "Bible Black New Testament" i had created for the hentai fanboy earlier in forum, went to incorrect match, searched, and voila, also "aid:5868" works as workaround as well. So in short, you did something wrong. Did you update the "__CODE__.py" file without creating the data folders ? I created an install procedure that should assist you. also, if no search go through, then looking at logs will tell you WHY.
     
    I need to make it nor fail when folders are missing when i have time, and make it create a report in case of errors that you can attach here so i can reply in one post on how to solve or ammend the code. That is as soon as i have enough time to do that of course...

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass
    edited May 2014

    Hey,

    So been using HAMA on and off, flicking between TVDB, MyAnimeList and old Anidb directly but noticed in the last few weeks its stopped pulling data, artwork, posters etc either when told to use AniDB or TVDB in the settings but it definitely is matching the series no problem.  Checking the logs only thing that stands out is;

    2014-05-25 20:46:05,323 (1d2c) :  CRITICAL (core:561) - Exception in the update function of agent named 'HamaTV', called with guid 'com.plexapp.agents.hama://6645?lang=en' (most recent call last):
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 970, in _update
        agent.update(obj, media, lang, force)
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 840, in update
        def update(self, metadata, media, lang, force ): self.parseAniDBXml(metadata, media, force,      False     )
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 541, in parseAniDBXml
        elif "s"+defaulttvdbseason+"e"+epNumVal in tvdbSummary:              tvdb_ep = "s"+defaulttvdbseason+"e"+epNumVal
    ValueError
     

    Anyone else seen something like this? I did a fresh install to try and get everything done by one agent and this is when its become an issue.  I have downloaded the latest from Github, created the folders etc

    Running off Windows - 0.9.9.10.458-008ea34

    2014-05-25 21:32:19,855 (19b4) :  DEBUG (networking:172) - Requesting 'http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=7405'
    2014-05-25 21:32:19,871 (13e8) :  DEBUG (logkit:13) - SearchByName: Local exact search for 'Nazo No Kanojo X' matched aid: 8853 Nazo no Kanojo X
    2014-05-25 21:32:19,875 (13e8) :  DEBUG (runtime:918) - Response: [200] str, 632 bytes
    2014-05-25 21:32:19,996 (13a4) :  DEBUG (logkit:13) - SearchByName: Local exact search for 'Nekomonogatari Black' matched aid: 9453 Nekomonogatari Black
    2014-05-25 21:32:20,003 (13a4) :  DEBUG (runtime:918) - Response: [200] str, 640 bytes
    2014-05-25 21:32:20,085 (1e54) :  DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.hama/messaging/function/X0FnZW50S2l0OlNlYXJjaA__/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoyCmRpY3QKZGljdAo0CnMyCmVuczQKbGFuZ3M3ClRWX1Nob3dzMTAKbWVkaWFfdHlwZWIwczYKbWFudWFscjEKczYKa3dhcmdzOQpzMTYKNWMxYWY2NTJkNzM4MmI2YXMxNwpvcGVuU3VidGl0bGVzSGFzaHMxCjVzNwplcGlzb2RlczEKMXM4CmVwaXNvZGljczIzCk51cnNlIFdpdGNoIEtvbXVnaSBDaGFuczQKc2hvd3MxCjFzNgpzZWFzb25zOTMKRSUzQSU1Q0FuaW1lJTVDTnVyc2UlMjBXaXRjaCUyMEtvbXVnaS1jaGFuJTVDTnVyc2UlMjBXaXRjaCUyMEtvbXVnaS1jaGFuJTIwLSUyME9WQSUyMDA1JTJFYXZpczgKZmlsZW5hbWVzNDAKMjk5NzUzZDRhOTk5MWQ0MTU0YzJlMjI3MDQ2MzcyYTEwM2U3MjhlOXM4CnBsZXhIYXNoczIKLTFzOApkdXJhdGlvbnM0CjgxOTJzMgppZHIwCg__
    2014-05-25 21:32:20,085 (1e54) :  DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.hama/messaging/function/X0FnZW50S2l0OlNlYXJjaA__/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoyCmRpY3QKZGljdAo0CnMyCmVuczQKbGFuZ3M3ClRWX1Nob3dzMTAKbWVkaWFfdHlwZWIwczYKbWFudWFscjEKczYKa3dhcmdzOQpzMTYKNWMxYWY2NTJkNzM4MmI2YXMxNwpvcGVuU3VidGl0bGVzSGFzaHMxCjVzNwplcGlzb2RlczEKMXM4CmVwaXNvZGljczIzCk51cnNlIFdpdGNoIEtvbXVnaSBDaGFuczQKc2hvd3MxCjFzNgpzZWFzb25zOTMKRSUzQSU1Q0FuaW1lJTVDTnVyc2UlMjBXaXRjaCUyMEtvbXVnaS1jaGFuJTVDTnVyc2UlMjBXaXRjaCUyMEtvbXVnaS1jaGFuJTIwLSUyME9WQSUyMDA1JTJFYXZpczgKZmlsZW5hbWVzNDAKMjk5NzUzZDRhOTk5MWQ0MTU0YzJlMjI3MDQ2MzcyYTEwM2U3MjhlOXM4CnBsZXhIYXNoczIKLTFzOApkdXJhdGlvbnM0CjgxOTJzMgppZHIwCg__
    2014-05-25 21:32:20,085 (1e54) :  INFO (agentkit:915) - Searching for matches for {'openSubtitlesHash': '5c1af652d7382b6a', 'episode': '5', 'episodic': '1', 'show': 'Nurse Witch Komugi Chan', 'season': '1', 'filename': 'E%3A%5CAnime%5CNurse%20Witch%20Komugi-chan%5CNurse%20Witch%20Komugi-chan%20-%20OVA%2005%2Eavi', 'plexHash': '299753d4a9991d4154c2e227046372a103e728e9', 'duration': '-1', 'id': '8192'}
    2014-05-25 21:32:20,114 (1e54) :  DEBUG (networking:172) - Requesting 'http://127.0.0.1:32400/library/metadata/8192/tree'
    2014-05-25 21:32:20,201 (19b4) :  CRITICAL (core:561) - Exception in the update function of agent named 'HamaTV', called with guid 'com.plexapp.agents.hama://7405?lang=en' (most recent call last):
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 970, in _update
        agent.update(obj, media, lang, force)
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 840, in update
        def update(self, metadata, media, lang, force ): self.parseAniDBXml(metadata, media, force,      False     )
      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 314, in parseAniDBXml
        if movie and orig != "" and orig != metadata.original_title: metadata.original_title = orig # If it's a movie, Update original title in metadata http://forums.plexapp.com/index.php/topic/25584-setting-metadata-original-title-and-sort-title-still-not-possible/
    ValueError

    ***Bah am now Banned for a bit on Anidb but as issue was present before starting afresh not sure what this issue is, will wait til ban clears.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014

    By experience the line the error is reported on is quite off seem line 561 is where it failed when 541 is the latest reported... Please attach the full log on your prev post so I can spot which serie and episode triggered the error. I reproduce by creating same folder structure with sample video files since the content is irrelevant...

    [EDIT:] PLEASE ATTACH FILES, not the content of them, it is not interesting for other peoples, it really tricky to see on a phone and i get less data

    That being said, please document " I have downloaded the latest from Github" because... well... you didn't... Here is why:

    This is line 334 on github:    if movie and orig != "" and orig != metadata.original_title: metadata.original_title = orig # If it's a movie,...

    This is line 314, on your code   if movie and orig != "" and orig != metadata.original_title: metadata.original_title = orig # If it's a movie, 

    Please update __code__.py with latest from git

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass
    edited May 2014

    Very odd, the log may of been when trying to compare and attempting the various versions.  But deleted plugin and redownloaded it. Same issue it matches series in this case Zombie Loan, it pulled two genres Horror and Manga both rated 600 but had present at 600 as well and several others over 200.

    Update - Some reason plex isn't accessing https://rawgit.com/ScudLee/anime-lists/master/anime-movieset-list.xml but browsers can, so modified the code to just go to http://rawgit.com/ScudLee/anime-lists/master/anime-movieset-list.xml and it started pulling images.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014

    Only two are ever directly displayed, pretty sure you see them all when clicking edit > Tags > Genre...

     
    Now the log:
    2014-05-26 01:22:33,010 (28c) :  DEBUG (logkit:13) - xmlElementFromFile - Loading XML file from url failed: https://rawgithub.com/ScudLee/anime-lists/master/anime-list-master.xml
    2014-05-26 01:22:33,012 (28c) :  DEBUG (logkit:13) - xmlElementFromFile - Loading XML file from Data folder failed: XMLs/anime-list-master.xml
     
    (logkit:13) - xmlElementFromFile - Loading XML file from url failed: https://rawgithub.com/ScudLee/anime-lists/master/anime-movieset-list.xml
    2014-05-26 01:22:36,726 (1418) :  DEBUG (logkit:13) - xmlElementFromFile - Loading XML file from Data folder failed: XMLs/anime-movieset-list.xml
     
    Well the agent cannot see https://rawgithub.com/ScudLee/anime-lists/master/anime-movieset-list.xml and cannot load it from data folder neither
    i tried the URL and it works for me. try loading it and saving it in data folder > XMLs > anime-movieset-list.xml. possibly you have a firewall or no internet on the given machine
     
    Have updated the source on git with:
       . download on http instead of https to reduce issues
       . It also can look in data folder and resource folder if you cannot download the base xml files
       . logging improvement
       . include "clear-cache" search title to clear the Plex cache
    I have attached a recent version that is supposed to work without the data folders created, although i recommend them
    Please let me know on this thread if anything goes wrong with that one
    I will probably be working on the xml cache in data folder

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass
    edited May 2014

    Hey :)

    Will attempt the latest you just posted, but since getting round the https http issue last night its been busy downloading metadata all day.  Gundam evolve is the one in question at the moment, there are others but attached the log just for it.

    It's definitely on the internet its my "play with/attempt everything" machine so probably something done to it at some point.

    Thanks

    Note - attached bundle has error

    File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 320
        except: raise ValueEr  ror

    except: raise ValueEr  ror looks like you added some spaces to that line, removed on mine and confirmed it's launching again once spaces removed.
     

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014

    @ shikaiwraith: thanks for the feedback, uploaded the fixed version but 17 people downloaded it already. It also proved people download on first page at 40-50% proportion so will continue to amend the first page with stable release

    new version use http only to download XMLs, fallback to latest downloaded XML if folder structure was done, and include a local copy just in case in ressources, in case people don't create folder structure in Data folder

    Please let me know if it get banned during scraping after how many series it happens (important, but i did everything to prevent that), and if anything else happens.

    This should be a stable release so please any of you feedback issues. updated agent in first page since many people used that version instead of latest, or didn't know how to update from git.

    [Edit] I need to solve that, i have seen it pop up few times, but the line pasted is wrong, the error seem to be when loading the XML as the title searched should have output something before the error we see pointed out...

    I posted edits on github to output a bit more after the file loading. could you update and paste the log again as you just did ?

    Did you try to actually remove open subtitles from agent order by unticking it?: (just to make sure the issue is with HAMA)

    Searching for matches for {'openSubtitlesHash': '7eb6de792040329c', 'episode': '10', 'episodic': '1', 'show': 'Gundam Evolve', 'season': '1', 'filename': 'G%3A%5CAnime%5CGundam%20Evolve%5CGundam%20Evolve%20-%20Omega%20(Episodes%206-10)%20[Solar][09D761B0]%2Emkv', 'plexHash': '2cb018f8afedab4f255c1ba84a7138af21cb9cdf', 'duration': '2693694', 'id': '23256'}

     
     

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass
    edited May 2014

    :) So progress is made.

    Think the ban was totally my fault, where I had tried refreshing/wiping and starting again and pulling too many requests in single day, as when using my mates pc on different IP he never got banned.

    So tried 3 series that failed before

    Gundam Evolve - metadata pulled

    Groove Rave Adventure - got genres and failed to do anything more

    ikkitousen: xtreme xecutor - metadata pulled

    Got quite a few pictures corrupted now as well, i.e grey bars etc but can't actually refresh poster to redownload new ones so manually redoing it.  They also if get series with follow on series they all get the same poster i.e. ikkitousen, initial d, votoms, macross etc but the I guess proper picture for the series or at least the unique picture tends to be the last one on the poster list.

    New errors in the logs about files not being there and I don't use open subtitles as all, going to try a refresh of library and see what else sticks out.

    Update - so think half issues are plex, refresh helped some items, others fix mismatch, doing a manual search and it pulled data but several series that still want to play ball;

    Groove Rave Adventure

    Kaikan Phrase

    Kujakuou

    Shin Kujakuou

    I have tried with caching ticked in options and without.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014

    THANKS! i could spot 3 things:

    TypeError: not enough arguments for format string

    ========================================

       . Serie: Groove Adventure Rave:

       . TVDB XML empty http://thetvdb.com/api/A27AD9BE0DA63333/series/252688/banners.xml

       . TVDB serie page: http://thetvdb.com/?tab=series&id=252688&lid=7

       . serie has no posters... does this cause the issue?

    not sure if issue related though but too much of a coincidence

    IOError: [Errno 2] No such file or directory: u'\\\\?\\Z:\\Plex Server\\Application Data\\Plex Media Server\\Plug-in Support\\Data\\com.plexapp.agents.hama\\DataItems\\TVDB\\seasonswide\\._80158-4.jpg'

    =======================================================================

    In GetImagesFromTheTVDB: Data.Save(filename, poster), if folder not there, trouble

    need to put a setting for local cache and set off by default

    (content type 'image/jpeg' not cacheable in Agent plug-ins)

    ===============================================

    Need to not cache again pictures in plex

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass
    edited May 2014

    Agreed, my wrong assumption if had AniDB ticked it would pull posters there and tvdb as well rather than one.  Bit weird they have pictures and things for the spanish version not english looking closer at it.

    Refresh again didn't work once I unticked all thetvdb stuff, had to do fix incorrect match, manual search and click that way to pull even though it was the same link as automatic and it eventually pulled each episode just from anidb.

    That has now fixed the remaining 4 with log attached, now to go through and fix the corrupted posters manually relinking them.

    Cheers

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited May 2014

    @shikaiwraith: thanks for the logs, i can see it working ok now apart from one crash on "Groove Adventure Rave" only. 

    When searching for "Groove Adventure Rave" on TheTVDB.com there is two hits:
    . "Groove Adventure Rave", a Spanish title with tvdbid 72973 (and english title "Rave Master") with english posters
    . "Groove Adventure Rave", an english title with tvdbid 252688 and no posters
     
    I submitted a request on TVDB forum to have the duplicate issue reviewed: http://forums.thetvdb.com/viewtopic.php?f=7&t=18365

    I have raised an issue on Github to have the mapping file amended to point to the series with posters: https://github.com/ScudLee/anime-lists/issues/37

    From what you are saying, update didn't work. If i understand it right,update pull info if no info was there before, since it crashed previously, you need to do a full metadata pull, which you did doing the manual search. That means if you had this version you have now, it wouldn't have crashed in the first place.

    I can only see an issue that popped up: TypeError: not enough arguments for format string

      File "Z:\Plex Server\Application Data\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 602, in parseAniDBXml
        elif "s"+defaulttvdbseason+"e"+str(epNumVal) in tvdbSummary:    tvdb_ep = "s"+defaulttvdbseason+"e"+str(epNumVal)
     
    it seem like the very same serie works afterwards so i am puzzled
     
    The first difference is: (cache/not cached) when the same serie was scrapped with no issue just afterwards
    440: try: tvdbanime = self.urlLoadXml( TVDB_HTTP_API_URL % (TVDB_API_KEY, tvdbid), TVDB_SERIE_CACHE+"/"+tvdbid+".xml" ).xpath('/Data')[0
    I have added logs in the urlLoadXml function to narrow down that bug i cannot narrow down more currently

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • shikaiwraithshikaiwraith Posts: 6Members, Plex Pass Plex Pass

    Yeah you would think a refresh with no posters would attempt to look for posters though? Just had fun getting some unmatched series to go through hate series naming conventions

    Alternative universes really cause chaos lol

    Classic series Tenchi Muyo! Ryo-Ohki http://thetvdb.com/?tab=series&id=80009&lid=7 1 series

    vs

    anidb listing as 3 series (nothing unusual just naming convention of having (airdate year)

    http://anidb.net/perl-bin/animedb.pl?show=anime&aid=205 Tenchi Muyou! Ryououki

    http://anidb.net/perl-bin/animedb.pl?show=anime&aid=2509 Tenchi Muyou! Ryououki (1994)

    http://anidb.net/perl-bin/animedb.pl?show=anime&aid=2510 Tenchi Muyou! Ryououki (2003)

    Scan went through it finds the folders find just doesn't like matching them and leaves me with just the first series http://anidb.net/perl-bin/animedb.pl?show=anime&aid=205 Tenchi Muyo! Ryo-Ohki. To get round it had to take away the brackets on the folders so they read Tenchi Muyou! Ryououki, Tenchi Muyo! Ryo-Ohki 1994 and Tenchi Muyo! Ryo-Ohki 2003 as it didn't like looking inside the brackets with the folder names

13468981
Sign In or Register to comment.