If you have not already, we suggest setting your Plex username to something else rather than email which is displayed on your posts in forum. You can change the username at https://app.plex.tv/desktop#!/account
Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

[Release] HTTP Anidb Metadata Agent (HAMA)

1356781

Comments

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    https://forums.plex.tv/topic/66658-multiple-episodes-in-one-file/

    I get you now, but the functionality is not implemented in Plex and a metadata agent doesn't manage file level, which is handled by the scanner, but even within the scanner, not sure if it can be coded for now

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • floatymanfloatyman Posts: 9Members, Plex Pass Plex Pass

    Hey thank you so much for this agent. It took a bit of re-reading of this thread to get it working but I generally have it working now. I do have a couple of things that are not working though.

    1. Series with long names are not being picked up (eg: Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru). Is there a character limit?

    2. OVAs and Specials are still not being picked up. Even specials that are classed as their own season on TVDB (eg: Black Lagoon Roberta's Blood Trail) are not picked up.

    3. I did see something about it but didn't see a solution, but I don't have genres coming through either.

    Thanks again and hopefully someone can tell me what's going on.

  • SirSmegheadSirSmeghead Posts: 14Members, Plex Pass Plex Pass

    After a little bit more testing, it seems the only way to get Plex to recognize two episodes in one file requires it to be renamed using the regular naming conventions (One_Piece_s01e215-e216).mp4 as example, seems there is no other way for Plex file scanner to use the absolute order episode names for such files unfortunately.

    Guess the file scanner itself would need modification to accept dual episodes in one file for absolute ordering.

    Time flies like the wind

    Fruit flies like bananas

  • SakuryuSakuryu Posts: 9Members ✭✭

    Movies aren't getting any Metadata other than the year. Tried with 34 movies with the same result each time.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    Issues reported:

       . long filename series (Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru)

       . OVAs and Specials are still not being picked up. Even specials that are classed as their own season on TVDB (eg: Black Lagoon Roberta's Blood Trail) are not picked up. TVDB is irrelevant for numbering (only covers and eps summaries)

      .  genres not coming through 

    Did the following, which should be done before reporting:
       . Configured test category as: TV series/Movies, Scanner: BABS, agent: HamaTV/HamaMovie, Added parent folder of "Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru"

       . Updated:

    /volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-list-master.xml
    /volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-movieset-list.xml
    /volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-titles.xml

       . http://anidb.net/api/anime-titles.xml.gz

    Restarted Plex

    scanned folder, and the serie "Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru" comes up, with poster, and episode summaries but no genre

    one piece however got genres

    Looked at logs located here:

       . /volume1/Plex/Library/Application Support/Plex Media Server/Logs/PMS Plugin Logs/com.plexapp.agents.hama.log

    2014-02-25 09:36:12,940 (3800b) : DEBUG (__init__:161) - === searchByName - Begin - ================================================================================================
    2014-02-25 09:36:12,940 (3800b) : INFO (__init__:162) - Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru
    2014-02-25 09:36:12,941 (3800b) : INFO (__init__:163) - Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru
    2014-02-25 09:36:12,942 (3800b) : INFO (__init__:164) - Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru
    2014-02-25 09:36:12,943 (3800b) : INFO (__init__:165) - Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru
    2014-02-25 09:36:12,943 (3800b) : INFO (__init__:166) - SearchByName (MediaContainer(art=None, noHistory=False, title1=None, title2=None, replaceParent=False),en,Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru,None)
    2014-02-25 09:36:13,415 (3800b) : DEBUG (__init__:217) - SearchByName: Local exact search for 'Ore No Nounai Sentakushi Ga, Gakuen Lovecome O Zenryoku De Jama Shiteiru' matched aid: 9787 Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru
    2014-02-25 09:36:13,432 (3800b) : DEBUG (runtime:915) - Response: [200] str, 708 bytes
    2014-02-25 09:36:13,523 (3c00b) : DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.hama/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKNwpzMgplbnM0CmxhbmdiMXM1CmZvcmNlbnMxMApwYXJlbnRHVUlEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczUKODY4NjRzNApkYmlkczM4CmNvbS5wbGV4YXBwLmFnZW50cy5oYW1hOi8vOTc4Nz9sYW5nPWVuczQKZ3VpZHM0Cjk3ODdzMgppZHIwCg__
    2014-02-25 09:36:13,525 (3c00b) : DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.hama/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKNwpzMgplbnM0CmxhbmdiMXM1CmZvcmNlbnMxMApwYXJlbnRHVUlEczcKVFZfU2hvd3MxMAptZWRpYV90eXBlczUKODY4NjRzNApkYmlkczM4CmNvbS5wbGV4YXBwLmFnZW50cy5oYW1hOi8vOTc4Nz9sYW5nPWVuczQKZ3VpZHM0Cjk3ODdzMgppZHIwCg__
    2014-02-25 09:36:13,526 (3c00b) : DEBUG (model:32) - Loading model with GUID com.plexapp.agents.hama://9787?lang=en
    2014-02-25 09:36:13,537 (3c00b) : ERROR (model:194) - Cannot read model from /volume1/Plex/Library/Application Support/Plex Media Server/Metadata/TV Shows/1/e0039a3f03a70c62f0ca548198ff585f9631212.bundle/Contents/com.plexapp.agents.hama
    2014-02-25 09:36:13,538 (3c00b) : DEBUG (networking:172) - Requesting 'http://127.0.0.1:32400/library/metadata/86864/tree'
    2014-02-25 09:36:13,552 (3c00b) : DEBUG (__init__:298) - --- Begin
    
    2014-02-25 09:36:13,553 (3c00b) : DEBUG (__init__:299) - parseAniDBXml (<Framework.models.metadata.com_plexapp_agents_hama.TV_Show object at 0xf4bf17ac>, <Framework.api.agentkit.MediaTree object at 0xf4bf278c>, True)
    2014-02-25 09:36:13,555 (3c00b) : DEBUG (__init__:302) - AniDB Serie XML: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=9787
    2014-02-25 09:36:13,615 (3c00b) : DEBUG (networking:172) - Requesting 'http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=9787'
    2014-02-25 09:36:14,707 (3c00b) : DEBUG (__init__:544) - AniDB title: 'Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru' original title: 'Ore no Nounai Sentakushi ga, Gakuen Lovecome o Zenryoku de Jama Shiteiru'
    2014-02-25 09:36:14,724 (3c00b) : DEBUG (__init__:545) - AniDB Start Date: 2013-10-10
    2014-02-25 09:36:14,725 (3c00b) : DEBUG (__init__:545) - AniDB Ratings: 4.49
    2014-02-25 09:36:14,733 (3c00b) : DEBUG (__init__:545) - AniDB Genres (Weight): [('Ecchi', 600), ('Novel', 600), ('High School', 600), ('Comedy', 600), ('Harem', 400), ('Shounen', 400), ('Present', 300), ('Parody', 300), ('Transfer Student', 200)]*
    2014-02-25 09:36:14,743 (3c00b) : DEBUG (__init__:776) - anidbCollectionMapping - related_anime_list: ['10208']
    2014-02-25 09:36:14,758 (3c00b) : DEBUG (__init__:793) - anidbCollectionMapping - anidbid is not part of any collection:
    2014-02-25 09:36:14,759 (3c00b) : DEBUG (__init__:545) - AniDB Creator data: Kasukabe Takeru is writer, Saita Hiroyuki is director, Inagaki Takayuki is director, Kanasugi Hiroko is producer, Hara Yumiko is director, Noda Yasuyuki is director, Studio: Diomedea,
    2014-02-25 09:36:14,760 (3c00b) : DEBUG (__init__:545) - TVDB - AniDB-TVDB mapping file
    2014-02-25 09:36:14,989 (3c00b) : DEBUG (__init__:661) - AniDB-TVDB Mapping - anidb:9787 tvbdid: 272314 studio: defaullttvdbseason: 1
    2014-02-25 09:36:15,029 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/api/A27AD9BE0DA63333/series/272314/banners.xml'
    2014-02-25 09:36:15,662 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/_cache/fanart/original/272314-4.jpg'
    2014-02-25 09:36:16,058 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/_cache/fanart/original/272314-4.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:16,059 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/fanart/original/272314-4.jpg
    2014-02-25 09:36:16,109 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/_cache/fanart/original/272314-1.jpg'
    2014-02-25 09:36:16,762 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/_cache/fanart/original/272314-1.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:16,763 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/fanart/original/272314-1.jpg
    2014-02-25 09:36:16,823 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/posters/272314-2.jpg'
    2014-02-25 09:36:18,496 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/posters/272314-2.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:18,497 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/posters/272314-2.jpg
    2014-02-25 09:36:18,560 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/posters/272314-1.jpg'
    2014-02-25 09:36:21,262 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/posters/272314-1.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:21,263 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/posters/272314-1.jpg
    2014-02-25 09:36:21,326 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/seasons/272314-1.jpg'
    2014-02-25 09:36:22,331 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/seasons/272314-1.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:22,332 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/seasons/272314-1.jpg
    2014-02-25 09:36:22,407 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/banners/graphical/272314-g.jpg'
    2014-02-25 09:36:22,843 (3c00b) : DEBUG (networking:200) - Not caching 'http://thetvdb.com/banners/graphical/272314-g.jpg' (content type 'image/jpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:22,844 (3c00b) : DEBUG (__init__:739) - getImagesFromTVDB - adding url: http://thetvdb.com/banners/graphical/272314-g.jpg
    2014-02-25 09:36:22,844 (3c00b) : DEBUG (__init__:741) - getImagesFromTVDB - Item number: 8, poster en: 2, Poster ids: 985260, 974435,
    2014-02-25 09:36:22,895 (3c00b) : DEBUG (networking:172) - Requesting 'http://thetvdb.com/api/A27AD9BE0DA63333/series/272314/all/en.xml'
    2014-02-25 09:36:23,360 (3c00b) : DEBUG (__init__:545) - TVDB - loaded serie xml: 272314 NouCome
    2014-02-25 09:36:23,360 (3c00b) : DEBUG (__init__:545) - TVDB - Build 'tvdbSummary' table
    2014-02-25 09:36:23,380 (3c00b) : DEBUG (__init__:545) - TVDB - Build 'tvdbSummary' table:[' s0e1', ' s1e1', ' s1e10', ' s1e2', ' s1e3', ' s1e4', ' s1e5', ' s1e6', ' s1e7', ' s1e8', ' s1e9']
    2014-02-25 09:36:23,797 (3c00b) : DEBUG (__init__:545) - url exist: 200
    2014-02-25 09:36:23,868 (3c00b) : DEBUG (networking:172) - Requesting 'http://tvthemes.plexapp.com/272314.mp3'
    2014-02-25 09:36:26,137 (3c00b) : DEBUG (networking:200) - Not caching 'http://tvthemes.plexapp.com/272314.mp3' (content type 'audio/mpeg' not cacheable in Agent plug-ins)
    2014-02-25 09:36:26,137 (3c00b) : DEBUG (__init__:545) - AniDB Poster
    2014-02-25 09:36:26,138 (3c00b) : DEBUG (__init__:545) - AniDB description + link
    2014-02-25 09:36:26,139 (3c00b) : DEBUG (__init__:545) - AniDB resources link
    2014-02-25 09:36:26,172 (3c00b) : DEBUG (__init__:563) - AniDB-TVDB mapping episode Summaries - mapped eps: None, Missing eps: None
    2014-02-25 09:36:26,173 (3c00b) : DEBUG (__init__:576) - --- end
    
    
       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/anime-list.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/themes.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/TVDB.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/AniDB.htm

     
    The links in the web page allow to update said databases, not only Hama agent but TheTVDB for posters and episodes summaries, AniDB for information, Plex for 30s TV themes, animelist for the xml mapping file (for plex+xbmx AniDB agents)
    I need logs from com.plexapp.agents.hama.log posted as it teels me what's happenning. I spent a lot of time to make readable logs so please paste them. I believe my version of the agent get banned less and is fast with the minimum requests to anidb so help me improve it further
     
    So the genres gets pulled in line 545. Bug of my code there. 

     DEBUG (__init__:545) - AniDB Genres (Weight): [('Ecchi', 600), ('Novel', 600), ('High School', 600), ('Comedy', 600), ('Harem', 400), ('Shounen', 400), ('Present', 300), ('Parody', 300), ('Transfer Student', 200)]*

    Downloaded Black Lagoon: Roberta`s Blood Trail" and renamed "by removing ":" as Anidb consider it a serie on its own.

    Scanned, and works perfect, poster, summary, even genres.

    So no real issue, apart from the genres which i need to investigate.

    The root cause might be bad mode selection(movie/serie) when creating plex category or not up to date data files or not using BABS scanner... 

    /volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/DefaultPrefs.json contains settings (language drop down doesn't work YET)

    /volume1/Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Code/__init__.py 

    it contains the following at the beginning. Might try to move all settings to the DefaultPrefs.json file if i can

     
    ### Language Priorities ###
    SERIE_LANGUAGE_PRIORITY      = [ 'x-jat', 'en']
    EPISODE_LANGUAGE_PRIORITY    = [ 'en', 'x-jat']
    SECONDS_BETWEEN_REQUESTS     = 2
    FILTER_CHARS                 = "\\/:*?<>|~- "
    SPLIT_CHARS                  = [';', ',', '.', '~', '-' ] #Space is implied
    WEB_LINK                     = "<A HREF='%s' target='_blank'>%s</A>"
    
    ### These are words which cause extra noise due to being uninteresting for doing searches on ###########################################################################
    FILTER_SEARCH_WORDS          = [                                                                                                      # Lowercase only
      'a',  'of', 'an', 'the', 'motion', 'picture', 'special', 'oav', 'ova', 'tv', 'special', 'eternal', 'final', 'last', 'one', 'movie', # En 
      'princess', 'theater',                                                                                                              # En Continued
      'to', 'wa', 'ga', 'no', 'age', 'da', 'chou', 'super', 'yo', 'de', 'chan', 'hime',                                                   # Jp 
      'le', 'la', 'un', 'les', 'nos', 'vos', 'des', 'ses'                                                                                 # Fr 
    ]
    

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited February 2014

    For the genre empty, i think i solved it

    http://carlgroner.me/Python/2011/11/09/An-Introduction-to-List-Comprehensions-in-Python.html

    Had to rethink my list comprehention... the point is to update the categories only if all categories are not 100% correct
     

    https://github.com/ZeroQI/Hama/blob/master/Hama.bundle/Contents/Code/__init__.py

    __init__.py line 345 if all(x in genres for x in metadata.genres) is true nothing happens, which in plain english is: Do X in genres for each x in metadata.genres. What if metadata.genres is empty? i might have my bug...

    Line 345 I replaced  "if all(x in genres for x in metadata.genres)" with " if all(x in metadata.genres for x in genres)" and it solved the issue in my instance.

    What should i implement? 

       . language working under agent settings ?

      .  Better collection functionality ? It kinda works using anidb related series, but i could use multiple iteration of the scan to make it work fully over time... It currently use movie collection which is wired in the XML and i piggy back the series in it...

       . Separate theme songs for seasons ? if so i would have to get a massive amount of theme songs, and label them by the anidbid... could use most tvdbid songs for first seasons to just complete with the missing ones...

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • floatymanfloatyman Posts: 9Members, Plex Pass Plex Pass

    Thanks so much for the patient help. I think most of my problems have been solved, with most of it being on my end. I'll post what I found as it may help some people in future who have similar problems.

    Genres now work with the latest release on Github. I don't know if ZeroQI can change the ratings or bring more in because it is picking up some odd genres. For example the two that come through for One Piece are 'Underworld' and 'Racism' XD

    OVAs and Specials now come through for me. The problem was that if the 'series' had less than 10 episodes, WebAOM which I use for my renaming, would only have a single digit for the number but adding a zero in-front of the number means it gets picked up properly (eg: 01 instead of 1).

    Series with long names are now also being picked up and once again it was a problem with my naming conventions. I was having my anime named 'Romaji (English)/(Year) Type = Romaji [Source]/Romaji - Ep Num - Ep Name [Res] [Group] [CRC]. With all those characters, especially long episode names it was getting confused so I have changed my naming to 'Romaji [English]/Romaji - Ep Num [Res] [Source] [Group] [CRC]. Having information you don't need/want scanned between [ ] seems to help.

    Thanks to ZeroQI for all the help and patience. 

     
    ZeroQI wrote on February 25 2014, 5:26 PM: »

    For the genre empty, i think i solved it

    http://carlgroner.me/Python/2011/11/09/An-Introduction-to-List-Comprehensions-in-Python.html

    Had to rethink my list comprehention... the point is to update the categories only if all categories are not 100% correct
     

    https://github.com/ZeroQI/Hama/blob/master/Hama.bundle/Contents/Code/__init__.py

    __init__.py line 345 if all(x in genres for x in metadata.genres) is true nothing happens, which in plain english is: Do X in genres for each x in metadata.genres. What if metadata.genres is empty? i might have my bug...

    Line 345 I replaced  "if all(x in genres for x in metadata.genres)" with " if all(x in metadata.genres for x in genres)" and it solved the issue in my instance.

    What should i implement? 

       . language working under agent settings ?

      .  Better collection functionality ? It kinda works using anidb related series, but i could use multiple iteration of the scan to make it work fully over time... It currently use movie collection which is wired in the XML and i piggy back the series in it...

       . Separate theme songs for seasons ? if so i would have to get a massive amount of theme songs, and label them by the anidbid... could use most tvdbid songs for first seasons to just complete with the missing ones...

    Personally I would vote for, in order, Language, Collection, Theme Songs. Language and collection would be helpful while the theme songs are nice, could be the most problematic, especially if you start considering the shows that have different openings half way through their 'seasons'. That is my opinion anyway.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    Thanks for the descriptive feedback. The language selection for the title actually works, but not from the agent settings window although the selection is present.

    You can see it in the code https://github.com/ZeroQI/Hama/blob/master/Hama.bundle/Contents/Code/__init__.py#L10

    PS: you can do more than two, will take in the order in sequence

    ### Language Priorities ###
    SERIE_LANGUAGE_PRIORITY = [ 'x-jat', 'en']
    EPISODE_LANGUAGE_PRIORITY = [ 'en', 'x-jat']

    I haven't tested the Kanji display first but would be worth a try.

    The genre system i picked from the intial writter of the agent (AtomicStrawberry) that did a briliant job.

    I wanted more functions, consistent poster size (my OCD thing) and sum up for episodes and wanted it like the XBMC plugin

    The way it works is actually configurable, the agent list the genre that are useful, and take their weight. it takes by default anything with weight above 200

    http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=69

    Actually edit the genres, you will see there is more than 3. Plex just show the first 3 and they all carry a weight of 600 so pretty high

    So works as designed but if you wish i could remove sorting so it takes the 3 first genres displayed in anidb order, which would make it Navy pirate friendship which seem better that "UnderworldRacismPirate" but need to see the code and might take some time. Language working from the agent setigns it will be then.

    I just move out so might take a bit but will do. :D

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • floatymanfloatyman Posts: 9Members, Plex Pass Plex Pass

    I did see that you could do that in an earlier post, but I haven't played with it yet as I had to redo my WebAOM script and rename all my anime XD

    If that is how the genre system works it is a bit of a pain. Display order may be better in this case but it is near sure to be a problem somewhere. I had a look if TVDB genres would be better but most are just 'Animation' and one other generic genre so it isn't that useful either. Not sure what the best solution would be sorry.

    Good luck with your move!

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    The first 3 tags of highest weight (600) are the only one shows by default (one piece: underworld, racism, pirate for One piece) may not be the best, but nearly 20 tags (weight 200 and above) are pulled and you can see them if you click the edit button, and one the ones showing are weight 600 so not wrong on paper. Had AniDB managed better weight, there would be no issue, and the tags get pulled in the database and can be selected so i am not inclined to fix a cosmetic glitch, and one due to AnidDB categories weight for the serie, i use the data they have and am really grateful the site exists.

    On Github, i have uploaded the latest mod:

       . from the agent settings: triple language selection for series (support 'main' anidb title as valid selection) and double language selection for episodes titles.

       . Japanese titles support. (Seem like Plex cache keeps the title the first time metadata were called) and japanese kanji search works

    Improvements possible:

       . automatic files download once a week ? need to find how to decompress gz files in "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" with python...

       . Check collection algorithm to see if works properly

       . split TheTVDB missing posters log and and episode overview (summary) logs as people may do the posters if missing...

    Please let me know if you experience any but, and what functionality is missing or could be improved

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • XellossMetalliumXellossMetallium Posts: 9Members ✭✭
    edited March 2014

    Hi ZeroQI, I just installed HAMA metadata agent, I recall using the old version some months ago. Anyway with the anidb agent I had banning issues, so here I am. 

    Sadly nothing is working. I'm not reporting an issue, it is most likely a problem on my side (this is the first time I seriously try to configure plex), so I came to ask support. 

    The situation:
    My library is divided into many folders, anyway I select the single folder which contains the show folders. So something like: >#ANIME>#HI10>#1080P>

    My files are renamed with the anidb client like suggested in the anidb mod thread on xbmc: 

    Bakemonogatari - 1x01v2 - Hitagi Crab - Part 1 [Coalgirls][BD][1920x1080][86FE4DBE]

    Bakemonogatari - 0x02v3 - Tsubasa Cat - Part 3 [Coalgirls][BD][1920x1080][32AF6740]

    So episode 101,102 for op and ending and so on..

    I installed BABS scanner in AppData\Local\Plex Media Server\Scanners\Common  (and by installing I mean pasted here and restarted)

    Same for HAMA in AppData\Local\Plex Media Server\Plug-ins\Hama.bundle (pasted and restarted)

    I populated the folder "anime-lists" with the 3 xml.

    Selected the above resources for the "Anime" library.

    Nothing is actually parsed. No cover, no data, "NaN" duration and so on, so I assume that I got something wrong.

    https://dl.dropboxusercontent.com/u/37448634/com.plexapp.agents.hama.log

    https://dl.dropboxusercontent.com/u/37448634/Plex%20Media%20Scanner.log

    These are the logs, which I am not capable to read. If you need some others please tell me.

    Thanks and sorry for the bother!

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    From the log, it look like it cannot find the anime-titles.xml file.

     

    AniDB_title_tree = HamaCommonAgent().xmlElementFromFile(ANIDB_ANIME_TITLES , ANIDB_ANIME_TITLES_URL ) # AniDB's: anime-titles.xml
    File "C:\Users\Mario\AppData\Local\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 799, in xmlElementFromFile
    except: raise ValueError

     
    2014-03-27 10:43:07,075 (36b8) : DEBUG (logkit:13) - xmlElementFromFile - Loading XML file from Resources folder failed:anime-lists/anime-titles.xml
     
    present files should be in :

       . Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-titles.xml [from anidb]

       . Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-list-master.xml [from xbmc anidb mod]
       . Plex/Library/Application Support/Plex Media Server/Plug-ins/Hama.bundle/Contents/Resources/anime-lists/anime-movieset-list.xml  [from xbmc anidb mod]
     
    Where did you download it from? the old zip file in the thread or github directly ?
    Download the anime title [http://wiki.anidb.net/w/API] xml dump from http://anidb.net/api/anime-titles.xml.gz and decompress

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • XellossMetalliumXellossMetallium Posts: 9Members ✭✭
    ZeroQI wrote on March 27 2014, 11:17 AM: »

    From the log, it look like it cannot find the anime-titles.xml file.

     

    Where did you download it from? the old zip file in the thread or github directly ?

    Download the anime title [http://wiki.anidb.net/w/API] xml dump from http://anidb.net/api/anime-titles.xml.gz and decompress

    Yes, I downloaded the files from the link you provided in a previous post but failed to notice the .gz extension of anime-titles.. it was compressed.. I knew it was a stupid mistake, just not so stupid :) Thanks.

    Here my report:

    I added almost all my library.. looks like I got banned, maybe for downloading anime-titles.xml two times.. so I don't know if this is the reason, but for all the shows the "Episode Duration" is "NaN min." (only in show infos, the single episode duration is reported correctly). I already run "deep scan" and "analyze".
     

    At last, this is an issue I recall with the old scrapers as well: the shows with a semicolon in the title (Chaos;Head, Steins;Gate, Myself;Yourself) are not parsed. Nor automatically nor manually.

    Some problems which I don't think concern HAMA:
    - Some shows (mostly the ones with the year in the title) are not included in the library at all. BABS scanner issue, maybe.
    - The show (this could be a missing line in the xml?) "Strange" has no match and no semicolon to blame, "Toaru Hikuushi e no Koiuta" has no tvdb match (I assume this since the links in the description are only to Anidb and ANN and the cover is missing) like "Onee-chan ga kita", "Jigoku Sensei Nube" and "Corpse Party Tortured Souls".

    Thanks again :)

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited March 2014
    If the scanner let it through, you will see it as a folder title can be found with manual search and one of anidb's titles or  aid:xxxxx with x being the anidb id
    If the folder isn't showing for "Steins;Gate" for exemple, then the scanner needs amending.
     
    Please view my wonderfull html report logs:
       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/anime-list.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/themes.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/TVDB.htm

       . /volume1/Plex/Library/Application Support/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/AniDB.htm

     
    If you list all that is wrong including serie folder name used and 1 episode name exemple and the symptoms, i might be able to fix the scanner
     
    Also, please give the anidb id or link like http://anidb.net/perl-bin/animedb.pl?show=anime&aid=10228
    Is Strange in effect "Strange+" ?
    If a serie isn't in the xmls, then it will show in anime-list.html log...
    You have the links in the description ? That is an old version...
    I left it but disabled by default in the latest version... might solve the time issue..

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • stbigelow12stbigelow12 Posts: 4Members

    How do I get this to work on a Mac?  I cannot find my plex media server plugins folder.

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • stbigelow12stbigelow12 Posts: 4Members
    ZeroQI wrote on March 28 2014, 12:52 PM: »

    Yeah I get it.  I searched wrong on Google.  I didn't just ask for funsies, I asked because I couldn't find what I was looking for when I searched.  I should learn2google better.

  • XellossMetalliumXellossMetallium Posts: 9Members ✭✭
    edited March 2014
    ZeroQI wrote on March 27 2014, 11:06 PM: »
    If the scanner let it through, you will see it as a folder title can be found with manual search and one of anidb's titles or  aid:xxxxx with x being the anidb id
    If the folder isn't showing for "Steins;Gate" for exemple, then the scanner needs amending.
     
    If you list all that is wrong including serie folder name used and 1 episode name exemple and the symptoms, i might be able to fix the scanner
     
    Also, please give the anidb id or link like http://anidb.net/perl-bin/animedb.pl?show=anime&aid=10228
    Is Strange in effect "Strange+" ?
    If a serie isn't in the xmls, then it will show in anime-list.html log...
    You have the links in the description ? That is an old version...
    I left it but disabled by default in the latest version... might solve the time issue..

    https://dl.dropboxusercontent.com/u/37448634/anime-list.htm

    https://dl.dropboxusercontent.com/u/37448634/TVDB.htm

    There are only these two, but I have to admit, they are wonderful! 

    So, As I was adding the subfolders one by one I checked which anime were not scanned (not on plex at all). They are all renamed by anidb, obviously. Here the list of the folder names:
    KissXSis (2010), 
    Black Rock Shooter (2012), Fate Zero (2012), K-On!, Seitokai Yakuindomo (2011), Tenjou Tenge (2005), Terra e..

    These too, but maybe because they are movies: Gekijouban Shakugan no Shana, Kowarekake no Orgel, Sora no Otoshimono Project Pink

    About chaos;head (and the others with semicolon).. it is on plex, just not parsed with anything. I try "correct wrong ID" (don't know how it is named in english) and in the "title" field I put: a6088 (this should be the id, by google search), 6088, Chaos, Head, Chaos Head.  No match at all. But I'm still banned, so maybe that is the reason (even if the ban was there when the biggest part of my library was compiled..).

    About the "Episode Duration: NaN min".. I don't know, this is similar on all shows.

    EDIT: oh yes, Strange is Strange+. The folder had the correct name but on plex only Strange were shown... corrected the ID, all fine.

    Incidentally.. do you think that automatic library scans with HAMA may be increasing my ban time? :o

  • ZeroQIZeroQI Posts: 1,225Members ✭✭✭
    edited April 2014

    Sorry for the delay.

    Correct syntax is "aid:6088" to specifically assign the anidb id

     

    For the duration fixed it, needed: season == "1" instead of season == 1 in the code...

    Movies are to be treated as ep 01

     

    Have been working thanks to AtomicStrawberry back on the agent for the following working functionalities:

       . automatic xml file download using 2 week cache (use aid:xxxxx with aid being the anidb id in the url if serie not picked up)

       . Correct TVdb season posters downloaded for poster and season

       . AniDB poster if selected present at the end

       . Moved XML to Data folders for troubleshooting, gets pulled from the cache, but you can cross reference logs that way

       . local cache for mp3 themes using AniDB id.mp3 convention. If replaced with full theme, would pick that one first

       . Anime average duration fixed, thanks for reporting (comparison with 1 instead of "1" was the cause)

    Ban time has to do with two many scans per period of time. how many scans per period of time is to be determined

    Downloading the title database is allowed once a day and it will be cached for two weeks, so it should not impact ban time,

    Will release a proper package that updates without interaction the XML files and cache them soon after correcting all known issues.

    Added code on Github, untested. Will test tonight

    Titles to test:

    ==========

    KissXSis (2010)

    Black Rock Shooter (2012)

    Fate Zero (2012)

    Seitokai Yakuindomo (2011)

    Tenjou Tenge (2005)

    K-On!

    K-On!!

    Love Live 2

    Terra e..
    Seitokai Yakuindomo'
    Semi-colon: Chaos;Head, Steins;Gate, Myself;Yourself
    Movies: Gekijouban Shakugan no Shana, Kowarekake no Orgel, Sora no Otoshimono Project Pink

    Absolute Series Scanner (ASS): Seasons+Absolute mode series scanner managing subdirectories and AniDB type specials,with a logging off all files (skipped on top of root folder, then sorted one line per file with regex used and info, all series sorted alphabetically) and one with the filelist to reproduce the library on my end with dummy files. It also add all files and recognise specials pretty accurately.

    http AniDB Metadata Agent (HAMA): Some good info there, will have to re-write it at some point... Data folders are kinda mandatory... Huge libraries can fetch 10gb in space. If you have no poster but other metadata is showing, create "Plug-in Support/Data/com.plexapp.agents.hama/DataItems" agent folders as indicated in the readme on GitHub

    Impressed by the agent recognition rate? Consider donating few euros through Paypal: Absolute Series Scanner + HTTP AniDB Metadata Agent. (Donators: xnaas, Sven-7, Niegen, Vorador, Fratal, T. Mayville,  A. Estremera, KJKingJ. Thanks guys)

  • floatymanfloatyman Posts: 9Members, Plex Pass Plex Pass

    Another you might want to add to your test folder are shows that end in a number or odd punctuation. There are not a lot of shows that end in numbers (second seasons tend to have their own name), but there is an annoying trend to add odd punctuation at the end for a new season.

    The ones in my test folder that are not be picked up correctly are:

    Love Live 2

    K-On!!

    Seitokai Yakuindomo* (I renamed mine to Seitokai Yakuindomo' as * can't be in a name convention, so any solution is almost impossible)

    Hopefully the latest update will automatically fix this but I won't be able to test it myself for a little while.

    Once again thanks for the update.

Sign In or Register to comment.