Great I found what caused HAMA to crash on Shinmai Maou no Testament it appears that it didn’t like that the file came with a version number so changing the file name from the first to the second allowed HAMA to process it without crashing.
Shinmai Maou no Testament Burst (2016) - 1v5 [Harunatsu].mkv Shinmai Maou no Testament Burst (2016) - 1 v5 [Harunatsu].mkv
Looking in the filelist log from you scanner I can see these entries:
"Shinmai Maou no Testament Burst (2016)" s2016e001 "Shinmai Maou no Testament Burst (2016) - 1v5 [Harunatsu].mkv" "4" ") - 1V5" "Shinmai Maou no Testament Burst (2016)" s0001e001 "Shinmai Maou no Testament Burst (2016) - 1 v5 [Harunatsu].mkv" "V5" "Shinmai Maou no Testament (2015)" s0001e001 "Shinmai Maou no Testament (2015) - 1v2 [AHorg].mkv"
So for some reason the scanner messes up the season indentification from file name and with Shinmai Maou no Testament Burst (2016) it appears that this error then caused HAMA to crash, or this is my best guess.
Now if I were to guess why this happens without checking your code to verify if it’s possible. My best guess is that the messed up season indentification causes HAMA to enter TVDB numbering mode (seasons), but Shinmai Maou no Testament Burst (2016) has no TVDB ID linked to the AnidbID so entering this mode and processing it as if it has one causes a crash when it attempts to run the TVDB section with the ID “” empty string.
The scanner mess up numbering, due to v5 but not a reason to crash. corrected to include v5 in whack words array to remove, please test new version
Now the error:
INFO (init/anidbTvdbMapping:698) - anidb: ‘11890’, tvbdid: ‘’, tmdbid: ‘’, imbdid: ‘’, studio: ‘’, defaulttvdbseason: ‘’, name: ‘Shinmai Maou no Testament Burst (2016)’
2016-11-10 13:29:26,339 - com.plexapp.agents.hama (80879ac00) : CRITICAL (agentkit/_update:1067) - Exception in the update function of agent named ‘HamaTV’, called with guid ‘com.plexapp.agents.hama://anidb-11890?lang=en’ (most recent call last):
File “/usr/local/share/plexmediaserver/Resources/Plug-ins-24ffd60/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py”, line 1065, in _update
agent.update(obj, media, lang, **kwargs)
File “/usr/local/plexdata/Plex Media Server/Plug-ins/Hama.bundle/Contents/Code/init.py”, line 936, in update
def update(self, metadata, media, lang, force ): self.Update(metadata, media, lang, force, False )
File “/usr/local/plexdata/Plex Media Server/Plug-ins/Hama.bundle/Contents/Code/init.py”, line 672, in Update
elif not movie and (len(media.seasons)>2 or max(map(int, media.seasons.keys()))>1): keys = [“tvdbid: %s” % (WEB_LINK % (TVDB_SERIE_URL % tvdbid, tvdbid) )]
UnboundLocalError: local variable ‘tvdbGenre’ referenced before assignment
no tvdbid seem to make it crash since it only loads tvdb variable (like tvdbGenre) then and implie there is a tvdbid since
I made a fix to initialise all tvdb variable even if no tvdbid. Please test new agent
The new Hama appears to have fixed the crash, from reading the logs it appears to run the same code for the series and no crashes in the log.
As for the scanner you forgot a comma, but that only helped in the case of Shinmai Maou no Testament Burst (2016) but did nothing for
[“To Heart 2 (2007) [anidb-4806]” s2007e001 “To Heart 2 (2007) - 2 [Avalon&AQS].avi” “4” “) - 2”]
I imagine that you already know this but “Shinmai Maou no Testament Burst (2016) - 1v5 [Harunatsu].mkv” still matches wrong, but “Shinmai Maou no Testament Burst (2016) - 1 v5 [Harunatsu]” is now identified correctly (assuming you fix the missing comma). A space is required between the episode number and the version number or any other data (Already changed my file renaming rules to reflect this).
To this I’ll add that “Shinmai Maou no Testament Burst (2016) - 1 v5 [Harunatsu]” was given the correct season and episode with the old version of the scanner it just used “v5” as a title for the episode.
@TheLion My German is quite rusty, but the HAMA metadata agent doesn’t allow manual data entry and it’s not meant to. You already have manual data entry through the web interface, by clicking on the pencil icon and if you have data files in your library from Kodi style setups, then there are other agents that can import such data.
I can see this log in the custom scanner log “WARNING - Grouping folder: ‘To Heart/To Heart’ skipped, need to be added as root folder if needed”. Did you add “/mnt/media/anime/To Heart” as root directory in a library? if not, the files should not be showing…
Filename: [“To Heart 2 (2007) [anidb-4806]” s2007e001 “To Heart 2 (2007) - 2 [Avalon&AQS].avi” “4” “) - 2”]
It didn’t match ‘^(?P.?) - (?P[0-9]{1,3}) - (?P.)$’ so changing to ‘^(?P.?) - (?P[0-9]{1,3})( - )?(?P.)$’ so this would support title - xxx format
It used the old american cartoon regex to match it: ‘(?(?P(19|20)[0-9]{2})[)]?[ _.-]+(?P.*?)$’ instead
The scanner should have taken the 2 as ep number unless too many times showing in logs, but it’s mean to substract the serie name from the ep name, maybe not scrubbing the brackets…
We need to remove the serie name first. This is already handled by:
for prefix in (folder_show, clean_string(folder_show, True), clean_string(folder_show, no_dash=True), clean_string(folder_show, True, no_dash=True)): # remove cleansed folder name from cleansed filename and remove potential space
if ep.lower().startswith(prefix.lower()): ep, folder_use = ep[len(prefix):].lstrip('- '), True; break
Please attach the filelist and scanner output for that folder and i will recreate and amend code to fix, which should prevent similar behaviour for other series
@Kyomujin : perfect logs. scanner issue.
issue when a number is in the title and an id forced causing the misc variable i search in for duplicate ep number logic not to be scrubbed correctly. updates source, now show:
“To Heart 2 (2007) [anidb-4806]” s0001e002 “To Heart 2 (2007) - 2 [Avalon&AQS].avi”
Alright first off, your updated code appears work correctly for me. But I’ve carried on adding more stuff and found that HAMA fails to get metadata for another entry and it’s a different problem.
The entry has this excerpt from the anime-list-master.xml <anime anidbid="9259" tvdbid="OVA" defaulttvdbseason="1" episodeoffset="" tmdbid="162975" imdbid=""> <name>Arata-naru Sekai: World's/Start/Load/End</name> </anime>
From reading the HAMA log I see the following: 2016-12-01 16:21:58,748 - com.plexapp.agents.hama (808593800) : DEBUG (networking/load:166) - Requesting 'https://api.tmdb.org/3/tv/162975/images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' 2016-12-01 16:21:58,917 - com.plexapp.agents.hama (808593800) : ERROR (networking/load:219) - Error opening URL 'https://api.tmdb.org/3/tv/162975/images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd'
Now this is wrong as tmdb:162975 only exists for movie but hama is trying to search for a tv series. From a cursory glance in the code, I’m not sure when the HAMA agent will do a tmdb movies search, but maybe you’ll want to catch the error of a tv search and attempt movie search when a tmdb tv search fails.
Or is this just not an intended use case and I need to use guid string or guid.id files to link it correctly for tmdb backdrop and poster data?
Works fine if I add TMDB_MOVIE_IMAGES_URL = 'https://api.tmdb.org/3/movie/%s/images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' #
to the global definitions at the top of the file, as TMDB_MOVIE_IMAGES_URL was undefined.
@Kyomujin : thanks for precising, did a bad manipulation and ended up wiping that line. committed changed to the master. That way, the moviedb id is used and posters used if present in ScudLee’s mapping file, therefore thanks for that bug report.
With the other examples in the mapping they all have a tvdbid, so you’ll still get some artwork for them but not for mardock scramble. Here’s the update excerpt from the hama log 2016-12-10 19:30:24,788 - com.plexapp.agents.hama (80864e400) : INFO (__init__/Update:589) - --- Update Begin ------------------------------------------------------------------------------------------- 2016-12-10 19:30:24,788 - com.plexapp.agents.hama (80864e400) : INFO (__init__/Update:674) - metadata source: 'anidb', id: '7439', Title: 'None', lang: 'en', (True) 2016-12-10 19:30:24,819 - com.plexapp.agents.hama (80864e400) : INFO (__init__/anidbTvdbMapping:700) - anidb: '7439', tvbdid: 'movie', tmdbid: '', imbdid: 'tt1754177,tt2231375,unknown', studio: '', defaulttvdbseason: '1', name: 'Mardock Scramble' 2016-12-10 19:30:24,826 - com.plexapp.agents.hama (80864e400) : DEBUG (networking/load:161) - Fetching 'http://api.tmdb.org/3/configuration?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' from the HTTP cache 2016-12-10 19:30:24,830 - com.plexapp.agents.hama (80864e400) : INFO (__init__/getImagesFromTMDB:796) - using TMDBID url: 'https://api.tmdb.org/3/movie//images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' 2016-12-10 19:30:24,833 - com.plexapp.agents.hama (80864e400) : DEBUG (networking/load:166) - Requesting 'https://api.tmdb.org/3/movie//images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' 2016-12-10 19:30:24,945 - com.plexapp.agents.hama (80864e400) : ERROR (networking/load:219) - Error opening URL 'https://api.tmdb.org/3/movie//images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd' 2016-12-10 19:30:24,945 - com.plexapp.agents.hama (80864e400) : ERROR (__init__/get_json:875) - Error fetching JSON url: 'https://api.tmdb.org/3/movie//images?api_key=7f4a0bd0bd3315bb832e17feda70b5cd', Exception: 'HTTP Error 404: Not Found'
Might be enough to get the first entry by id.split(",")[0] on both IMDBIDs and TMDBIDs (found one mapping with multiple tmdbids see below). <anime anidbid="10977" tvdbid="275609" defaulttvdbseason="0" episodeoffset="" tmdbid="334361,357413" imdbid=""> <name>Wake Up, Girls! Zoku Gekijouban</name> <mapping-list> <mapping anidbseason="1" tvdbseason="0">;1-3;2-4;</mapping> </mapping-list> </anime>
I’ve been having problems with matching anime series when using ASS + HAMA using series folder names with years. Around 50% of my anime does not get matched, and others get matched incorrectly.
It is formatted as such:
RootLibraryFolder\SeriesName (year)\SeriesName - EXX (episodedate) [audio_and_video_info][subbergroupname]
Shouldn’t the scanner or metadata agent separate out the year and place it into the “Year”-field when trying to match series? The year seem to stay in the “Title”-field. If I manually move the year over to the “Year”-field I will get a 99% match most of the time. Since the year stays in the “Title”-field it gets confused and show other series which has the year in the title.
Examples of what happens normally:
And examples of what happens when I manually alter the “Year”-field:
Is this a bug or how it should work? I know there are series out there with the year in the title, but they would be correctly picked up if the year gets put into the “Year”-field during matching anyway, right?
@Kyomujin Amazing precision there. Didn’t know this could be possible. will add support asap @rogergh thanks for the well documented post. This agent is called HTTP AniDB Metadata Agent for a reason. It takes AniDB titles for search first and TVDB second for titles if no good AniDB match is found.
There is two Air series in AniDB that came in 2005
Gekijouban Air
Air 剧场版
Theatrical Feature AIR
Air: The Movie
Air (Movie)
AIR - Le film
Высь - Фильм
Въздух Филмът
air movie
劇場版 AIR
Air: The Motion Picture
While for movies the year in the title is plex practice, for series, i wouldn’t recommend it.
Currently work as designed but since you don’t use anidb titles, you have poorer recognition rate…
While i could do what you advise, it would break it for people using anidb compatible titles that include years in the official title since some do so i can’t really have that instead…
I can see if a year is present, add both results, but ultimately you pick an AniDB agent and don’t match your title to match
Current to-do list:
search with year (low priority)
double imdbid/tmdbid: if “,” in imdbid: imdbid= imdbid.split(",")[0] and same for tmdbid
@ZeroQI If you want a lazy solution that only gets metadata from one of the IDs, you can: id = id.split(",")[0]
Or maybe go recursive with something like: if len(id.split(",") > 1: for ident in id.split(","): self.getImagesFromTMDB(metadata, ident, num)
@ZeroQI
Thanks for the response. I can understand why it is like that, but it makes the folder-structure so much more difficult to reuse for stuff other than Plex (ex. if you use other media software that doesn’t match against databases it is good to have a year to get a quick overview).
All the titles themselves are fetched from AniDB using Filebot, but I added the creation year to have it somewhat organized for other uses than Plex. This worked perfectly for normal TV-series (TVDB titles) and default Plex agents, so I had hoped it would work for anime through this agent as well. Plex actually recommend adding the year so that it can differentiate series with same names, so I thought; Why not add it to all my series so it would be futureproofed in case a newer serie with same name would be made. But I guess that only applies when using TVDB?
Anyway, thanks for considering to add some kind of year filter\matching in the future.
@rogergh Data from AniDB is always special compared to other databases. AniDB has customized their entire structure around the reality of how Anime is, which have had some consequences that make strucutres suitable for western media databases unsuitable for Anime the data AniDB carries.
Some examples are how most western media and databases have the concept of seasons which doesn’t exist within anime e.g. One Piece is a single series/season with 768 episodes from 1999 to current day. The taggins system of AniDB is also a system that to my knowledge is unique and in my opinion, with its tree structure and tag merging is superior to the common category systems.
As for titles, AniDB has no duplicate “Main Titles” and in addition to these main titles they record all official titles and some fan translated titles, where appropriate. For an example you can see these 2 titles. Kimi to Boku Kimi to Boku.
To this I’ll add that AniDB is mainly meant to use hash-sums (ed2k) of files to identify their series and episode, which makes names irrelevant and usually set by whatever client you use to hash.
All these things make AniDB a wonderful database, but can also make it difficult to use AniDB style data in other systems. So at some point something has to be compromised when integrating AniDB data with other systems.
Though for compatibility with plex and ASS, you can have your renamer append the AnimeID to the end “%ann (%yea) [anidb-%aid]” -> “Kimi to Boku. (2011) [anidb-8367]” or whatever renaming markup it uses.
