[Rel] HTTP Anidb Metadata Agent (HAMA)

Also, not sure if you are still updating character maps:

*Character missing in CHARACTERS_MAP: 52352:‘̀’ , #‘̀’ [’\xcc’, ‘\x80’], string: ‘Zegapain E05 ‘Déjà Vu’ mkv’

CHARACTERS_MAP: 52352 added accent will be removed

“Root Folder (show name will be taken from filename)”
“Evangelion Shin Gekijouban Ha” s0001e001 “Evangelion Shin Gekijouban; Ha - Complete Movie [1920x1080] [WiKi].mkv”
“Evangelion Shin Gekijouban Jo” s0001e001 “Evangelion Shin Gekijouban; Jo - Complete Movie [1920x1080] [THORA].mkv”
“Evangelion Shin Gekijouban Q” s0001e001 “Evangelion Shin Gekijouban; Q - Complete Movie [1920x814] [UTW-THORA].mkv”
“Gekijouban Clannad” s0001e001 “Gekijouban Clannad - Complete Movie [720x400] [EnA].avi”
“Gekijouban Gintama Kanketsuhen Yorozuya yo Eien Nare” s0001e001 “Gekijouban Gintama Kanketsuhen; Yorozuya yo Eien Nare - Complete Movie [1920x1080] [UTW].mkv”
“Gekijouban Hayate no Gotoku! Heaven is a Place on Earth” s0001e001 “Gekijouban Hayate no Gotoku! Heaven is a Place on Earth - Complete Movie [848x480] [Commie].mkv”
“Gekijouban K Missing Kings” s0001e001 “Gekijouban K; Missing Kings - Complete Movie [1920x1080] [JacobSwaggedUp].mkv”
“Gekijouban Macross F Itsuwari no Utahime” s0001e001 “Gekijouban Macross F; Itsuwari no Utahime - Complete Movie [1920x1080] [THORA].mkv”
“Gekijouban Macross F Sayonara no Tsubasa” s0001e001 “Gekijouban Macross F; Sayonara no Tsubasa - Complete Movie [1920x1080] [Coalgirls].mkv”

“Cowboy bebop”
“Cowboy bebop” s0001e005 “Cowboy.Bebop.S01E05.1080p.BluRay.x264-RedBlade.mkv” “1”
“Cowboy bebop” s0001e015 “Cowboy.Bebop.S01E15.1080p.BluRay.x264-RedBlade.mkv” “1”
“Cowboy bebop” s0001e025 “Cowboy.Bebop.S01E25.1080p.BluRay.x264-RedBlade.mkv” “1”

“Maria-sama ga Miteru” from foldername: “Maria-sama ga Miteru/Season 1”, Season: “1”
“Maria-sama ga Miteru” s0000e501 “[GrimRipper]Maria_Watches_Over_Us_Omake01(C7D43BD4).mkv” “Watches Over Us Omake01”

“Maria-sama ga Miteru” from foldername: “Maria-sama ga Miteru/Season 2”, Season: “2”
“Maria-sama ga Miteru” s0000e002 “[GrimRipper]Maria_Watches_Over_Us_S2_Printemps_Omake01(B7A4D05A).mkv” “Printemps Omake01”

“Maria-sama ga Miteru” from foldername: “Maria-sama ga Miteru/Season 4”, Season: “4”
“Maria-sama ga Miteru” s0000e001 “[Toe-ko]_Maria-sama_ga_Miteru-4th-Omake_01[38d1e057].mkv” “10” “01”

omake stuck to the omake number: will not correct (Omake01)

“Zoids Genesis”
“Zoids Genesis” s0001e031 “[T-N]Zoids_Genesis-31[9DCCACD3]CM.avi” “10” “CM”
“Zoids Genesis” s0001e047 “[T-N]Zoids_Genesis-47[C3E5C322]andCM.avi” “10” “andCM”

Uploaded new version to correct all to github

@Worldz: all good on latest version? it should have corrected all issues reported

I added a new functionality to the series scanner: DVD folder support.

Now some DVD works (*.VOB files shows with mpc hd as 20 min long) and some don’t (show as 20s long)

Will also be on hollidays from tuesday to saturday. Please let me know if the DVD support works as expected knowing it can’t map vob files to episodes so it is like multi episodes (each ep get the whole shebang). if i could set aqn offset would be cool but so far no. Let me know the proportion of DVD folders working and the protection if possible if not working.

Please try to keep scanner issues on scanner thread and agent issues on agent thread and in the eventuality it does work and you had a bunch of DVD folders, let me know on the scanner thread and there is a donation link in my signature in case this crap actually works :slight_smile:

Hey, I have probably done something wrong here but I’ll try explaining this;
I found your agent and scanner because theTVDB one mismatches all over the place, I have found that yours matches everything fine as long as I name everything right however it can’t seem to grab any posters, banners, or backgrounds for anything even with all sources for such things selected. If I i use theTVDB it can find images but mismatches far too often. This is happening on every thing regardless of how old or new it is.

I am running Plex off a freeNAS box and I have made sure the permissions on everything in “var/db/plexdata/Plex Media Server/” are correct. I am using ‘Plex 0.9.12.8.1362’ on ‘FreeNAS-9.3-STABLE-201505130355’ and the latest versions from the github master zip files. I don’t think the client os should matter but I am running 8.1 Pro up to date.

Let me know if you need anything else.

You can find the readme:
. when clicking the agent link in my signature
. on github page
. in the zip release on this thread

It says in the readme:
. under installation to pick the zip file in this thread then update the agent and scanner py code… Since you didn’t download there i will guess you didn’t follow the readme
. listed diag files under troubleshooting…

Also in the last few pages of the thread people had the same issue and one had an ad-aware blocking the urls with “banners” in them all the other didn’t create the data folders… Please confirm you created the agent data folders listed in the readme and also contained in the last few zip file releases on that thread (would be faster that way)…

While the readme says it is optional, actually it crash without even if i put the code with exceptions due to faults in the framework itself (i put except around code but still crash, cannot prevent it without removing caching which i will not do). I don’t update the readme because it’s a beta code in a developper forum, and honestly nobody reads it…

If it doesn’t work include hama agent logs.
If i am right please consider a donation of the cost of a pint of beer in your local country, especially since a few people got this issue and i documented the same things previously in this very thread.

Hmmm… that is odd, I had grabbed the latest zip here the first time I set it up but something must have gone wrong with it when I updated plex. I guess from now on I should just re-do everything when I update.

Thanks for the help, seeing it all update properly was beautiful!

The agent crash if it can’t write, so could have been the episode screenshot folder added recently that was missing… The answer is always in the agent log… Glad it works though…

Did quite some reflexion to avoid duplicate posters and have the most accurate matching. I think it has the best recognition rate and since you can have series and movies in same library and even force match with title or anidb.id file or even show an absolute serie with seasons by using tvdb.id file, thinks it is the best agent… You can event group series and movies in a grouping folder and it still works. Donation always welcome…

Thanks to Niegen, Vorador, Fratal for donating, updated by signature to reflect that.
Please let me know if you have any feature request, cannot think of any right now…

I have re-written the readme on github, should avoid most questions and issues when installing now.
Please let me know if anything is missing from there

Did anybody tried the DVD folder support of the latest scanner btw? Naming convention is "Serie name/s01e01-05/VIDEO_TS.IFO (you can optionally have folder “VIDEO_TS”, ep numbers follow same naming conventions as episodes so absolute numbers are ok too). It will gather metadata and screenshots from the vob files, BUT some vob files showing 20s duration under windows will not work, those showing 20 min for 2gb files will be ok.

Currently i’m fine. Movies work, and series too. The agent gets confused a bit with series on root with a number on the name but those series can be counted with the fingers of one hand. I stuck those in folders and worked like a charm.

Thanks for the feedback, i can’t really fix it though with major changes as you probably guessed.
Ii use the folder name as guide and i check if the number in an episode title is present in the other files of the folder, but i can’t do that for root folder due to the diverse files present…

However i do think that the scanner recognise nearly anything properly and the agent should have a good recognition rate, and when it can’t generally there’s a good reason :smiley:

I have created full archives that together with the read-me file amended should avoid most install issues
I have included my test library, includes all files that gave issues at some point, good to compare recognition rate with other agents if they can get close, that is…

Just a question regarding the naming of seasons.
For example I have Chuunibyou Demo Koi ga Shitai! and it’s second season Ren, the first season is obvious to name but with the second season what should it be?;
eg.
Chuunibyou demo Koi ga Shitai! - s02e01 - Ren
Chuunibyou demo Koi ga Shitai! Ren - s02e01
Chuunibyou demo Koi ga Shitai! Ren - s01e01

at the moment I have it as the first option and it is splitting it into a separate entry with only a season 2 inside.
the first season is name in romaji and the second uses the english title.
the folders are setup like this.

Chuunibyou demo Koi ga Shitai!
\Chuunibyou Demo Koi ga Shitai!
\Chuunibyou Demo Koi ga Shitai! Ren

it seems like it should work but I think the ren is screwing this. this is happening for other series that have similar naming, like JoJo’s.

The closest folder is used as serie name
The season folders are removed as well as as ark xxx and video folders
The numbering in filename overrides default season 1…

If you want season numbering:
Chuunibyou demo Koi ga Shitai!
\Season 1 Chuunibyou Demo Koi ga Shitai!
\Season 2 Chuunibyou Demo Koi ga Shitai! Ren

If you want absolute numbering remove s02 from filenames in season 2 and it will be two entities in Plex and the second serie will be without season 2

I am trying to use this but it is just not working at all. I have 175 episodes of Fair Tail and it bundled them all under 1 episode with 175 parts… please help :frowning:

Scanner issue. As per the readme in the zip and github hama page attach scanner log and filelist to scanner thread

Since you asked on the HAMA thread, i just had an idea and a feature request.

I’ve noticed that the scanner is much faster if the filelist is smaller. Less than one thousand is done in ten minutes. More than five thousand is equal to hours. Might be because of files on root instead of folders, i don’t know if anybody has the same problem.

In any case, my request would be a change to work around it. I think a way would be to use the existing filelist to compare the library for changes and only scan and send those changes to the agent. If doable, it should be much faster, and it would avoid having to rework the scanner.

Currently all files are added by the scanner during the scanner call for root folder. Other folders’ scanner calls are ignored as already processed at root level… i cannot check what is there as Plex does that…

Using root call is the only way i found to have multpliple series in a single parent folder at root level, which only my scanner allows that (like all dragon ball series together in a ‘dragon ball’ folder)

I can change it to only use root scan for several series in a singe subfolder and use proper folder call for the rest but that will be tricky:
. Build tree for grouping folders if on root folder scanner call.
. Skip folder if multiple folders not seasons and not root call as already managed on root call. Will have to check folder deph without season folder probably

That would be faster and less possibility of corruption and would start metadata searching earlier while still scanning…

Will implement but need to start thinking about how to code it…

Hi ZeroQI. I have just started using your scanner and came across an issue with downloading banners and posters. The scanner does a GREAT job of grabbing the rest of the metadata and I use it in conjunction with filebot.

When I read through the com.plexapp.agents.hama.log I notice that it is having an issue with writing. I have attached it to the post.

Here is an example of the issue:

2015-09-07 12:35:09,825 (3134) : DEBUG (logkit:13) - metadata_download() - metadata_download - could not write filename ‘TVDB/fanart/original/145211-3.jpg’ in Plugin Data Folder
2015-09-07 12:35:09,845 (3134) : DEBUG (networking:166) - Requesting ‘http://thetvdb.com/banners/_cache/fanart/original/145211-8.jpg
2015-09-07 12:35:09,917 (3134) : DEBUG (networking:198) - Not caching ‘http://thetvdb.com/banners/_cache/fanart/original/145211-8.jpg’ (content type ‘image/jpeg’ not cacheable in Agent plug-ins)
2015-09-07 12:35:09,917 (3134) : CRITICAL (core:572) - Exception writing to C:\Users\Anthony\AppData\Local\Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama\DataItems\TVDB\fanart\original\145211-8.jpg (most recent call last):
File “bundles-release/Framework.bundle-dist/Contents/Resources/Versions/2/Python/Framework/components/storage.py”, line 81, in save
File “bundles-release/Framework.bundle-dist/Contents/Resources/Versions/2/Python/subsystem/ospathfix.py”, line 64, in builtins_open
IOError: [Errno 2] No such file or directory: u’\\?\C:\Users\Anthony\AppData\Local\Plex Media Server\Plug-in Support\Data\com.plexapp.agents.hama\DataItems\TVDB\fanart\original\._145211-8.jpg’

You need to create the folder structure mentioned in the first post.

I don’t know what to do to avoid this type of questions as this is the most encountered one, i wrote a readme which covers installation and upgrade and and i even indicated in my signature “If you have no poster but other metadata is showing, create “Plug-in Support/Data/com.plexapp.agents.hama/DataItems” agent folders as indicated in the readme on GitHub”. it is also in the readme in on the last zip archive. if you didn’t follow the install guide there, please do noe and maybe consider a donation to restore my faith in humanity :smiley:

On a side note, i am coding a major overall of the scanner to avoid corruption and slowness by using the plex normal scanning bahavior, while juggling to keep grouping parent folder function still working.

AtomicStrawberry suggested afunctionality: tying custom matches to folder name in the agent, will implement that next so you don’t have to use ids in file or folder name, it will remember custom matching…
Will do that after the scanner overhall