[Rel] HTTP Anidb Metadata Agent (HAMA)

Dev Community Dev/API Corner
681

That’s fair tips. I read through it and added the folders. I did get a few posters so thanks. Now I did notice that when I check the ones I don’t have posters and such for that it says found no matching anidbid. Any recommendations?

682

that is odd… aid should be 191 while the metadata id should be anidb-191. That is a wrong agent code…

Latest agent code: https://github.com/ZeroQI/Hama.bundle/blob/master/Hama.bundle/Contents/Code/init.py
Line 195: Log.Debug("Update - metadata source: '%s', id: '%s', Title: '%s',(%s, %s, %s)" % (metadata.id.split('-')[0], metadata.id.split('-')[1], metadata.title, "[...]", "[...]", force) )

If i look at the log you sent the update says update2 and it’s not the same code…
update2 - AniDB ID: 'anidb-191', Title: 'Great Teacher Onizuka',([...], [...], True)

So the readme says “Get the latest zip package in (https://forums.plex.tv/discussion/77636/release-http-anidb-metadata-agent-hama#latest).” which is attache to my post just bove : _ASS + HAMA 2015-08-31.7z

I checked and it says update not update 2 there and same for the latest code on github…
I am sorry, but i really cannot make the readme clearer…

May i ask where the link and/or filename of the agent you have installed as i need to delete any old package to avoid this happening again. Use the package above and update with code on github and all will update nicely and will have all the posters. if upgrading from an old version, you might have to re-create the library since the metadata id format changed recently

On a unrelated note, it look like the scanner redesign necessary as had corruption which needed to clean bundles, empty trash, optimize before it saw all series… New scanner is 75% done, functionality is there, but few cosmetic things and folders exclusion not working. the annoying functionality are the grouping folders, that makes it more complex…

683

I am having problems for the first time with this scanner/agent, its been working well for months now (yesterday too), all of a sudden today the metadata will not seemingly bind to the series. All the eps are there, and if the DataItems/TVDB folder has metadata for the two series, but all I get is the contents of the screenshot. Two new items have this problem, I will also attach the log for a scan which tries to grab metadata for both of them.

I have tried cleaning bundles, trash then optimise four times, didn’t help. Unmatching then matching didn’t help, clicking refresh didn’t either. Refresh normally does the trick. Also made sure I used the zip from the last page to add any folders needed in DataItems.

684

Try to open the anidb xml link in a web browser.
Look like you are banned or a software prevent downloading the xml…
Search manualy for a serie labeled clear-cache but no need to select it.

685

Are you talking about the api links in the log? They don’t load, I get nothing. (Can’t reach this page) Could that be part of the problem? Also, not sure what you mean by search manually. Is this in anidb.net? The API?

686

Yes if the file is blank you have a firewall/antivirus/adware/spyware issue. If banned it would tell you in the file. Now that page would also be cached in plex so find why the api.anidb.net link in logs show blank, then pick any serie and custom maych as if it was the wrong one detecred and search for clear-cache. You do not have to select it just custom match search it…

687

I unmatched it, clicked Search Options, then searched clear-cache. All that came up is some Computer anime. It didn’t help unfortunately. I also run bare Windows, no Norton programs, just Windows Firewall/Defender. I disabled them both, didn’t help either. I can use this link http://anidb.net/api/anime-titles.xml.gz and get the full dump, anidb.net works fine, its just the API that comes up blank (like http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10938). Not sure why it broke today, was working perfectly before hand.

Edit: tested it on my phone on a completely different network, link still doesn’t work.

688

Hi, tried loading, it also times out and doesn’t work so shouldn’t be cached since no file is received
Anidb is down then. can’t do anything about that.

An an unrelated note, the scanner corruption also caused by zero size files…

689

@ZeroQI said:
I don’t know what to do to avoid this type of questions as this is the most encountered one, i wrote a readme which covers installation and upgrade and and i even indicated in my signature “If you have no poster but other metadata is showing, create “Plug-in Support/Data/com.plexapp.agents.hama/DataItems” agent folders as indicated in the readme on GitHub”. it is also in the readme in on the last zip archive. if you didn’t follow the install guide there, please do noe and maybe consider a donation to restore my faith in humanity :smiley:

On a side note, i am coding a major overall of the scanner to avoid corruption and slowness by using the plex normal scanning bahavior, while juggling to keep grouping parent folder function still working.

AtomicStrawberry suggested afunctionality: tying custom matches to folder name in the agent, will implement that next so you don’t have to use ids in file or folder name, it will remember custom matching…
Will do that after the scanner overhall

That reminded me that I still hadn’t donated like I’d been meaning to. Thanks ZeroQI!

690

Thanks for the donations, that keeps me motivated (I tried to use the Login name to name all that donated in my signature, but sometimes i cannot find the login name from email or name).

New scanner seem to work and i hope to release it in the next 2 weeks but still in testing phase as it doesn’t fully behave like the current one.
code will be shorter (388 lines about), memory usage better since no longer building a dictionary of folders and start metadata sooner… since it will have different scanner calls and metadata can start after a scanner call instead of doing everything in the first scanner call which prevent to use Plex way of only scanning for changes in subsequent calls

“Exception caught determining whether we could skip ‘Zoids Genesis’ ~ Null value not allowed for this type” is present for zero size files i used in my tests… took a while to realise. Real files works no exceptions. I keep you all posted here when i release it along with the persistant custom match on folder name suggested by AtomicStrawberry

691

current scanner works very well but is slow apparently for scanning so i am re-writing it to use more Plex compatible behaviour so we have the correct status of scan and metadata when running it at the bootom, and much quicker and less prone to corruption

i rewrote the engine at last and shed few lines in the way (and lost plexignore support for now, dvd and bluray support stil lthere) BUT it still support grouping folders (dump multiple dragon ball series in one parent “dragon ball” folder which plex doesn’t support as it put all episodes in the first series, even when giving the right series name).

For those willing to test it, i need feedback to move forward with the code:
. install this scanner as separate file https://gist.github.com/ZeroQI/26d26f91af04c876b1a0
. create a new library towards the same folder(s)
. Tell me the number of plex series and episodes, initial scan time and refresh scan times in seconds
. Tell me if it behave the same or what is missing/not behaving
. Issue: attach “plex media scanner (ASS) - library.log” and “plex media scanner.log”.

692

It seems that creating a new library doesn’t work. In my case, after creating the new library with the new experimental scanner it started updating the previous library (didn’t create a new log, instead adding to the previous log) and filling the new library. It was using the new scanner too.

Weird.

I’ve deleted both libraries and created a new one with the Alpha agent. Now it is working correctly, creating a new log.

Scan still ongoing. But it is still slow, taking exactly 10 minutes to scan 349 files out of 4484 on the base folder.

It is possible that Plex has issues with huge folders? I will be doing some tests.

693

Confirmed, lots of individual folders are far faster, even with the same amount of files. Ten times faster, in fact.

694

Thanks to you i know multiple active versions are a bad idea :stuck_out_tongue:
òoooh seem better :D. the refresh should be much better too if i am correct…
the root files and grouped series are still handled the old way but separate folders allow to only rescan tge folder when needed. have more issues when files on root too
I believe the folders with 200+ files are slower too…

695

Well, it takes five minutes with a 900~ files folder. The bigger the folder, the slower the scanner. At that point is about 50% faster than the previous version. With my entire collection on root, it takes 2:30 hours a new library scan. About an hour less than before.

We’re probably hitting some Plex limitation.

696

Nice improvement it was worth it then… That is exactly the stats i was looking for.
is refresh any faster by any chance as well?
Still trying to reduce and simplify the code, am at 341 lines now, and should be readable.
not sure many use “.plexignore” files but it went and too complex to include back. will include old and new scanner maybe in package

697

Came across something interesting today. The IRL boss been keeping me really busy so not gotten around to doing anything with my plex in months. So today I updated my agent and scanner to the latest version on github, restarted the server and hit a library refresh.

At first glance everything seemed fine but at closer inspection I noticed it started to split some seasons for example “Buso Renkin” it made two entries one with episode 1 to 12 and then a second entry with episode 13 to 26. All episode identified and numbered correctly just listed the anime twice in my library. Same for Gintama one entry with episode 1 to 49 and a second entry with 50 to 201. Did the same with a few more titles.

Fix for it was creating a new library from scratch and everything is fine now, one entry for the anime that was displaying the issue. What was causing the problem I have no idea but thought Id share incase someone else came across the same problem.

On a side note can clearly see a speed improvement over the previous version I was on.

698

@chrno: i did change the metadata id at some point that needed to re-create the library… Probably that but if so make sure you have in plugin data folder the tvdb/episodes folder.
There is an alpha version of the scanner meant to be faster here https://gist.github.com/ZeroQI/26d26f91af04c876b1a0 that use the normal scanner calls to be faster. doesn’t support plexignore files though…

699

Yeah I have had the folders since the 1st time we needed to create them. Recreating the lib fixed the problem so not to bothered about it.

Only issue I have now is it seems to be skipping pokemon advanced ep 58 to 109. Busy moving it to a separate folder for testing to see why its not picking up.

700

Please attach custom scanner logs the answer should be in it