Absolute Series Scanner (for anime mainly)

@deusxanime The issue is that anidb2 is a lazyman tag that relies on all xmls to map properly. Since it can’t download ithe xml it will fail mapping to thetvdb
To group it need the same guid. anidb2 translate into tvdb force id.

You could in the meantime do the translation by hand with [tvdb-78881] or [tvdb-78881-s1] or [tvdb-78881-s1e1]

Try to open the xml from the very same machine but the certificate currently is valid for that path so dunno how i could fix scanner side

@ZeroQI Looks like someone has opened an issue on github for the same problem, so doesn’t appear to be only my problem on the legacy scanner. Using the tvdb tag would probably work for those series, but more complicated ones that have specials and such is where anidb2 mappings really come in handy.

Btw, did you have any thoughts on the other part of my questions above, the beta scanner? Wondering why it is not merging the titles together. Could it be the same issue with anidb2 tagging, but the beta scanner handles it a bit better and just defaults over to regular anidb matching rather than converting it to tvdb as it should? Or something completely different?

@deusexanime

  • Since all series share the same guid (tvdb-xxxx) then they are merged automatically…anidb2 is live replaced by the scanner in the serie title with the[tvdb-xxxx] type of force id while offsetting the ep and season so the agent doesn’t even know something changed, and it’s not working there possibly an xml issue in “Plex Media Scanner.log”.
  • What do you think about the last filelist, scanner, agent search, agent update kept ?I thought it was clever for troubleshooting
  • how is the grouping folder without tag needed working ?

Major version change is incoming:

  • adding multiple Anidb XML if multiple seasons using scudlee xml default season. If season on disk = default season, load the anidb xml.
  • redoing of the beta scanner as the recursive scanner call works but seem to give path missing in internal xml’s
  • once that is completed and tested, will merge beta into main branch

@ZeroQI Yep I prefer anidb2 tagging as I like to keep them named in anidb naming and episode numbering, but Plex of course works better with tvdb especially for combining seasons into a single series/entry. Still not sure why it isn’t working in the beta scanner. It is not merging them into a single tvdb series but seems almost like it is treating them as anidb (no “2”) tags and keeping them as separate entries.

Does the new beta scanner not require the [grouping] tag for grouping folders anymore? Or should that still be in the folder name for those grouping folders?

It does not anymore require tagging, that is why i wrote it BUT it seem to cause other issues so am redoing another version with plexignore support. Need to finish agent with multiple season anidb metadata support and that is tricky, then new beta scanner version then i merge all…

@deusxanime After 2 full days of swearing at the screen while coding, i reverted to kind of legacy scanner that calls sub folders from root scan if there is video files actionnable in said folders, and it does seem to work
For the life of me couldn’t get the plex caching and grouping folders working by using Plex naturally called functions as soon as i loaded manual folders, must have added checks since i last tested scanner boundaries,… Functions are including video file in zip, .plexignore file support including relative subfolders and proper logging for root folder and per series in agent data folders (*.filelist.log showing files and plexignore and .scanner.log for the serie season and ep translation and assignments) so the agent puts the files ).agent-search.log and *.agent-update.log in same folder (in a folder with library name if you created your plex token in the plex root folder one level higher than logs folder where it used to reside at)

Please let me know if testing:

  • the speed and if any issue it is for me with dummy files at 1.5s/ series including DB super and Bleach. -
  • if all is detected as it should
  • logging file appearance and functionnality

Improvements possible

I’ve been trying for hours but I can’t figure out how the .plexignore file works. The only way I seem to be able to get it to work is with . in the library root directory.

So I have the the .plexignore file in the root of my library and it’s body is this

Blend S/
Citrus/
Yuru Yuri/
Shokugeki no Soma OVA/
But I can’t get it to ignore those specific folders. Adding the .plexignore file with . to those specific folders doesn’t seem to work either. Any clue what I’m doing wrong I’ve been googling all over for hours but I still can’t figure it out.

The readme was explicit
Latest version supporting grouping folders without having to add the folder is from the 15th of November 2015 and also support plexignore files which are not supported with the scanner currently (let me know if i am wrong) https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/Scanners/Series/Absolute%20Series%20Scanner.py

I suppose you weren’t using this version. I actually added last week on the beta scanner the full .plexignore support: https://github.com/ZeroQI/Absolute-Series-Scanner/blob/Beta/Scanners/Series/Absolute%20Series%20Scanner.py

Rules are here: https://support.plex.tv/articles/201375253-excluding-new-content-with-plexignore/
Rules states the star at the end you have nothing no note sure it will work as you write it…

Configure either

  • root/.plexignore with ‘Shokugeki no Soma OVA/*’ inside
  • root/Shokugeki no Soma OVA/.plexignore with ‘*’ inside

I am using the beta version actually but I can’t get it to work. I changed the root/.plexignore with ‘Shokugeki no Soma/*’ but that didn’t seem to work. Switching to plex scanner, scanning and then switching back seems to work fine though.

ah yes that’s normal
Plex doesn’t re-launch the scan for folders if nothing changed video files wise but re-run it upon changing scanner…

Oh haha did not know that. I just thought I kept messing something up. Thanks!

new beta release, supports grouping folders without tags, .plexignore, and caching, pretty fast.
Please feedback any issues. Fixed also logging so all logs saved to agent data folders and also usual hama log

tl;dr: Automatically avoiding Ordered Chapter multiple OP/ED with multiple seasons episode collisions?

I hesitate to post this, but I’ve looked for over 30 minutes and can’t find an answer. (Too much search noise; best hints are: here, here and here. “Normally.” What about AB-normally? :slight_smile: ) I suspect this is an easy fix or probably just “NO!”, but:

I’ve got Plex on a minimal CentOS7 LXC container hosted on Proxmox 5.1 using a Synology DSM 6.1 Samba share as media storage. It’s all working fine, I’m loading up a little bit of media at a time to check things out. (I’m moving Plex away from physical boxes – a 2nd VM hosts only Movies, TV shows, and the DVR. If things overload the CPU I’ll move them to the 2nd Proxmox physical box. They’re not clustered, but if they’ll auto-balance I’ll consider it.)

But ASS/HAMA is all working TOO good: Anime Ordered Chapters OP/EDs map to EP 101 and 151 and on, fine. Now that’s Season 1 OP/EDs. Season 2 OP/EDs map to … 101 and 151 and on, so Plex dutifully shows overlaid special episodes. I can force the Season N issue by naming them S00E123 – but isn’t there a way to use the “next” number?

Of course the scanner itself is non-stateful, so any OP2 is just going to be 102. But what about OP1a, OP1b, etc?
Is there a hint so that I could easily NOT worry about naming collisions like this? I can’t be the only person with this problem. (… can I? Why are you all looking at me like that?? :smile: )

@cbussa Because your smiley is creepy… It looks like the joker trying to really smile with blackface make-up… That’s why people look at you… still…

A distributed transcoding backend for Plex may apeal to you: https://github.com/wnielson/Plex-Remote-Transcoder

Notes:

  • OP/EDs are only supported by AniDB but only have 1 season so following AniDB numbering would be better for OP support. AniDB doesn’t label OP/EDs as 1a, 1b, etc…
  • If you have multiple seasons then the series uses tvdb numbering…
  • Plex doesn’t do specials per seasons
  • TheTVDB doesn’t do specials per seasons
  • ASS doesn’t do specials per seasons
  • “what about OP1a, OP1b, etc?” this format is ALREADY supported. regex for openings support letter: (?P\d{1,2}[a-z]?) and i did code that ages ago
  • You might consider Kodi (former XBMC) which can insert specials between episodes…
  • you have to follow anidb numbering or TheTVDB and if you followed none, no support is given
  • if you just dumped a loosely anidmbered series (nyaatorrent?) into season folders and complain, i will shame you, and same if you are not using beta branch

OP1a, OP1b, OP2 should become s00e1-3 according to code below:

    if not ep.isdigit() and len(ep)>1 and ep[:-1].isdigit():                                                                                              ### OP/ED with letter version Example: op2a
                AniDB_op [ offset + int(ep[:-1]) ] = ord( ep[-1:].lower() ) - ord('a')                                                                              # {101: 0 for op1a / 152: for ed2b} and the distance between a and the version we have hereep, offset                         = str( int( ep[:-1] ) ), offset + sum( AniDB_op.values() )                             # "if xxx isdigit() else 1" implied since OP1a for example... # get the offset (100, 150, 200, 300, 400) + the sum of all the mini offset caused by letter version (1b, 2b, 3c = 4 mini offset)
                ep, offset                         = int( ep[:-1] ), offset + sum( AniDB_op.values() )                                                       # "if xxx isdigit() else 1" implied since OP1a for example... # get the offset (100, 150, 200, 300, 400) + the sum of all the mini offset caused by letter version (1b, 2b, 3c = 4 mini offset)

Could you actually attach the **beta scanner logs output **for what you complain about?
I don’t have any logs, despite your first link showing logs attached and the third being my readme for main branch… Why are you not using the beta branch?

@ZeroQI said:
@cbussa Because your smiley is creepy… It looks like the joker trying to really smile with blackface make-up… That’s why people look at you… still…
Thanks for looking at me – even if you **are ** crosseyed. :smiley:

Plex distributed transcoding backend Hmmm. I’ll have to look into that.

OP1a, ED2b, etc – oops, misspoke there, I meant that they ARE supported, so is all the info iterated on the back end or is there some state stored there? Otherwise how does it know OP2/105 comes after OP1d/104?

“if you just dumped” – Nyaa, I wouldn’t do that. Although occasionally I’ll get lazy and see what decodes correctly and then fix the remainder.

Sure, I’ll attach the logs – I’ll strip out the other MKV files and rescan so it’ll be easy to see.

Oh! The message links earlier were just me referencing previous searches (basically you) I tried to use to solve the problem. I hate people that say “Solve my problem, and BTW I’ve not even opened the box it comes in yet.” Wha…?? if you don’t put at least SOME effort into it, then why on Earth should I – unless it’s literally my job and you’re paying me to do so. [Hi Jerry! You reading this? :slight_smile: ]

I’ve killed the log files before 11PM last night, they shouldn’t be useful. I’ve still got them if you think they are, though.

These are NOT the beta log files! Plex log files, my verbose description file. You want to read the last 40 lines of the latter file, it has the manual details of what the Plex log will show.

Beta branch. Sigh, I tire of updates, of bugs, of fixes; I’ve been seeing them for on 40 years now. I want “known stable works” or “fixes obvious broken”, which will STILL have bugs but maybe less-so. Or maybe “yum/apt update” which are mostly automatic, but NOT like Chrome or Windows which occasionally remakes the GUI and muscle memory workflow. Sigh … if you really want me to, I will. Github Hama Beta, right?

I get a bit verbose now and then, sorry about that. Thanks a bunch for helping out, I know you’re busy! ASS/Hama works great and I’ll rename files or just dump 'em in Extras if I have to.

@cbussa Looking at the same thing from different angles at the same time is a skill as long as you control it… :smiley:

How does it know OP2/105 comes after OP1d/104? I pasted the code but i might be obscure since i coded that myself and am self taught python… 1a is 1 1b is 2 etc the 2 afterwards is shifted to 3 or if you had a 1c… You have to have the titles similar so the natural sorted filelist is giving them in the right order…

You wouldn’t believe what i see sometimes in logs… Not renaming anything but saying it’s not 100% perfect or the source like tvdb has a problem but the agent should magically know… i think it is currently the best agent despite bugs as you can have movies and series and mix anidb and tvdb numbering…

The thing is i only keep one version on my computer of the code and i need reports to solve problems as to be honest i don’t use Hama for myself so i need users to report bugs. i introduced load of changes as well, the beta splitted the source into files and allow to select per fields the metadata source you want…

@ZeroQI Sorry I was out of town for a couple weeks last time you posted and just hadn’t gotten back to testing until now. I tried the current beta ASS & HAMA and it seems to be working well for me in my small test folder. “The World God Only Knows” (which consisted of a grouping folder with 6 individual series/OVAs with each in their own folder) previously would not combine completely. It was giving me 2 separate folders. I deleted the library, updated ASS/HAMA just now, restarted Plex, and recreated the library. Now with the latest version it is working again and combining into a single series. The others in my test library are combing as well.

One problem though I do see is that the sort title is not working properly. “The World God Only Knows” is sorting on “The” rather than on “World” as you would expect. Ideally it should drop “The”, “A”, and “An” from the beginning of sort title (at least when English titles are selected).

Amazing work and appreciate it as always!

@deusexanime Agent setting “Sort Title Prefix Removal” which is disabled by default governs that. could you tell me your opinion then, do you feel it it should be enabled by default or the setting removed alltogether and sorting done every time moving to the end ["The ", "A ", and "An "] ? that would remove a setting from a already long list, not a bad thing… The question is, is there somebody that will regret the default mode behavior ?

@ZeroQI Thanks, I changed that setting and it fixed it. (Had to Plex dance to get it to change.) I don’t know if I’ve ever noticed that setting was there before, though it has been a while since I completely deleted a library and re-added it. Is it fairly new or am I just forgetting about setting it previously?

I think the default should be to have it enabled, for 2 reasons:

  • It is standard in English language to drop The, A, and An articles when sorting books/movies/etc
  • Plex does this by default as well I believe, so it is expected to globally be the same

You could remove it completely as you said and just have it enabled always. That wouldn’t bother me, but I can’t say if others might feel different or not.

Thanks!

@deusxanime Will therefore do that. It does make sense, that’s what the sort title is for, after all…
Btw, working at the moment on:

  • full bi-directinnal multi anidb to tvdb mapping which will avoid absence of meta for season 2+ and specials… That means major re-write as multi anidb xmls coexist in anidb source if tvdbids or anidb2 are used…
  • tvdb api v2 which should allow ep titles in multiple languages withh language selection fully working (currently works if anidb before tvdb in title metadata source)