Absolute Series Scanner (for anime mainly)

frosty5689 · March 12, 2017, 11:32pm

Hi,

I recently updated ASS and Hama to latest and noticed grouping folder no longer behave as intended. For some reason, ABS only creates 1 show for different seasons of an anime when using grouping folder.

I tested using Aldnoah.Zero sample from the sample library provided on Hama’s GitHub releases page.

I’ve attached the logs below.

It seems for some reason it only created one of the two shows? (Should be “Aldnoah.Zero” and “Aldnoah.Zero (2015)”)

From the ASS logs everything seems fine, but the standard scanner log is saying only 1 show was created out of 2.

Not sure if this is ABS problem or Plex problem.

Thanks!

ZeroQI · March 13, 2017, 1:26am

@frosty5689 thanks for attaching relevant logs on first post. You restore my faith in humanity.

Grouping folders were meant to be skipped and to be added as additional root folder

Plex can only allow multiple series to be created if done from the root scan folder , not the subsequent serie folder scan, otherwise it puts all in the first one.
putting all in root folder scan make caching not work, needing to be scanned every time even if series folder is unchanged, is a hack, which is why it is now meant to be skipped but failing there…

github.com

ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute Series Scanner.py

# -*- coding: utf-8 -*-

###### library  ########################################################### Functions, Constants #####
import sys                                                           # getdefaultencoding, getfilesystemencoding, platform, argv
import os                                                            # path, listdir
import tempfile                                                      # NamedTemporaryFile
import time                                                          # strftime
import datetime                                                      # datetime
import re                                                            # match, compile, sub
import fnmatch                                                       # translate
import logging, logging.handlers                                     # FileHandler, Formatter, getLogger, DEBUG | RotatingFileHandler
import Utils                                                         # SplitPath
import Media                                                         # Episode
import VideoFiles                                                    # Scan
import Stack                                                         # Scan
import inspect                                                       # getfile, currentframe
import ssl                                                           # SSLContext
import zipfile                                                       # ZipFile, namelist
import json                                                          # loads
from lxml import etree                                               # fromstring

This file has been truncated. show original

    if len(reverse_path)>1 and path.count("/"):         #if grouping folders, skip and add them as additionnal folders
      Log.warning("Grouping folder: '%s' skipped, need to be added as root folder if needed" % path)
      Log.info("".ljust(157, '-'))
      return

For some reasons it is not seen as grouping folder, so not skipped so impacted
Please try newly published code done following your post?

github.com

ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute Series Scanner.py

# -*- coding: utf-8 -*-

###### library  ########################################################### Functions, Constants #####
import sys                                                           # getdefaultencoding, getfilesystemencoding, platform, argv
import os                                                            # path, listdir
import tempfile                                                      # NamedTemporaryFile
import time                                                          # strftime
import datetime                                                      # datetime
import re                                                            # match, compile, sub
import fnmatch                                                       # translate
import logging, logging.handlers                                     # FileHandler, Formatter, getLogger, DEBUG | RotatingFileHandler
import Utils                                                         # SplitPath
import Media                                                         # Episode
import VideoFiles                                                    # Scan
import Stack                                                         # Scan
import inspect                                                       # getfile, currentframe
import ssl                                                           # SSLContext
import zipfile                                                       # ZipFile, namelist
import json                                                          # loads
from lxml import etree                                               # fromstring

This file has been truncated. show original

if doesn’t work Above the if test above , aligned, add:
Log.info("reverse_path: '{}', len(reverse_path): '{}', path: '{}', path.count('/'): '{}'".format(reverse_path, len(reverse_path), path, path.count('/')))
if work let me know. it should say grouping folder ‘Aldnoah.Zero’ skipped and you should add as root folder

frosty5689 · March 13, 2017, 9:54am

@ZeroQI

Haha. I work as a Software Developer so I feel your pain all the time.

I tried the latest code from master, it no longer adds any of the two season as shows now.

It seems to have skipped both entirely? Not sure if that is the intended behavior? It used to work this way, so I have no choice but to move them to root folder?

I have attached the logs.

Thanks

ZeroQI · March 13, 2017, 7:56pm

@frosty5689 Yes that the expected behavior. You should have in your library two root folders now:

S:\Media\Anime2
S:\Media\Anime2\Aldnoah.Zero

It was specified in the Readme (always in my signature): https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/README.md

Grouping folder

If you use “Grouping folder / Show Name / Season 1 / Show Name e01.ext” convention from the root, it will now be skipped. You can just add it as additional root folder in the library: “D:/Anime/Dragon Ball/” for “D:/Anime/Dragon Ball/[2] Dragon Ball Z” folder for example…

it didn’t recognise the path separator character correctly so kept grouping folders and they merged into the first serie, which is why i remove them and you have to add them as additional root folder.

Yes, it use to work when i scanned all in the root folder BUT Plex had to scan all folders and filed every time whereas plex doesn’t re-scan if nothing changed if done right and had frequent database corruption and allowed removing folders full of crap at the same time)…

frosty5689 · March 13, 2017, 8:28pm

@ZeroQI I see… That is what I’m afraid of. The description in the README.md confused me for a second thinking it was still supported.

Maybe change “If you use” to “If you used to use”. That makes it clear a feature is no longer supported more.

Thanks for all the help. Now I have to debate over adding each one to the library scan folder list, or putting them all in root

ZeroQI · March 13, 2017, 10:13pm

@frosty5689 Thanks for the precision. Am French so this language nuance i didn’t consider. updated the readme accordingly

i did find the function cool, but not many were interested when i coded it and it came at the great expense of scanning time and corruption…
you shouldn’t have that many grouped series, but all will be indicated in the scanner log

frosty5689 · March 14, 2017, 3:28am

@ZeroQI Glad I was able to provide a better description for the README.md

Yeah, I did notice scanning is a lot faster than what I was used to. I will reorganize my library to work around this change.

Keep up the good work!

It would be great if the scanner functions are unit tested though, rather than having to rely on Plex itself to test. Just my two cents, but probably too much to ask for

ZeroQI · March 14, 2017, 8:20am

@frosty5689 unit tested though? i don’t get it… Please expend…

frosty5689 · March 15, 2017, 2:33am

I am not too familiar with unit testing Python code. But the general idea is to be able to isolate code down to as small as possible. So they can be easily tested by calling it with predefined input and verified using expected output.

If the scanner is broken down into multiple functions or classes, you can inject dependencies or “fake” dependencies and test them individually.

With unit tests, you can easily modify something and not worry about it accidentally breaking something else, because a unit test will fail and let you know of the problem.

I know English is not your first language so I’m not sure if any of what I said made any sense.

https://docs.python.org/2/library/unittest.html

Here’s a link to French Wikipedia

frosty5689 · March 15, 2017, 5:47am

@ZeroQI I just encountered a problem using anidb2. It seems the scanner is stuck in an infinite loop? As nothing is being logged and it is perpetually scanning.

I have attached the custom ASS logs.

In the anidb2.id I put “11273” without quotes

There are no custom mappings.

Thanks

frosty5689 · March 15, 2017, 7:04am

@ZeroQI I found the issue and created a pull request for it on GitHub.

I believe Linux is also infinite looping with anidb2 right now after your change to use os.sep. (it wasn’t before since ‘is not’ somehow worked when both are strings)

Windows has always had the problem of infinite looping with anidb2 (not sure if that code block has always been like that)

ZeroQI · March 15, 2017, 8:22am

@frosty5689 My english is fluent, i work in english daily, but just never heard of that method or module… Thanks for the pull request, will be wary of “is” from now on…

dkenpachi · April 14, 2017, 10:43am

no log folders on mac in directory option on server to download logs but none make sense dates every are 1979 none of my anime show using this scanner any other works can some kindly upload a screenshot of how their library folders ? just so i can get an idea of this please?

ZeroQI · April 14, 2017, 10:59am

github.com

ZeroQI/Absolute-Series-Scanner/blob/master/README.md

## Plex scanner responsibilities
A Plex Series Scanner makes the video files showing in Plex and populate the following for the video files:
- Series name (including forced id passed to agent search() function)
- Series year (optional)
- Season number
- Episode number
- Episode title (not filled by plex default series scanner, but ASS fills it, but this will be overwritten by the metadata agent)

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

If you post anything on the scanner github or thread for any issue like poster missing/wrong, episode thumbnail/screenshot/title/summary missing/wrong, then you clearly haven't read this and need to pay the RTFM tax by donating (or you just find this the best anime scanner and/or agent, much apreciated average is 5 euros). Also applies if you want assistance but didn't follow troubleshooting steps or include logs...

## Plex Agent responsibilities
A Plex metadata agent will:
- Search the metadata source using the Series name given by the scanner and assign the series a guid, a unique identifier used to download the metadataa informations
- Update all the metadata information ((Series Title, summary year, episode title, summary, posters, fanart, tags, ...) for series and files showing in Plex thanks to the scanner. 
- Full list of fields here:  see https://github.com/ZeroQI/Hama.bundle/wiki/Plex-Metadata-Fields-Available for full list of fields)
Any information missing or wrong inthere in Plex is an Agent issue, refer to the Agent readme here: https://github.com/ZeroQI/Hama.bundle/blob/master/README.md

## Which Metadata/Title source to select?

This file has been truncated. show original

Install

Download https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner.py
Save into […]/Plex/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py Note:
“Scanners” and “Series” folder are not created by default and will need creating.
“Scanners” folder will be at the same level as “Plug-in Support” folder (in other words the same parent directory)
“Absolute Series Scanner.py” resides in Series folder, do not create an additional folder not listed like “absolute-series-scanner-master”

Logs, check url: https://support.plex.tv/hc/en-us/articles/200250417-Plex-Media-Server-Log-Files

dkenpachi · April 14, 2017, 11:14am

found my log folder, my bad for not checking the forum support the downloaded folder made zero sense to me thisis the only scanner log to show after i made a library

ZeroQI · April 14, 2017, 11:26am

Apr 14, 2017 07:11:51.926 [0x7fffc54f23c0] ERROR - No module in Absolute Series Scanner
Apr 14, 2017 07:11:51.927 [0x7fffc54f23c0] ERROR - Error in Python: Looking up module:
Traceback (most recent call last):
File “/Users/renaldobryan/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py”, line 119, in
set_logging(“Root”, LOG_FILE_LIBRARY)
File “/Users/renaldobryan/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py”, line 108, in set_logging
handler = logging.handlers.RotatingFileHandler(os.path.join(LOG_PATH, filename), maxBytes=1010241024, backupCount=backup_count) #handler = logging.FileHandler(os.path.join(LOG_PATH, filename), mode)
File “/Applications/Plex Media Server.app/Contents/Resources/Python/lib/python2.7/logging/handlers.py”, line 117, in init
BaseRotatingHandler.init(self, filename, mode, encoding, delay)
File “/Applications/Plex Media Server.app/Contents/Resources/Python/lib/python2.7/logging/handlers.py”, line 64, in init
logging.FileHandler.init(self, filename, mode, encoding, delay)
File “/Applications/Plex Media Server.app/Contents/Resources/Python/lib/python2.7/logging/init.py”, line 913, in init
StreamHandler.init(self, self._open())
File “/Applications/Plex Media Server.app/Contents/Resources/Python/lib/python2.7/logging/init.py”, line 943, in _open
stream = open(self.baseFilename, self.mode)
IOError: (2, ‘No such file or directory’, ‘/Users/renaldobryan/Library/Application Support/Plex Media Server/Logs/Plex Media Scanner (custom ASS).log’)

Yeah the scanner crash due to logs file missing, possibly cannot write log file/Users/renaldobryan/Library/Application Support/Plex Media Server/Logs/Plex Media Scanner (custom ASS).log due to permissions…

Were the logs from that same folder?
Can you check folder permissions for write access ?
Please give the command lines used to solve and mac os version and will add to the readme as known issue

dkenpachi · April 14, 2017, 11:48am

Yes from same folder
check permissions were read only , changing to read & write still produces that same error in log “no such file”
didnt use terminal here found log folder at ~/Library/Logs/Plex Media Server used get info to see permissions (sorry if that’s not what you meant)
MacOS version 10.12.4 (16E195)

dkenpachi · April 14, 2017, 12:13pm

Hama agent works with normal plex media scanner so i guess i can use that for now wont get folder in library saying Arc xxx (which is what i believe the purpose of ASS is along with episode numbering_)

ZeroQI · April 14, 2017, 1:54pm

make sure it’s write only for everybody or even create an empty file with that name by hand…

well absolute numbering, more permissible and skips Arc xxx folders transparently, yes
can show seasons in plex when ep are in absolute numbering (tvdb2/3 mode) and map anidb numbering to a tvdb display in plex seamlessly (anidb2 mode), etc…

dkenpachi · April 16, 2017, 12:39pm

@ZeroQI said:
make sure it’s write only for everybody or even create an empty file with that name by hand…

well absolute numbering, more permissible and skips Arc xxx folders transparently, yes
can show seasons in plex when ep are in absolute numbering (tvdb2/3 mode) and map anidb numbering to a tvdb display in plex seamlessly (anidb2 mode), etc…

made file set permissions still nothing attached log idk if you’ll see something different

Topic		Replies	Views
Series Better ABsolute Scanner (S - BABS) Dev/API Corner scanner-agent-dev	68	3027	December 21, 2019
Better ABsolute Scanner (BABS) Dev/API Corner scanner-agent-dev	67	24019	December 20, 2019
Custom Plex Scanners Français - French	23	1045	December 20, 2019
[Rel] HTTP Anidb Metadata Agent (HAMA) Dev/API Corner scanner-agent-dev	2122	71295	July 28, 2023
[REL] Extended Personal Media Scanner Dev/API Corner scanner-agent-dev	77	14517	December 21, 2019

Absolute Series Scanner (for anime mainly)

Grouping folder

Related topics