[Rel] HTTP Anidb Metadata Agent (HAMA)

Dev Community Dev/API Corner
961

@Saitoh: “Plex Media Scanner (custom ASS).log” was created by the non-alpha scanner.
I know wince “2015-11-28 16:19:14 === filelist created - now processing it” this phrase is not present in the alpha scanner

The alpha scanner is the only one i am working on since the non alpha is a hack and doesn’t let plex cache properly so the alpha will replace the normal scanner as soon as all bugs are removed.

I specifically modified logging in the alpha since you were having issues but you seem to be still using an old scanner.

Please use https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner%20-%20Alpha.py

962

@ZeroQI said:
@Saitoh: “Plex Media Scanner (custom ASS).log” was created by the non-alpha scanner.
I know wince “2015-11-28 16:19:14 === filelist created - now processing it” this phrase is not present in the alpha scanner

The alpha scanner is the only one i am working on since the non alpha is a hack and doesn’t let plex cache properly so the alpha will replace the normal scanner as soon as all bugs are removed.

I specifically modified logging in the alpha since you were having issues but you seem to be still using an old scanner.

Please use https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner%20-%20Alpha.py

im using the Alpha i just remove Alpha from the name when i add it to plex scanner folder.

963

@Saitoh183: Please read what i am writing. the custom log you sent contain the string “filelist created - now processing it” which is not in the alpha version.
https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner%20-%20Alpha.py

It is however in the normal version:

Please search for the string “filelist created - now processing it” (without the double quotes) in your library configured scanner source code. if you can find it, then it clearly isn’t the alpha version i gave the link for since it doesn’t have it. you can search it in the alpha version on github and you will see you are using the wrong scanner… If you are using the same scanner then the log files you sent are not generated by it…

964

@ZeroQI said:
@Saitoh183: Please read what i am writing. the custom log you sent contain the string “filelist created - now processing it” which is not in the alpha version.
https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner%20-%20Alpha.py

It is however in the normal version:
Absolute-Series-Scanner/Scanners/Series/Absolute Series Scanner.py at master · ZeroQI/Absolute-Series-Scanner · GitHub

Please search for the string “filelist created - now processing it” (without the double quotes) in your library configured scanner source code. if you can find it, then it clearly isn’t the alpha version i gave the link for since it doesn’t have it. you can search it in the alpha version on github and you will see you are using the wrong scanner… If you are using the same scanner then the log files you sent are not generated by it…

Nope…i did a search for the string filelist created - now processing it in Absolute Series Scanner.py and it is not there. As i said i have the right file…i just rename it to Absolute Series Scanner.py when i copy it.

in the log i sent you, filelist created - now processing it is dated

2016-02-05 20:52:37 === filelist created - now processing it

this date is old and not relevant and is the only reference i found

I can create a new log if you prefer :slight_smile:

965

all files in logs.7z are from 2015-11-28… beginning of log created with old scanner then…

Fate Zero (2012)[anidb-8880]: you are missing a space between “Fate Zero (2012)” and “[anidb-8880]” so the anidb force will be ignored

I always wipe the logs and re-create the library (temporary one) to create new logs.

i do not have the problem with *.jpg, so it might be a python windows or os specific issue only you have and if so might need a remote control using teamviewer.

Please replace “ext = os.path.splitext(file)[1].lstrip(’.’).lower()” with “ext = os.path.splitext(os.path.basename(file))[1].lstrip(’.’).lower(); Log(“Ext: ‘%s’” % ext)” and attach log again including new filelist

if it fail i would be faster with a team viewer to your pc and edit the scanner file live and test.
Shouldn’t tae long but this is baffling me. below all not in the video_exts list is skipped with messages, which doesn’t seem to hapend. Therefore, the jpgs should appear in the filelist, please check that as the one you sent is 2015-11-28…

  files_to_remove = []
  for file in files:
    ext = os.path.splitext(file)[1].lstrip('.').lower()
    if ext in video_exts:
      for rx in ignore_files_rx:                                                                                        # Filter trailers and sample files
        if re.match(rx, file, re.IGNORECASE):  Log("File:   '%s' match ignore_files_rx: '%s'" % (file, rx)); files_to_remove.append(file);  break
      else:  
        with open(os.path.join(LOG_PATH, FILELIST), 'a') as log_file:  log_file.write(file + "

") #add to filelist
else: Log(“file: ‘%s’, ext: ‘%s’ not in video_ext” % (file, ext)); files_to_remove.append(file); continue
for file in files_to_remove: files.remove(file)

966

Updated to latest version of HAMA and alpha scanner and so far no issues.

967

@ZeroQI said:
all files in logs.7z are from 2015-11-28… beginning of log created with old scanner then…

Fate Zero (2012)[anidb-8880]: you are missing a space between “Fate Zero (2012)” and “[anidb-8880]” so the anidb force will be ignored

I always wipe the logs and re-create the library (temporary one) to create new logs.

i do not have the problem with *.jpg, so it might be a python windows or os specific issue only you have and if so might need a remote control using teamviewer.

Please replace “ext = os.path.splitext(file)[1].lstrip(‘.’).lower()” with “ext = os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower(); Log(“Ext: ‘%s’” % ext)” and attach log again including new filelist

if it fail i would be faster with a team viewer to your pc and edit the scanner file live and test.
Shouldn’t tae long but this is baffling me. below all not in the video_exts list is skipped with messages, which doesn’t seem to hapend. Therefore, the jpgs should appear in the filelist, please check that as the one you sent is 2015-11-28…

  files_to_remove = []
  for file in files:
    ext = os.path.splitext(file)[1].lstrip('.').lower()
    if ext in video_exts:
      for rx in ignore_files_rx:                                                                                        # Filter trailers and sample files
        if re.match(rx, file, re.IGNORECASE):  Log("File:   '%s' match ignore_files_rx: '%s'" % (file, rx)); files_to_remove.append(file);  break
      else:  
        with open(os.path.join(LOG_PATH, FILELIST), 'a') as log_file:  log_file.write(file + "

") #add to filelist

    else:  Log("file: '%s', ext: '%s' not in video_ext" % (file, ext));  files_to_remove.append(file);  continue
  for file in files_to_remove:  files.remove(file)

Lol… I just realized I attached the wrong zip… No wonder you can’t see anything… Lol… My bad

968

@saitoh183 said:

@ZeroQI said:
all files in logs.7z are from 2015-11-28… beginning of log created with old scanner then…

Fate Zero (2012)[anidb-8880]: you are missing a space between “Fate Zero (2012)” and “[anidb-8880]” so the anidb force will be ignored

I always wipe the logs and re-create the library (temporary one) to create new logs.

i do not have the problem with *.jpg, so it might be a python windows or os specific issue only you have and if so might need a remote control using teamviewer.

Please replace “ext = os.path.splitext(file)[1].lstrip(‘.’).lower()” with “ext = os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower(); Log(“Ext: ‘%s’” % ext)” and attach log again including new filelist

if it fail i would be faster with a team viewer to your pc and edit the scanner file live and test.
Shouldn’t tae long but this is baffling me. below all not in the video_exts list is skipped with messages, which doesn’t seem to hapend. Therefore, the jpgs should appear in the filelist, please check that as the one you sent is 2015-11-28…

  files_to_remove = []
  for file in files:
    ext = os.path.splitext(file)[1].lstrip('.').lower()
    if ext in video_exts:
      for rx in ignore_files_rx:                                                                                        # Filter trailers and sample files
        if re.match(rx, file, re.IGNORECASE):  Log("File:   '%s' match ignore_files_rx: '%s'" % (file, rx)); files_to_remove.append(file);  break
      else:  
        with open(os.path.join(LOG_PATH, FILELIST), 'a') as log_file:  log_file.write(file + "

") #add to filelist

    else:  Log("file: '%s', ext: '%s' not in video_ext" % (file, ext));  files_to_remove.append(file);  continue
  for file in files_to_remove:  files.remove(file)

Lol… I just realized I attached the wrong zip… No wonder you can’t see anything… Lol… My bad

All my anime folders contain

Poster.jpg
Banner.jpg
season01-poster.jpg
fanart.jpg

Some also contain

clearart.png
clearlogo.png
logo.png
eps-name - thumb.jpg
landscape.jpg

Also in plex i have the agents setup as Local first then Hama so that my local artwork is picked up instead of internet

969

This filelist is in too perfect order and only contain video files… Was it created by the new alpha scanner ?
If the jpg files make it as episode they should be in the filelist as well and aren’t…

Please replace “ext = os.path.splitext(file)[1].lstrip(’.’).lower()” with “ext = os.path.splitext(os.path.basename(file))[1].lstrip(’.’).lower(); Log(“Ext: ‘%s’” % ext)” and attach log again including new filelist

if the jpg fiels makes it to the new filelist i will need a teamviewer to your desktop

970

@ZeroQI said:
This filelist is in too perfect order and only contain video files… Was it created by the new alpha scanner ?
If the jpg files make it as episode they should be in the filelist as well and aren’t…

Please replace “ext = os.path.splitext(file)[1].lstrip(‘.’).lower()” with “ext = os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower(); Log(“Ext: ‘%s’” % ext)” and attach log again including new filelist

if the jpg fiels makes it to the new filelist i will need a teamviewer to your desktop

did you mean replace

ext = file[1:] if file.count(‘.’)==1 and file.startswith(‘.’) else os.path.splitext(file)[1].lstrip(‘.’).lower()

with

ext = file[1:] if file.count(‘.’)==1 and file.startswith(‘.’) else os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower(); Log(“Ext: ‘%s’” % ext)

If so, i did and removed Fate Zero(2012) and added it back to plex and the scan fired but nothing was added and nothing was written to filelist or scanner log. I even delete all content from logs to make sure they were fresh.

I also did a test without making the change by adding Fate Zero (2012) to my test library. here is the result:

Plex Media Scanner (custom ASS) - ani - filelist Ani.log:

D:\Videos\Ani\Fate Zero (2012)[anidb-8880]\Fate Zero (2012) - 01 -  The Mion River Battle [HorribleSubs].mkv
D:\Videos\Ani\Fate Zero (2012)[anidb-8880]\Fate Zero (2012) - 02 -  Golden Shine [Hatsuyuki].mkv

as you can see, it stop adding after eps 2

Plex Media Scanner (custom ASS) - ani.log :

2016-02-15 12:26:36 === Library "ani", Root: "D:\Videos\Ani",  Launched: '2016-02-15 12:26:36'===================================================================================
2016-02-15 12:26:36 keep_zero_size_files: 'False'
2016-02-15 12:26:36 no_timestamp:         'False'
2016-02-15 12:26:36 season_from_folder:   'False'
2016-02-15 12:26:36 =============================================================================================================================================================
2016-02-15 12:26:36 Scanner call - root: 'D:\Videos\Ani', path: '', dirs: '6', files: '0'
2016-02-15 12:26:36 =============================================================================================================================================================
2016-02-15 12:26:36 Scanner call - root: 'D:\Videos\Ani', path: 'Fate Zero (2012)[anidb-8880]', dirs: '0', files: '32'
2016-02-15 12:26:36 =============================================================================================================================================================
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0000e501 "clearart.png" "Clearart" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0000e502 "fanart.jpg" "Fanart" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e001 "Fate Zero (2012) - 01 -  The Mion River Battle [HorribleSubs].mkv" "The Mion River Battle" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e002 "Fate Zero (2012) - 02 -  Golden Shine [Hatsuyuki].mkv" "Golden Shine" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e003 "Fate Zero (2012) - 03 -  The End of Honor [Hatsuyuki].mkv" "The End of Honor" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e004 "Fate Zero (2012) - 04 -  The Eighth Contract [Hatsuyuki].mkv" "The Eighth Contract" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e005 "Fate Zero (2012) - 05 -  Distant Memories [Hatsuyuki].mkv" "Distant Memories" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e006 "Fate Zero (2012) - 06 -  Where Justice Is Found [HorribleSubs].mkv" "Where Justice Is Found" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e007 "Fate Zero (2012) - 07 -  Return of the Assassin [HorribleSubs].mkv" "Return of the Assassin" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e008 "Fate Zero (2012) - 08 -  Knight on Two Wheels [HorribleSubs].mkv" "Knight on Two Wheels" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e009 "Fate Zero (2012) - 09 -  All the Evil in the World [UTW].mkv" "All the Evil in the World" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e010 "Fate Zero (2012) - 10 -  The Sea at the End of the World [UTW].mkv" "The Sea at the End of the World" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e011 "Fate Zero (2012) - 11 -  The Last Command Seal [UTW].mkv" "The Last Command Seal" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0001e012 "Fate Zero (2012) - 12 -  Fate Zero [UTW].mkv" "Fate Zero" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0000e503 "poster.jpg" "Poster" 
2016-02-15 12:26:36 "season01" s0000e504 "season01-poster.jpg" "Season01-Poster" 
2016-02-15 12:26:36 "Fate Zero (2012)[anidb-8880]" s0000e505 "Thumbs.db" "Thumbs" 
2016-02-15 12:26:36 -------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-15 12:26:36
971

all files even wrong ones should be in file list. Please check the plex scanner logs for errors (should have unsupported character type errors, but any other error paste there)

I meant line 203. replace “file” with “os.path.basename(file)” and add at the end ; Log(“Ext: ‘%s’” % ext)
it should display the extension doubling the output, but allowing me to see the extension to see what part of the tests are conditioned then…

“file: ‘%s’, ext: ‘%s’ not in video_ext” is never displayed which is odd, as it check every extension is in the list of video exts “video_exts” and add the file to “files_to_remove” list, then remove all entries from “files” list the scnner function receives from Plex. i cannot remove while in the “for file in files:” loop or it skips genuine items then…

972

@ZeroQI said:
all files even wrong ones should be in file list. Please check the plex scanner logs for errors (should have unsupported character type errors, but any other error paste there)

I meant line 203. replace “file” with “os.path.basename(file)” and add at the end ; Log(“Ext: ‘%s’” % ext)
it should display the extension doubling the output, but allowing me to see the extension to see what part of the tests are conditioned then…

“file: ‘%s’, ext: ‘%s’ not in video_ext” is never displayed which is odd, as it check every extension is in the list of video exts “video_exts” and add the file to “files_to_remove” list, then remove all entries from “files” list the scnner function receives from Plex. i cannot remove while in the “for file in files:” loop or it skips genuine items then…

Ok… I will do the replacement when I get to work. If we have to do a Teamviewer session, I will be free at around 10pm eastern.

973

@saitoh183 said:

@ZeroQI said:
all files even wrong ones should be in file list. Please check the plex scanner logs for errors (should have unsupported character type errors, but any other error paste there)

I meant line 203. replace “file” with “os.path.basename(file)” and add at the end ; Log(“Ext: ‘%s’” % ext)
it should display the extension doubling the output, but allowing me to see the extension to see what part of the tests are conditioned then…

“file: ‘%s’, ext: ‘%s’ not in video_ext” is never displayed which is odd, as it check every extension is in the list of video exts “video_exts” and add the file to “files_to_remove” list, then remove all entries from “files” list the scnner function receives from Plex. i cannot remove while in the “for file in files:” loop or it skips genuine items then…

Ok… I will do the replacement when I get to work. If we have to do a Teamviewer session, I will be free at around 10pm eastern.

@ZeroQI

changed line to look like

if os.path.splitext(os.path.basename(file))[1].lstrip('.').lower(); Log("Ext: '%s'" % ext) not in video_exts: files.remove(file); continue

When i add a series to folder, scan does not pick it up…no logs are generated

974

You will have error in plex scanner log. Use
if os.path.splitext(os.path.basename(file))[1].lstrip(’.’).lower() not in video_exts: files.remove(file); Log(“Ext: ‘%s’” % ext); continue
Be xarefull at nunber of spaces before command too…

975

@ZeroQI said:
You will have error in plex scanner log. Use
if os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower() not in video_exts: files.remove(file); Log(“Ext: ‘%s’” % ext); continue
Be xarefull at nunber of spaces before command too…

I tried copying your code. The first try , i just replaced the needed sections

Error in plex scanner log:

ERROR - Error in Python: Running scanner:
Traceback (most recent call last):
  File "P:\Data\Plex Media Server\Scanners\Series\Absolute Series Scanner.py", line 203, in Scan
    if os.path.splitext(os.path.basename(file))[1].lstrip('.').lower() not in video_exts: files.remove(file); Log("Ext: '%s'" % ext); continue
UnboundLocalError: local variable 'ext' referenced before assignment
976

@saitoh183 said:

@ZeroQI said:
You will have error in plex scanner log. Use
if os.path.splitext(os.path.basename(file))[1].lstrip(‘.’).lower() not in video_exts: files.remove(file); Log(“Ext: ‘%s’” % ext); continue
Be xarefull at nunber of spaces before command too…

I tried copying your code. The first try , i just replaced the needed sections

Error in plex scanner log:

ERROR - Error in Python: Running scanner:
Traceback (most recent call last):
  File "P:\Data\Plex Media Server\Scanners\Series\Absolute Series Scanner.py", line 203, in Scan
    if os.path.splitext(os.path.basename(file))[1].lstrip('.').lower() not in video_exts: files.remove(file); Log("Ext: '%s'" % ext); continue
UnboundLocalError: local variable 'ext' referenced before assignment

got it to work with some help from my sister…

here is the log

977

Am much slower since i can’t access my plex from work ,should be solved soon

We now at last have the line: “file: ‘D:…\xxx-thumb.jpg’, ext: ‘jpg’ not in video_ext” so extension is read correctly, not in “video_exts” list BUT the “file” entry should be removed from “files” list and isn’t…

I had to put it in a temporary list “files_to_remove.append(file)” as otherwise when i remove an entry in the first loop it skips an element and i then loop in “files_to_remove” list created to remove from “files” list all entries whose extension is not in the “video_exts” list. This doesn’t seem to be present/working BUT should be a quick fix.

Can you please attach the scanner file your kind sister edited so i compare with your output and directly edit that ?

978
979

She look like she knows her stuff, possibly better than me since it’s my first project with hama+absolute series scanner and stil can’t figure github beyond basic stuff

line 211: for file in files_to_remove: files.remove(file)
Can you replace it with 3 lines, same number of spaces (2) as initial line
Log(“files: ‘%s’” % str(files))
for file in files_to_remove: files.remove(file)
Log(“files: ‘%s’” % str(files))

i know it detects file to remove, but i struggle to see where it goes wrong
i will display the “files” list before and after removing entried from it and know then

980

@ZeroQI said:
She look like she knows her stuff, possibly better than me since it’s my first project with hama+absolute series scanner and stil can’t figure github beyond basic stuff

line 211: for file in files_to_remove: files.remove(file)
Can you replace it with 3 lines, same number of spaces (2) as initial line
Log(“files: ‘%s’” % str(files))
for file in files_to_remove: files.remove(file)
Log(“files: ‘%s’” % str(files))

i know it detects file to remove, but i struggle to see where it goes wrong
i will display the “files” list before and after removing entried from it and know then

Got it to work…seems for some reason i was missing the line 211 entirely. After adding the 211, the jpgs are now gone…

now all that is left is Ranma 1/2 Nettou Hen [anidb-193]

Episode are not coming out right and season is Season 1989 instead of season 1

My folder is named as followed:

Ranma 1 2 Nettou Hen [anidb-193]

here are the logs