I run it locally on freebsd.
That is the scanner fault, adding filename but not the path, solved that very bug 16 days ago: https://github.com/ZeroQI/Absolute-Series-Scanner/commit/9a22ed02027a305eb9c01b52507c261f17a0da57
Please update to the latest code and let me know the result, pasting logs if it isn’t solved:
Re-wrote the agent read-me. If imprecise/incorrect please let me know
@djmixman you can put jacksepticeye’s channel id [UCYzPXprvl5Y-Sf0g4vX-m6g]
@djmixman Added support for collection without any channel or playlist id and collection gets assigned. no series or season info since i have no id to take that from but video will show proprer info at least. you have to number eps properly though since the scanner has no playlist to follow
@polayer2017 you have to put the playlist id in the series folder name: ‘Playlist [PLxxxxxxxxxxxxxx]’ for the scanner to assign a proper ep number and the agent to rename the titke properly (TouTube video id on every filename in any case)
@ZeroQI said:
@polayer2017 you have to put the playlist id in the series folder name: ‘Playlist [PLxxxxxxxxxxxxxx]’ for the scanner to assign a proper ep number and the agent to rename the title properly (TouTube video id on every filename in any case)
I have tried both ways and the scanner is still not working.
Naming folders with just ‘Playlist [PLxxxxxxxxxxxxxx]’
This results in the first folder getting imported properly, but 2 and 3 are ignored.
Contents of Playlist Folder#1
Contents of Playlist Folder#2
Using other information with ‘Playlist [PLxxxxxxxxxxxxxx]’
This results in the first folder getting imported, but 2 and 3 are ignored. Metadata and naming are also inconsistent in the multiple trials I have tested.
*All other uploader folders are ignored when scanner is told that files exist on the W:/ drive.
*When I point to an uploader/playlist set of folders I get the results like shown above.
My goal is to have a root folder with uploader/channel name and then playlist(s) found inside. Once I click on the poster I would be taken to the three playlists (seasons) in this case. I have everything stored on a networked folder in the uploader/playlist name/file format. Everything imports fine when using other videos and stock plex scanners (All files are added as single files with no folder organization unless folder view is selected). I know youtube is a different beast when it comes to importing information and I really want to get this to work! Thanks in advance.
RECAP:
“Collection folders” with “custom non-playlist folders” are not being properly grouped into collections and instead are being listed as “_unknown_folder”.
So adding a little more debug info gives me this:
2018-06-22 13:24:21,169 (805ac6400) : INFO (__init__:167) - search() - no id in title nor matching YouTube title: "Gmod Prop Hunt", closest match: "VanossGaming", description: "Vanoss merch HERE!: https://vanoss.3blackdot.com/ Friends in the vid: Nogla - http://bit.ly/$
2018-06-22 13:24:21,169 (805ac6400) : INFO (__init__:53) - Gmod Funny Moments Ep.2 (TTT, Ninja Turtles, MOAB, Prop Hunt) [2EJmwtzKR6o].mkv
2018-06-22 13:24:21,169 (805ac6400) : INFO (__init__:59) - [!] [DJMIXMAN] Call To GetLibraryRootPath
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:60) - [!] [DJMIXMAN] Call To GetLibraryRootPath: dir:
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:76) - [!] Library access denied
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:78) - [!] [DJMIXMAN] Acceess denied to library:
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:79) - [!] [DJMIXMAN] Acceess denied to root:
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:80) - [!] [DJMIXMAN] Acceess denied to path:
2018-06-22 13:24:21,170 (805ac6400) : INFO (__init__:84) - [!] ASS root scanner file present: "/usr/local/plexdata/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/_Logs/_root_.scanner.log"
2018-06-22 13:24:21,171 (805ac6400) : INFO (__init__:174) - Putting folder name "_unknown_folder" as guid since no assign channel id or playlist id was assigned
It looks like it is failing because ASS isn’t giving the video files a proper path. I’m still trying to track down the issue further but maybe this will give you something to go off of.
Edit:
OK! I think I got this part fixed
In the ASS py:
### Add files into Plex database ########################################################################
def add_episode_into_plex(media, file, root, path, show, season=1, ep=1, title="", year=None, ep2="", rx="", tvdb_mapping={}, unknown_series_length=False, offset_season=0, offset_episode=0, mappingList={}):
# Mapping List
ep_orig = "s{}e{}{}".format(season, ep, "" if not ep2 or ep==ep2 else "-{}".format(ep2))
ep_orig_single = "s{}e{}".format (season, ep)
ep_orig_padded = "s{:>02d}e{:>03d}{}".format(int(season), int(ep), " " if not ep2 or ep==ep2 else "-{:>03d}".format(int(ep2)))
if ep_orig_single in mappingList:
multi_ep = 0 if ep_orig == ep_orig_single else ep2-ep
season, ep = mappingList[ep_orig_single][1:].split("e")
if '-' in ep or '+' in ep: ep, ep2 = ep.split("+"); ep2 = int(ep2) if ep2 and ep2.isdigit() else None
season, ep, ep2 = int(season), int(ep), int(ep)+multi_ep if multi_ep else ep2
elif 's%d' % season in mappingList and int(mappingList['s%d' % season][0])<=ep and ep<=int(mappingList['s%d' % season][1]): ep, season = ep + int (mappingList['s%d' % season][2]), int(mappingList['s%d' % season][3])
elif season > 0: season, ep, ep2 = season+offset_season if offset_season >= 0 else 0, ep+offset_episode, ep2+offset_episode if ep2 else None
if title==title.lower() or title==title.upper() and title.count(" ")>0: title = title.title() # capitalise if all caps or all lowercase and one space at least
if ep==0: season, ep, ep2 = 0, 1, 1 # s01e00 and S00e00 => s00e01
if not ep2 or ep > ep2: ep2 = ep # make ep2 same as ep for loop and tests
if tvdb_mapping and season > 0 :
max_ep_num, season_buffer = max(tvdb_mapping.keys()), 0 if unknown_series_length else 1
if ep in tvdb_mapping: season, ep = tvdb_mapping[ep ]
elif ep > max_ep_num and season == 1: season = tvdb_mapping[max_ep_num][0]+season_buffer
if ep2 in tvdb_mapping: season, ep2 = tvdb_mapping[ep2]
elif ep2 > max_ep_num and season == 1: season = tvdb_mapping[max_ep_num][0]+season_buffer
ep_final = "s%de%d" % (season, ep)
filename=os.path.basename(file)
for epn in range(ep, ep2+1):
if len(show) == 0: Log.warning("show: '%s', s%02de%03d-%03d, file: '%s' has show empty, report logs to dev ASAP" % (show, season, ep, ep2, file))
else:
tv_show, tv_show.display_offset = Media.Episode(show, season, epn, title, year), (epn-ep)*100/(ep2-ep+1)
if filename.upper()=="VIDEO_TS.IFO":
for item in os.listdir(os.path.dirname(file)) if os.path.dirname(file) else []:
if item.upper().startswith("VTS_01_") and not item.upper()=="VTS_01_2.VOB": tv_show.parts.append(os.path.join(os.path.dirname(file), item))
### RIGHT HERE
# else: tv_show.parts.append(file)
### SHOULD BE
else: tv_show.parts.append(os.join.path(root, path, file))
media.append(tv_show) # at this level otherwise only one episode per multi-episode is showing despite log below correct
index = str(SERIES_RX.index(rx)) if rx in SERIES_RX else str(ANIDB_RX.index(rx)+len(SERIES_RX)) if rx in ANIDB_RX else "" # rank of the regex used from 0
Log.info('"{show}" s{season:>02d}e{episode:>03d}{range:s}{before} "{regex}" "{title}" "{file}"'.format(show=show, season=season, episode=ep, range=' ' if not ep2 or ep==ep2 else '-{:>03d}'.format(ep2), before=" (Orig: %s)" % ep_orig_padded if ep_orig!=ep_final else "".ljust(20, ' '), regex=index or'__', title = title if clean_string(title).replace('_', '') else "", file=filename))
So, assuming that this will not break anything else:
This line (Line 296 on git):
else: tv_show.parts.append(file)
Should be:
else: tv_show.parts.append(os.join.path(root, path, file))
I don’t know how to use git for crap or I would submit a correction.
Revisiting the “add episode by date” method
Check out the code for Extended Personal Media Scanner around line number 204 and check out how he did the serialization of files. It seems to be pretty effective as long as the upload/air date is known…
Credit goes to @mjarends
Edit:
So thinking about this more,
You can grab the date from “publishedAt” which gives you DATETIME
Take the year and make that your Season
Take the Month/Day and turn it into a day:
Example: 5/13 would be day 133 (Padded 001-365)
then from there you can index each video for that day starting with the oldest
Season 2018 Episode 133xx
xx = 01-99
That would allow for 99 episodes per day before giving a sorting issue. If problems would arise you could always pad it to 001-999
Hopefully this makes sense. And of course this is all just a suggestion! 
Edit: Damn plex sucks at documentation.
Anyways, it looks like a call to get youtube data would have to go in the ASS script, which iirc you mentioned before… 
Edit:
Well, its a really really ugly hack but I got it kind of working, except ASS will only work for youtube right now cause of all the changes.
New bug!
Shows with like names are being grouped into which ever collection is first. Say I have two shows called “GMod” from H2o and Cartoonz… Both GMod subfolders will be grouped into the first “Collection” it finds.
I’ll start using the issue / bug tracking on git unless you would like to keep it all here…
I’ll also try to figure out how to properly use git and help out with some push requests…
Since i mostly use zero size file for tests i cannot play any videos so i didn’t saw the error… Then i fixed it and my scanner code was working locally, but wasn’t pushed on the master branch, as i pasted wrong code then reverted and ended up never pasting the code, now resolved… Sorry about that, we lost a bit of time because of that…
Serious point now, the development direction. Plex allow for custom scanners and agents and i need the scanner to be the best (i get more donations that way…) and the scanner support grouping folders playlists and channels already
Date based suggestion for scanner:
- season=year,
- ep number = 133xx for 13 may for ex.(Padded 001-365) and xx allow for 99 files per day… i don’t like eps missing in between with that method though
We need a list for the index building, so channel or playlist. Considering having a year as season mode (‘youtube2-xxx’) for both channel AND playlist IF there is a use in both cases
playlist: huge ones split per year seem useful, episode matching playlist order., not resetting each season would be both most logical and simplest
- Playlist id: Playlist rank as episode number, season=01 (already supported)
- mode for year as season to add, same ep number
Channel: huge ones split per year seem useful but unsure one would have all videos of a channel or whether we should index using only file present on disk, and if ep number+season can be found on YouTube videos [https://github.com/ZeroQI/Absolute-Series-Scanner/pull/160#issuecomment-396053419] but cannot get ep or season from youtube, just an episode type video.
- not supporting indexing days due to missing eps numbers and super long ep numbers
- year as season mode
- partial channel mode indexing videos on disk per date so no missing eps showing, neater but you won’t know if you are missing vids that way
- full channel mode indexing video per date (first vid of the year is ep 1, etc…) so missing videos will be missing in Plex, not sure it is a concern for youtube videos, will probably skip this for now unless you could argument a case use for it
So my though process above leads to [youtube2-xxxxx] forced id for playlist/channels:
- year as season
- playlist rank or for channels rank of file on disk based on matching youtube “publishedAt” date.
Does that fit your view of date based media scanning? i want to keep media without renaming, just the series folder if possible
for grouping folders, Plex work on search() giving unique ids to series/movies and update() downloading meta for those ids Did you assign channels OR playlist ids to series folder? If not, you may have an option to de-group series So you did folders (from root) like Gmod/H2O and Gmod/Cartoonz it’s working as designed if H2ODelirious/Ben and Ed | CaRtOoNz/Ben and Ed then it’s a bug i will resolve
I struggle with classes in python so rewrote to be simpler (to me), it helped finding the ep number is just date based and ep range possible:
vid = Media.Episode(showTitle, seasonNumber, episodeNumber, episodeTitle, seasonYear)
vid.parts.append(file)
mediaList.append(vid)
def setValues(self, match):
self.showTitle = self.scrub(match.group('showTitle').strip())
self.showTitle = self.scrub(match.group('showTitle').strip())
self.seasonNumber = int(match.group('seasonNumber').strip()) if 'seasonNumber' in match.groupdict() else int(match.group('episodeYear' ).strip()) if 'episodeYear' in match.groupdict() else '1900'
self.episodeYear = int(match.group('episodeYear' ).strip()) if 'episodeYear' in match.groupdict() else int(match.group('seasonNumber').strip()) if 'seasonNumber' in match.groupdict() else '01'
self.episodeReleaseDate = datetime.datetime(self.episodeYear, int(match.group('episodeMonth').strip()), int(match.group('episodeDay').strip()))
episodeIndex = int(match.group('episodeIndex').strip()) if 'episodeIndex' in match.groupdict() and match.group('episodeIndex') is not None else 1
self.episodeNumber = int(str(self.episodeReleaseDate.timetuple().tm_yday) + str(format(episodeIndex, '02'))) # compute the episode index using the episode's day of year and index values
self.episodeTitle = self.scrub(self.stripPart(match.group('episodeTitle').strip())) # get the episode title
log('setValues', 'episode contains index %s', episodeIndex)
log('setValues', 'episode date: %s', str(self.episodeReleaseDate))
log('setValues', 'episode number %s', self.episodeNumber)
I know what you mean, I am the same way.
“<root>/H2O/Gmod” and “<root>/Cartoonz/Gmod” is the structure i’m using. Will be moving this and future issues over to git as its a bit more organized for this type of stuff. This particular issue can be seen here Collection Groupings · Issue #3 · ZeroQI/YouTube-Agent.bundle · GitHub
Any chance of making it so the agent and scanner could work with a simple directory layout such as:
/Cody’s Lab [UCu6mSoMNzHQiBIOCkHUa2Aw]/24K Pure Gold Foil Ball [bt2BDCwu18U].ext
Where it just assigns the channel name as the TV show and all videos in the folder just all get put in one season.
There really isn’t anything great out there for watching archived youtube channels on plex but that would make it work perfectly. Could literally just sort by newest first and hide seasons and it would act just like it shows up under the video tab on youtube.
Playlists do look way nicer but not every channel actively tags all their videos correctly.
Check out the latest update… It doesn’t do exactly what you request but I think the agent is getting better at organizing the content. Also keep an eye on the github page as we have moved most of the dev discussion over there. There was quite a bit of discussion on that previous link about playlists, season/episode numbers/ sorting by date, etc.
I have been using this plugin and it does achieve what I was looking for.
I still would like to replicate this with my local media files. Also is there a gitter page for further conversation?
@selkie707
One workaround for this is to manually add videos to a playlist you have created. My goal is to organize by uploader (series), then playlist 1 (season 1), playlist 2 (season 2), etc. You then have control over playlist videos and the content you want. The plugin I posted does this directly from youtube, but I still want to accomplish this locally with the work being done here.
Here is what the agent support already:
- Playlist [PLxxxxxxx]/file [xxxxxxx].ext
- Uploader (also called channel) [UCxxxxxxx]/Folder x/file [xxxxxxx].ext
- Uploader (also called channel) [UCxxxxxxx]|Subject/Playlist [PLxxxxxxx]/file [xxxxxxx].ext
- ex: /Cody’s Lab [UCu6mSoMNzHQiBIOCkHUa2Aw]/24K Pure Gold Foil Ball [bt2BDCwu18U].mp4
The uploader/Channel will become a collection, The playlist a series, The files episodes.
@polayer2017 How is that structure collection not serving the purpose you are looking for: [UCxxxx]/series/file [video_id].ext ?
@ZeroQI
I started from scratch today with ASS and the youtube plugin bundle and everything works as advertised! Thanks for your hard work. Is there anyway to pipe in the uploaders picture from the youtube channel so it can be used as the cover of the series via the scan script? I know you mentioned local assets could be used or the possibility of creating a database on the github page. Any thoughts?
@polayer2017 it is possible to do but multiple folders from the same channel would get the same image. i know the poster look bad but YouTube give 16:9 ratio and poster is 1:2. i am using the image for roles already.
Working on posters from the first video and channel name and/or folder name
1 Like
Hi Thanks for this
I Have a question
I have downloaded a youtube channel for my boy and use this to scan for the metadata and it pulls everything it should, however it displays in plex as movies, so all the thumbnails it grabs are in poster format and dont look right, anyway for me to fix this?
Thanks
@popeye2468 Sadly Plex doesn’t allow options and poster format (1:2) while youtube gives only horizontal orientation (16:9 or 4:3) so it is horrendously cropped.
I tried for a bit to edit pictures but could not find a workable image library but am self-taught python so am hoping one of the users can import a workable python image library (PIL or PILLOW) to get me started. There is a bug report for that: https://github.com/ZeroQI/YouTube-Agent.bundle/issues/11
Love what you’re doing with the agent.
What would be your recommended youtube-dl.exe config settings for your agent so I can correctly group these under their playlist groups?
like correct folder structure and everything for your agent.
The readme give examples https://github.com/ZeroQI/YouTube-Agent.bundle
- Funhaus [UCboMX_UNgaPBsUOIgasn3-Q]/…
- Dude Soup Podcast [PLbIc1971kgPDJjVdQcLZZz4Exygpo2cuB]/…
- Funhaus [UCboMX_UNgaPBsUOIgasn3-Q]/Dude Soup Podcast [PLbIc1971kgPDJjVdQcLZZz4Exygpo2cuB]/… You can use grouping folders (channel name) if using my scanner (ASS)
I just give the playlist link to youtube-dl and no need to rename files
Hmm ok I’ll try that thanks man.
