If you have not already, we suggest setting your Plex username to something else rather than email which is displayed on your posts in forum. You can change the username at https://app.plex.tv/desktop#!/account
Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

scanner for VDR Recordings

buescherbuescher Posts: 49Members, Plex Pass Plex Pass

Hi all, 

 

i'm trying to create a Scanner for VDR Recordings. Basics are working as below:

# -*- coding: iso-8859-15 -*-
# VDR Recording Scanner
#
import re, os, os.path, sys, datetime
import Media, VideoFiles, unicodedata

def Scan(path, files, mediaList, subdirs):

  if len(files) >= 1:

    # Iterate through all the files
    for file in files:
      print "working on file",file
      if (file.endswith(".ts")):
        dir = os.path.dirname(file)
        infoname = dir + "/info"
        # later, parse dir to get recording time and date
        # dir = re.search('^.*/(.*)\.rec',dir).groups(1)[0]
        # print "dirname:", dir
        infoFile = open(infoname).read()
        title = re.search('^T (.*)$', infoFile,re.M)
        if title:
          title = title.groups(1)[0]
        else:
          title = ""

        print "found title ", title

        print "adding movie", (title)
        movie = Media.Movie(title)
        movie.source = VideoFiles.RetrieveSource(file)
        movie.parts.append(file)
        mediaList.append(movie)

Now my Question is: How do i stack several files ending with .ts into one movie? VDR can be configured to only create files of a certain size. So maybe, one have 10 files in the recording-dir, names would then be 0001.ts, 0002.ts, 0003.ts and so on.

 

Any Tipps on that?

 

regards,

Friedhelm

 

«1

Comments

  • buescherbuescher Posts: 49Members, Plex Pass Plex Pass

    hi, seems like i found out, how:

    # -*- coding: iso-8859-15 -*-
    # VDR Recording Scanner
    #
    import re, os, os.path, sys, datetime
    import Media, VideoFiles, unicodedata, Stack
    
    def Scan(path, files, mediaList, subdirs):
    
      if len(files) >= 1:
    
        # Iterate through all the files
        for file in files:
          if (file.endswith("/info")):
            print "working on file",file
            dir = os.path.dirname(file)
            infoFile = open(file).read()
            title = re.search('^T (.*)$', infoFile,re.M)
            subtitle = re.search('^S (.*)$', infoFile,re.M)
            if title:
              title = title.groups(1)[0]
            else:
              title = ""
    
            if subtitle:
              subtitle = subtitle.groups(1)[0]
            else:
              subtitle = ""
    
            print "found title ", title
            print "found subtitle ", subtitle
    
            print "adding movie", (title)
            movie = Media.Movie(title)
    
            print "getting content of dir "+dir
            for ts_filename in os.listdir(dir):
              if (ts_filename.endswith(".ts")):
                print ts_filename
                if (ts_filename.endswith("00001.ts")):
                  print "first element"
                  movie.source = VideoFiles.RetrieveSource(dir+"/"+ts_filename)
                movie.parts.append(dir+"/"+ts_filename)
            mediaList.append(movie)
    
    

    this scanner works reasonable well. If any of you uses this, please give feedback.

    regards,

    Friedhelm

  • HelmiHelmi Posts: 20Members, Plex Pass Plex Pass

    Hi Friedhelm,

    just wanted to say thanks for you scanner. couln't get it to work so far which is mostly because i couldn't get it activated in PMS (synology based) so far. Already found the directory and created the py file but even after restarting PMS i can't find it anywhere to set it up for my recordings directory.

    I then still need to find out how to copy recordings over from my VDR server.

  • HelmiHelmi Posts: 20Members, Plex Pass Plex Pass

    I finally got it working. Didn't know i had to create subdirectories in scanners/ to attach the scanners to the regarding category of files (Movies, Music,etc.).

    Your scanner works fine and i think it does what it is intended to do. Nonetheless i'd like to have additional information in the meta data. I don't know exactly how plex works differently on tv series and movies but for tv recordings i have lots of series types recordings like regular talk shows or comedy shows. therefore at least the date should be in the subtitle or the title and having them grouped together like episodes on tv series would be great.

    Is there a documentation anyhwere how scanners probably work differently for series and movies and how fields are named for custom scanners? I don't know much python but i think i could play around and get it working if i'd know how plex expects things to be.

    Thanks

  • buescherbuescher Posts: 49Members, Plex Pass Plex Pass

    i did not find any doc on that. I am unhappy with the result of my scanner on series, too, but december is a bad month for hobby .. :)

  • dfodfo Posts: 5Members, Plex Pass Plex Pass

    Hi Friedhelm,

    thank you very much, works nicely!

    I've written up a Plex Agent that scrapes additional information (channel, summary,..) from the /info

    file. I would like to bundle my agent with your scanner and publish it on Github.

    May I ask for permission to publish your code on Github under the BSD license?

    happy holidays
    Dariush

    buescher wrote on November 23 2013, 8:55 PM: »

    hi, seems like i found out, how:

    # -*- coding: iso-8859-15 -*-
    # VDR Recording Scanner
    #
    import re, os, os.path, sys, datetime
    import Media, VideoFiles, unicodedata, Stack
    
    def Scan(path, files, mediaList, subdirs):
    
      if len(files) >= 1:
    
        # Iterate through all the files
        for file in files:
          if (file.endswith("/info")):
            print "working on file",file
            dir = os.path.dirname(file)
            infoFile = open(file).read()
            title = re.search('^T (.*)$', infoFile,re.M)
            subtitle = re.search('^S (.*)$', infoFile,re.M)
            if title:
              title = title.groups(1)[0]
            else:
              title = ""
    
            if subtitle:
              subtitle = subtitle.groups(1)[0]
            else:
              subtitle = ""
    
            print "found title ", title
            print "found subtitle ", subtitle
    
            print "adding movie", (title)
            movie = Media.Movie(title)
    
            print "getting content of dir "+dir
            for ts_filename in os.listdir(dir):
              if (ts_filename.endswith(".ts")):
                print ts_filename
                if (ts_filename.endswith("00001.ts")):
                  print "first element"
                  movie.source = VideoFiles.RetrieveSource(dir+"/"+ts_filename)
                movie.parts.append(dir+"/"+ts_filename)
            mediaList.append(movie)
    
    

    this scanner works reasonable well. If any of you uses this, please give feedback.

    regards,

    Friedhelm

  • buescherbuescher Posts: 49Members, Plex Pass Plex Pass
    dfo wrote on December 24 2013, 5:03 PM: »

    Hi Friedhelm,

    thank you very much, works nicely!

    I've written up a Plex Agent that scrapes additional information (channel, summary,..) from the /info

    file. I would like to bundle my agent with your scanner and publish it on Github.

    May I ask for permission to publish your code on Github under the BSD license?

    happy holidays
    Dariush

    Hey dfo,

    i'd appreciate that!

    regards,

    Friedhelm

  • iPhonedationiPhonedation Posts: 170Members, Plex Pass, TunerTester Plex Pass

    Hi Friedhelm,

    what shall I say ... WOW! Thanks a lot for that nifty script.

    Some smaller issues/thoughts

    • it does not work with my Series folders in VDR ... at least it does not create an overview, though I can crawl through the folder structure to reach the .ts
    • how about incorporating the EPG info ... because I use EPG-Daemon with epgdata and tvm my description is already there and high quality
    • transcoding/syncing - did you play with that?

    However. This is really great, at least for my setup!

    Cheers,

    Alex

  • dfodfo Posts: 5Members, Plex Pass Plex Pass

    Okay i've published the code here:

    https://github.com/forouher/VDR.bundle

    It's a combination of buecher's scanner and an agent for EPG scraping.

    Have fun!

    Dariush

  • iPhonedationiPhonedation Posts: 170Members, Plex Pass, TunerTester Plex Pass

    Hey Dariush,

    thanks a lot ... by accident you do not want to check if you could mod the dreambox bundle to work with streamdev also into this? I am not a python guy and am struggling to find out how I need to change the enigma2 web interface stuff to work with streamdev's http output. 

    However - if you like to look into this and need explanation what in theory is what is required PM me :-) otherwise I will spend tonight and tomorrow night working myself into python a bit for this.

    Because - if we have a channel for live TV and this baby here for recordings I am more than happy.

    Cheers

    Alex

  • dfodfo Posts: 5Members, Plex Pass Plex Pass

    Hi Alex,

    sounds definitely interesting! But I won't be able to do this anytime soon, sorry. I hacked that stuff together at my parents place over Christmas, but I won't dare touch their setup over SSH while being away. ;) And currently I don't have a VDR installation at my place.

    But if you get somewhere, feel free to publish it somewhere. I'd be happy to try it out at some point in time. :)

    cheers

    Dariush

    iPhonedation wrote on December 29 2013, 8:37 PM: »

    Hey Dariush,

    thanks a lot ... by accident you do not want to check if you could mod the dreambox bundle to work with streamdev also into this? I am not a python guy and am struggling to find out how I need to change the enigma2 web interface stuff to work with streamdev's http output. 

    However - if you like to look into this and need explanation what in theory is what is required PM me :-) otherwise I will spend tonight and tomorrow night working myself into python a bit for this.

    Because - if we have a channel for live TV and this baby here for recordings I am more than happy.

    Cheers

    Alex

  • iPhonedationiPhonedation Posts: 170Members, Plex Pass, TunerTester Plex Pass

    Hi,

    I 'made' something that looks promising for a first step to me. Will announce it here under Plugins and in the VDR Portal.

    Cheers

    Alex

  • mhasselberg@web.demhasselberg@web.de Posts: 2Members

    Hi,

    thanks for providing this scanner.

    However for me it happened that under certain load conditions on the nfs server recording parts were dis-ordered in Plex.

    E.G. a movie started somwhere in the middle and then the beginning followed as the next part in playback.

    This seemed to be caused by Python os.listdir(....) not returning an ordered list and then the scanner added a wrong

    order into the data base.

    Please find attached an updated version that solves this problem (at least for me)

    # VDR Recording Scanner
    #
    # Source:
    # https://forums.plexapp.com/index.php/topic/86990-scanner-for-vdr-recordings/
    #
    # Copyright: Friedhelm Buescher 2013
    
    import re, os, os.path, sys, datetime
    import Media, VideoFiles, unicodedata, Stack
    
    def custom_listdir(path):
        """
        Returns the content of a directory by showing directories first
        and then files by ordering the names alphabetically
        """
        dirs = sorted([d for d in os.listdir(path) if os.path.isdir(path + os.path.sep + d)])
        dirs.extend(sorted([f for f in os.listdir(path) if os.path.isfile(path + os.path.sep + f)]))
        return dirs
        
    def Scan(path, files, mediaList, subdirs):
    
      if len(files) >= 1:
    
        # Iterate through all the files
        for file in files:
          if (file.endswith("/info")):
            dir = os.path.dirname(file)
            infoFile = open(file).read()
            title = re.search('^T (.*)$', infoFile,re.M)
            rectime = re.search('^E \d+ (\S*) .*$', infoFile,re.M)
    
            if title:
                title = title.groups(1)[0]
            else:
                title = "Unnamed VDR recording"
    
            if rectime:
                title = datetime.datetime.fromtimestamp(int(rectime.groups(1)[0])).isoformat(" ") + " -- " + title
            else:
                title = "????-??-?? ??:??:?? -- " + title
    
            movie = Media.Movie(title)
    
    #        for ts_filename in os.listdir(dir):
            for ts_filename in custom_listdir(dir):
              if (ts_filename.endswith(".ts")):
                if (ts_filename.endswith("00001.ts")):
                  movie.source = VideoFiles.RetrieveSource(dir+"/"+ts_filename)
                movie.parts.append(dir+"/"+ts_filename)
    
    	if len(movie.parts)>0:
        	    mediaList.append(movie)
    
    
    

    Cheers

    Mike

  • WyseWyse Posts: 2Members Plex Pass
    edited June 2014

    Thank you for providing this Scanner. I use it with my yaVDR Movies directory without any problems.

    Because I have a lot of TV Shows and no plan of writing a scanner in python I made a work around for my tv shows.

    The PMS internal scanner need a structure like

    /Heroes
    /Season 1
    Heroes - s01e01.avi
    Heroes - s01e02.avi
    /Season 2
    Heroes - s02e01.avi
    Heroes - s02e02.mkv

    The VDR stores the recordings in a path like:

    Serien/Abenteuer/Black_Sails/Season_01/01~03_Unheilige_Allianzen/2014-04-27.20.15.71-0.rec/00001.ts

    I have this structure for tv shows since I set up my first VDR years ago. In this path I have all the information I need to create a Plex like path with a single symlink. So I made a pretty basic awk script to create a symlink looking like this

    S01E03_Unheilige_Allianzen.ts -> 01~03_Unheilige_Allianzen/2014-04-27.20.15.71-0.rec/00001.ts

    The PMS Scanner has now a correct Path to scan

    /Black_Sails/Season_01/S01E03_Unheilige_Allianzen.ts

    and get the tv show information from the TVDB. I also made another simple scipt to run as a cronjob. I use my yaVDR headless ubuntu 64 bit system ro run the PMS.

    The cron script:

    #!/bin/sh
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    
    #Use this script with care it maybe delete all your symbolic links or create a hack of symbolic links.
    
    inputFile=/tmp/videofiles
    seriesFolder=/srv/vdr/video.00/Serien/
    awkScript=/usr/local/bin/createSeriesSyms.awk
    
    # Delete all symbolic links - use only to reset the the original VDR structure
    # find /srv/vdr/video.00/Serien/ -type l -exec rm -rf {} \;
    
    # Delete all symbolic dead links in the series folder
    echo "CSS: Deleteing old symbolic links"
    find -L $seriesFolder -type l -exec rm -rf {} \;
    
    # find all 0001.ts files in the series folder and put it into die inputFile
    echo "CSS: Searching for videofiles"
    find $seriesFolder -name 00001.ts > $inputFile
    
    # Call the awk File to create the symbolic links
    echo "CSS: Starting the AWK Script"
    awk -f $awkScript $inputFile
    
    
    

    The awk Script:

    #! /usr/bin/awk -f
    
    # set the file separator for my series folder.
    # Example: /srv/vdr/video.00/Serien/Crime/Banshee:_Small_Town._Big_Secrets./Season_02/02~05_Die_Wahrheit_über_Einhörner/2014-05-15.22.05.38-0.rec/00001.ts
    # so the separator here is the / for the normal file structure and the ~ for my season and episode
    # If you want to use this script you have to specify your own separators.
    
    BEGIN{
    FS="[/|~]*";
    }
    {
            #print FILENAME;
            #f=FILENAME;
    }
    {
            # Because the result of the find command does not have any escaped chars I use gsub to escape special characters.
            # There maybe some characters more. They can be added by a another gsub command
            gsub(/\:/,"\:");
            gsub(/\(/,"\(");
            gsub(/\)/,"\)");
            gsub(/\'/,"\'");
    
            #This combination will only work for MY structure of folders. If you want to use this you have to determine your own structure.
            #The example from the top will look like:
            #$0             /srv/vdr/video.00/Serien/Crime/Banshee:_Small_Town._Big_Secrets./Season_02/02~05_Die_Wahrheit_über_Einhörner/2014-05-15.22.05.38-0.r$
            #$1
            #$2             srv
            #$3             vdr
            #$4             video.00
            #$5             Serien
            #$6             Crime
            #$7             Banshee:_Small_Town._Big_Secrets.
            #$8             Season_02
            #$9             02
            #$10    05_Die_Wahrheit_über_Einhörner
            #$11    2014-05-15.22.05.38-0.rec
            #$12    00001.ts
            #This results in: ln -s 02~05_Die_Wahrheit_über_Einhörner/2014-05-15.22.05.38-0.rec/00001.ts /srv/vdr/video.00/Serien/Crime/Banshee\:_Small_Town._Bi$
            #For Testing you can use the print statement
            #print("ln -s " $9"~"$10"/"$11"/"$12" ""/"$2"/"$3"/"$4"/"$5"/"$6"/"$7"/"$8"/""S"$9"E"$10".ts");
            system("ln -s " $9"~"$10"/"$11"/"$12" ""/"$2"/"$3"/"$4"/"$5"/"$6"/"$7"/"$8"/""S"$9"E"$10".ts");
    
            # With this you can get your own variables
            #for (i=0;i<15;i++)
            #{
            #print ("Field "i" "$i);
            #}
    
    }
    
    
    

    This scripts are only working if you have a single ts file 00001.ts. I had a lot of old recordings with more than one ts file. So I made another simple script to join all ts files to one 00001.ts.

    #!/bin/bash
    find "/srv/vdr/video.00/" -path "/srv/vdr/video.00/Serien/*" -prune -o -name 00002.ts -exec dirname {} \; | sort -u > tsfiles
    
    PFADE=$(cat tsfiles)
    
    for f in $PFADE
    do
      echo  $f
      # add every 0000x.ts file to the 00001.ts file
      cat $f/00002.ts $f/00003.ts $f/00004.ts $f/00005.ts >> $f/00001.ts
      rm $f/00002.ts $f/00003.ts $f/00004.ts $f/00005.ts
    done
    

    So the scripts are bretty basic but I do not have enough time nor the knowlege to make it much better. I also know that thsi is now scanner for VDR TV Shows but if you search for a solution of this problem you can use this way till somebody who is good in python creates a real Scanner. I can say that this script is working without any Problems on my System. Sure you have to adapt it to your own file structure but I think that should not be a problem.

    Ciao

    Wyse

  • thorsten@gehrig.infothorsten@gehrig.info Posts: 24Members, Plex Pass Plex Pass
    edited August 2014

    Hi

    i need some help with installing (my first) scanner plugin.

    I have PMS run on my debian installation - and i have

    /var/lib/plexmediaserver/....

    and 

    /usr/lib/plexmediaserver/...

    I´m not sure where i have to put what - and how to enable.

    I have meanwhile a scanner "vdr-scanner" in ,my drop down - but it does not work.

    The scanner is there since i downloaded this:

    https://forums.plex.tv/topic/86990-scanner-for-vdr-recordings/#entry505335

    and copied it there:

    /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Movies

    I expect the github download is not istalled properly on my device...

    I´ve just re-read the manual - and copied the proper "VDR recordings scanner.py" the the correct location - but i´m still not find the new Agent.

    How to enable the agent?

    Thanks in advance.

    Thorsten

    ===================

    I solved the issue - sometime its really simple.

    The GIT-Directory was called "VDR.bundle.git" - and i have to rename to "VDR.bundle".... somtimes it´s so simple...

  • thorsten@gehrig.infothorsten@gehrig.info Posts: 24Members, Plex Pass Plex Pass
    edited August 2014

    Deleted. Sorry.

  • thorsten@gehrig.infothorsten@gehrig.info Posts: 24Members, Plex Pass Plex Pass

    Hi,

    if i use "VDR Recordings Importer" it detects the movie title - but there are no further information grabbed from the internet.

    If i change to "the movie database" the Movies are not detected correctly.

    So - i would love "the movie database" importer - but with the name detection on "VDR recordings importer"

    Is there any way to do this?

    I expect i need the name with Series... so a "VDR Recordings Importer - Series" to detect the Episode and TheTVDB...

    I did not investigate yet to modify agents on my own... did someone started this - or can tell me if this is the wrong way?

    Kind regards

    Thorsten

  • dfodfo Posts: 5Members, Plex Pass Plex Pass
    thorsten.gehrig wrote on August 7 2014, 10:15 PM: »
    I solved the issue - sometime its really simple.

    The GIT-Directory was called "VDR.bundle.git" - and i have to rename to "VDR.bundle".... somtimes it´s so simple...

    Hi Thorsten,

    glad you figured it out by yourself. Thx for the feedback, I've amended the README to clarify this.

    cheers

    Dariush

  • dfodfo Posts: 5Members, Plex Pass Plex Pass
    thorsten.gehrig wrote on August 11 2014, 4:15 AM: »

    if i use "VDR Recordings Importer" it detects the movie title - but there are no further information grabbed from the internet.

    If i change to "the movie database" the Movies are not detected correctly.

    So - i would love "the movie database" importer - but with the name detection on "VDR recordings importer"

    Is there any way to do this?

    I expect i need the name with Series... so a "VDR Recordings Importer - Series" to detect the Episode and TheTVDB...

    If you use "VDR Recordings Importer" as the movie agent, it should fetch the EPG data recorded by VDR. It won't scrape anything from Internet databases, you're right about that.

    If you want Plex to scrape metadata from Internet Databases instead, you have to use the classic Plex agents for that (in combination with the "VDR Recordings Scanner"). Unfortunately my recordinged movies rarely got any matches. So I've never seriously explored this scenario.

    And I expect TV shows to be even more problematic. In most recordings you will only have the show's title and the name of the episode (in German, to make matters worse). Season and episode number are usually burried in the EPG text (if there at all). Scraping thetvdb based on that will be quite a challenge, I assume.

    However if you manage to amend the Scanner or Agent to work with Internet Scrapers, please give me a note. I'll gladly pull your merge.

    cheers

    Dariush

  • Lutz ThielemannLutz Thielemann Posts: 1Members, Plex Pass
    edited August 2014

    Hi - i am new to plex and start enjoying it. I stumbled across this thread as i was looking for a Scanner for my VDR-recordings.

    The python-script works fairly well - i needed to make some changes to have my "Serien" - folder and my "Konzerte" - folder excluded - inside the Video-Directory there is a flat-file to put patterns to exclude from the scan.

    here is what i did:

    # -*- coding: iso-8859-15 -*-
    # VDR Recording Scanner
    #
    import re, os, os.path, sys, datetime
    import Media, VideoFiles, unicodedata, Stack
    
    
    def Scan(path, files, mediaList, subdirs):
    # Define List of Patters to exclude
      exc = []
    
      with open("/nas/Video/vdr0/.plex/exclude", "r") as f:
        for line in f:
          exc.append(line.rstrip())
    
      if len(files) >= 1:
    
        # Iterate through all the files
        for file in files:
          print "working on file",file
    
          ex = 0
          if len(exc) >= 1:
            print "need to check patterns to exclude"
            for e in exc:
              if re.search(e, file ):
                ex = 1
    
          if (file.endswith("/info") and ex == 0):
            dir = os.path.dirname(file)
            infoFile = open(file).read()
            title = re.search('^T (.*)$', infoFile,re.M)
            subtitle = re.search('^S (.*)$', infoFile,re.M)
            if title:
              title = title.groups(1)[0]
            else:
              title = ""
    
            if subtitle:
              subtitle = subtitle.groups(1)[0]
            else:
              subtitle = ""
    
            print "found title ", title
            print "found subtitle ", subtitle
    
            print "adding movie", (title)
            movie = Media.Movie(title)
     
    

    I have a couple of questions:

    - where do i get the logging info?

    - would plex be able to handle old ".vdr" - Files as well - surely the Scanner would have to be adapted

    - has anyone a proper Scanner for Series yet? where would be a general documentation howto on custom scanner  - parameters and return values?

    If there is no scanner to handle structured series files i tend to create one

  • vento72vento72 Posts: 2Members

    Hi all, i also have a vdr server running vdr 1.7.21 (reelbox ubuntu based 10.04 LTS ) with the latest PMS, i have installed the scripts according your instructions (latest PMS) /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Movies/VDR Recordings Scanner.py and and init_.py to /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-ins/VDR.bundle/Contents. What i also did is to copy the VDR.bundle into /usr/lib/plexmediaserver/Resources/Plug-ins-e0b7243/VDR.bundle. I can connect to my VDR server from PC and Dreambox clone (VU+), and see my TS Vdr recordings if i use the default scanners. It results in many 00001.ts to 00003.ts files.
    If i use the vdr recording scanner and vdr importer, which should solve the problem, it stops scanning after a minute and nothing is found. I unable to find out the reason why it isn't working. Do i need to have a newer VDR / Ubuntu version? Is something in the wrong place?

    Kindly
    Vento

«1
Sign In or Register to comment.