Auto-Transcoding scripts

server-asustor

#1

*****   WARNING   *****  Setting up these scripts requires extensive use of WinSCP/terminal and/or PuTTY!!!  If you haven't used these apps for accessing your NAS before, you are at a risk of screwing up your system!  If you don't know what you are doing STOP NOW!  You been warned!   I'm not responsible for any damage you may do!    *****   WARNING   *****

 

*****   WARNING   *****  Cayar's scripts require Python 2.7 which is available from Asustor's App Central.  Install this FIRST before trying to execute his scripts.  The manual.py  and several other scripts require editing before you can test them!   *****   WARNING   *****

 

A while back Cayars (Carlo) posted some scripts he uses to do transcoding/remuxxing of MKV to MP4.  As the MP4 H264 with AAC (or AC3) and Web Optimized settings seem to be the easiest to Direct Play, I started messing with his scripts a bit.  It solves having to move files back and forth and using HandBrake to do the task. 

 

I have 2 NASes now. The AS-202T, the smallest and least capable of Asustor's line up.  And the AS-7004T, the opposite end of the spectrum, the real work horse of Asustor's stable.  The differences between the two are truly awe-inspiring.  Even with the CPU in the 7004T, though, I have decided I would continue with remuxxing all of my movies/TV shows to make sure the NAS was able to Direct Play as much as possible.  Bandwidth is still an ongoing problem in Alaska, so getting everything into a good format to begin with is a major win.

 

Cayars' post about his scripts can be located here:  (This is in a Plex Pass only section of the forums, if you don't have a Plex Pass you can't get to the post!)

 https://forums.plex.tv/discussion/comment/931888/#Comment_931888 

As he runs a Windows PMS, all of his scripts are geared toward this.  Paths to ffmpeg and ffprobe, paths for the working directory and the done directory all reflect his local Windows structure.  As such, there was a bit of editing needed to make these truly amazing scripts do their magic on an Asustor NAS.

 

First, you will need to download his scripts.  The zip is linked in his post above.  This zip file also contained the WINDOWS flavor of ffmpeg, so those aren't useful for us.  So we're also going to need to download a working Linux flavor.  (Don't get the one from Asustor's App Central, it's an older version and only designed for some photo manipulations for Asustor's lacking photo apps.)

 

You can get a good working copy of ffmpeg and all the rest of that suite here:

 http://johnvansickle.com/ffmpeg/

 I grabbed the 32-bit static, as I started messing with this on the 202T which is a 32-bit OS.  If you are running on a 60x, 50xx, 51xx or 70xx series you can grab the 64-bit static build and it should work great.  For the 20x and 30x stay with a 32-bit.  I use a bit different folder structure, as Asustor seems to wipe out any of the standard Linux user folders on each reboot, so I put all of my scripts into /volume1/misc. 

 

I unzipped the static build of ffmpeg into a folder named /volume1/misc/FFMpeg.  (I also put the 32-bit in it's own folder, and the 64-bit into it's own, just to keep them straight in my own mind, since I started this on the 202T and now working on a 7004T.)  You can make this folder structure from within the ADM File Manager or Access Control apps.  Refer to Asustor's documentation on making a new share. 

 

Next you need to unzip Cayar's scripts in a folder.  I put them into /volume1/misc/Convert.  Here you will find a few folders, including /process and /done.  The /process is where you are going to put the movie/TV show you want to convert.  When the conversion happens, the new file will be in the /done folder. 

 

Now for the editing on Cayar's scripts.  These are HIS scripts, and I'm not posting the whole mess here, just the sections I have edited.  The first file that needs editing is the manual.py file.  (Same edit needs to happen in all the other .py files in his folder structure.  Not sure how many are actually used, but I did them all anyway.)

 

This edit is just to point to Asustor's strange location for python, so the first line needs to read:

 

#!/usr/local/AppCentral/python/bin/python

Cayar's script pointed to his local install of python, which won't work for your NAS.  Other scripts I modified this way are /converter/_init_.py, /converter/avcodecs.py, /converter/ffmpeg.py, and /converter/formats.py.

 

The next script that needs editing is the auotProcess.ini file.  this is just an initialization file, and contains pointers to ffmpeg, ffprobe and where we put the new files.

 

This is what I edited:  (If you are running a 32-bit OS and have 32-bit FFMpeg installed as I outlined above, change the 64-bit folder name below to 32-bit.)

 

ffmpeg = /volume1/misc/FFMpeg/64-bit/ffmpeg
ffprobe = /volume1/misc/FFMpeg/64-bit/ffprobe
output_directory = /volume1/misc/Convert/done

 

The last bit of editing is the run.bat.  I renamed this run.sh, even though, at this time, I'm not "running" it, but copying a line from it to use in a PuTTY session.  The edits I did are in the last line and should read:

 

/volume1/misc/Convert/manual.py -a -i 
/volume1/misc/Convert/process

 (This should all be on one line, the code function turned it into 2 lines by default and I can't get it looking right for some reason...)

 

To run the script now, just open up PuTTY with root access and copy/paste the line in run.bat (or run.sh) into the PuTTY command line and hit enter.  Any movies/TV shows, even in deeper folders will be converted. 

 

This strips out any embedded subtitles, keeping only the English ones, if you want other languages, refer to Cayar's documentation with his scripts.  It makes an AAC audio track on track 1, and an AC3 track on track 2.  What used to take several hours to do copying files, handbraking them and copying them back again, now takes minutes to perform.

 

Soon, I hope I can get a script to move files from a working folder into the process folder, then call all of these scripts with a cron job.  If I get it working I'll post them as a follow up here.

 

GL all!  And HEED THE WARNINGS!


#2

Link is not working?


#3

The first link should work for all Plex Pass members now. If you don't have a Pass, you can send Cayars a PM to ask him for the link to his files.

They are HIS files, so I'm not posting them without his specific permission. Hope every one understands.


#4

One thing I found while using this script was if it was set up to run on a cron job, and the previous job hadn't finished yet, you could get multiple instances of ffmpeg running, each slowing the system down. None of them would be able to finish, as the files might get deleted or locked by previous instances, etc.

So I did some checking around and came up with a solution for this that didn't require going into the cron files every time to comment out the call.

I created a test-conv.sh in the misc folder. this file contains the following code:

#!/bin/sh
SERVICE='ffmpeg'

if ps ax | grep -v grep | grep $SERVICE > /dev/null
then
    echo "$SERVICE service running, everything is fine"
else
    echo "$SERVICE is not running"
    /volume1/misc/Convert/manual.py -a -i /volume1/misc/Convert/process
fi

exit 0

What it does is look at the running apps and services to see if ffmpeg is already running. If it is, it exits out. If not it starts the conversion script. This SHOULD prevent multiple instances from running together. It doesn't interfere with making thumbnail bif files, nor with transcoding sessions.

You can then set up automatic downloads to the process folder and all of the files will automatically be converted. The next step would be automatically naming and moving to Plex's folder structure, and I'm working on that. I want to also try getting the scripts to clean up the process and done folders, getting rid of extraneous files/folders and just general housekeeping tasks.


#5

Hi,

I found a similar script which seems to be more up-to-date, you can found it here: https://github.com/mdhiggins/sickbeard_mp4_automator

It also provides third party scripts that can be installed with utorrent, nzbget, couch potato... to process the files as soon as they have been downloaded. It might worth a try


#6

Cayar's scripts are based on those, I think. I've had issues with getting any of the scripts from SABnzb+ or CouchPotato working, though. It's probably my config or set up causing the problems.


#7

I tried getting it to work on my 202T but end up with this error:

admin@SERVER:/volume1/home/admin $ /volume1/misc/Convert/manual.py -a -i /volume1/misc/Convert/process
Traceback (most recent call last):
File "/volume1/misc/Convert/manual.py", line 14, in <module>
from mkvtomp4 import MkvtoMp4
File "/volume1/misc/Convert/mkvtomp4.py", line 8, in <module>
from converter import Converter
File "/volume1/misc/Convert/converter/__init__.py", line 7, in <module>
from converter.ffmpeg import FFMpeg, FFMpegError, FFMpegConvertError
File "/volume1/misc/Convert/converter/ffmpeg.py", line 13, in <module>
console_encoding = locale.getdefaultlocale()[1] or 'UTF-8'
File "/usr/local/AppCentral/python/lib/python2.7/locale.py", line 543, in getdefaultlocale
return _parse_localename(localename)
File "/usr/local/AppCentral/python/lib/python2.7/locale.py", line 475, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8


#8

@tgmo said:
I tried getting it to work on my 202T but end up with this error:

admin@SERVER:/volume1/home/admin $ /volume1/misc/Convert/manual.py -a -i /volume1/misc/Convert/process
Traceback (most recent call last):
File "/volume1/misc/Convert/manual.py", line 14, in <module>
from mkvtomp4 import MkvtoMp4
File "/volume1/misc/Convert/mkvtomp4.py", line 8, in <module>
from converter import Converter
File "/volume1/misc/Convert/converter/__init__.py", line 7, in <module>
from converter.ffmpeg import FFMpeg, FFMpegError, FFMpegConvertError
File "/volume1/misc/Convert/converter/ffmpeg.py", line 13, in <module>
console_encoding = locale.getdefaultlocale()[1] or 'UTF-8'
File "/usr/local/AppCentral/python/lib/python2.7/locale.py", line 543, in getdefaultlocale
return _parse_localename(localename)
File "/usr/local/AppCentral/python/lib/python2.7/locale.py", line 475, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

Ok so i think i got it sorted. In ffmpeg.py i changed:
console_encoding = locale.getdefaultlocale()[1] or 'UTF-8'
to:
console_encoding = 'UTF-8'