Concerns regarding Cayars' FFMPEG script

Dev Community Dev/API Corner
1

I use SABNZBD+ as my downloader, Sonarr for handling my TV Shows, CouchPotato for Movies, and PMS for streaming these remotely. I’m using all these in a Windows 10 box.

So my goal is to use Cayars’ script to convert all my media to MP4 for best compatibility with PMS and the Plex clients. I followed Cayars’ instructions in this post and they were pretty clear cut. I have some questions though:

1.) In the instructions, it says “In all cases the final MP4 file has the MOOV ATOM at the beginning of the file (ie web optimized)” but when I open autoprocess.ini I see that “relocate_moov = False”. Shouldn’t this be set to true?

2.) I understand that the subtitles will be stripped out and make SRT sidecar files out of them. What is the main purpose of doing this? Is there an advantage to using SRT sidecar files as opposed to embedded ones?

3.) Sonarr doesn’t process/move these SRT sidecar files to the completed TV show folders. Wouldn’t it be a waste of time letting the script extract the SRT files then or is there any other way of processing these SRT files so that they are moved to the completed TV show folders accordingly?

4.) I know Cayars’ script is based off of this: https://github.com/mdhiggins/sickbeard_mp4_automator . That original script has the option of using sub-scripts (postSonarr.py) that are specifically designed for the media managers (Sonarr, CouchPotato, etc.) so that would eliminate the need to use the Drone folder and letting Sonarr do everything automatically in its end. I just don’t know how to properly set it up using that though. Would you rather use postSonarr.py or stick to just using the Drone folder?

5.) There are some parameters in the autoprocess.ini file that are missing. One example is the parameter “thread” which the github link above shows as an available option. Does this mean Cayars’ version of the script is just outdated and that it doesn’t use those new parameters yet?

6.) Can Cayars’ script be run in multiple instances for batch processing when the succeeding file download completes while there is still an ongoing conversion? Does it intelligently handle this kind of situation? If so, wouldn’t there be a problem where multiple instances of ffmpeg running at the same time make things come to a complete crawl on the system?

Thanks for the help.

2
  1. I modified the original scripts to move the ATOM a different way. This will be different if you use the new beta version.
  2. Personal preference for a couple of reasons. First most of the time you won’t need the SRT files so there is no point streaming a “bigger” file that has them embedded. Second, many older devices choke with subtitles embedded in MP4 files.
    3 to 5: The new version keeps all the original functionality of the automator since many people are now using 3rd party d/l managers.
  3. Before I answer this, why would you want to do this and are you running it on a different computer than the Plex server?

Carlo

3

1.) So are you saying that if I use the files that you have in your FTP server, the ATOM will still be moved to the beginning of the file as expected?

2.) If the subtitles are stripped out and then PMS processes the media files, would it automatically get subtitle files again if the PMS configuration is at its default? Please bear with me as I’m very new with PMS.

I prefer watching everything with subtitles so I’m not sure what is the best way to go about it. Would streaming be faster if the subtitles are read from an SRT file compared to them being embedded? If yes, do you know of a way so that Sonarr can process the SRT files together with their corresponding media files?

3 to 5.) What do you exactly mean by your answer? Have you tried using postSonarr.py to achieve the same goal?

6.) I actually don’t want to do anything. I’m just if it will run in multiple instances or not because I’m not sure which between SAB, Sonarr, and CouchPotato should run the run.bat file to convert the files. If I let SAB do the post-processing with your script and it calls the script while a previous call was already attempted, what happens? I just want to understand what will happen behind the scenes.

4

I’m not sure if it’s fine to bump a thread in these forums but mods please let me know if it’s not allowed.

5
  1. In Sabnzbd you can define if the download should pause when post-processing.
    Then you don’t have to worry about over lapping processes.
6

I don’t want the download to pause when the post-processing script starts. I want it to still continue download but only start to initiate the post-processing script when the first one finishes. Or I’m not sure if the script is already intelligent enough to do this.

7

1 yes
2 the script creates the SRT files along with the MP4. Just keep them in the same plex directory and plex will use them.
3 From my earlier testing with all the different plex clients there was far less issues having SRT as standalone files then being embedded.
5. This is a modified version of mp4 automator and it doesn’t use the full subset of things the original did because of the changes I wanted to quality, meta handling and other things.
6. Yes you can run multiple instances if they are in different directories. You can’t run multiple in the same directory.

8

2.) Well, the thing is that my setup would be something like this: from the destination folder of the script, Sonarr will pickup the files and process them accordingly and move them to another destination folder that Plex monitors. The moving of the file by Sonarr will only process the mp4 file itself and NOT the created SRT file. That’s my dilemma here.

Do you think it would be better if I just setup the script to strip out the subtitles from the video file and not create an SRT file so that when moved to the monitor folder of Plex it will be the one to just get a proper SRT file from the known sources? If so, how do you setup the script to make it that way?

6.) What happens then if you let SAB initiate the script every after TV Show/Movie download and it comes to a point wherein the second initiation of the script is ran while the first conversion process is not yet done?

Additional questions:

7.) How exactly is your setup in Windows using the script? Which 3rd party DL manager do you use?

8.) You also said that the script will clear all tags in the mp4 output file. Which tags are these? When you say tags, what comes to mind are the file information (metadata) like codec used, duration, bit rate, etc. but I don’t think those were ever removed in the output file when I tested the script.

9.) I was told that you’re somehow updating the script to improve it further. How’s your progress going regarding this? Will there be any chance that you will also upload the updated script in the public (non-Plex Pass) forum?

9

All of these questions you are asking except maybe the SAB integration are in the cayars thread. That is the place to ask questions about the scripts since there are others who are quite familiar with it and will help you.

10

You mean the one in the Plex Pass forums? So I need to pay to have answer to these questions? :frowning:

11

The link in my sig. yes it’s in the plexpass forum area. :frowning:

I don’t use the programs you do so I can’t help you with the setup of them. I thought by posting in my thread there would be others who use the script and the d/l managers who could help you.

12

No, you shouldn’t need to get a Plex Pass. As @cayars was already helping you I stepped away from this discussion. Now I suppose it’s time to step in.

I have a copy of the run.bat (but edited for Linux paths and an sh file) in the SABnzb+ scripts folder. This uses the same paths as the file that’s called from the cron job, so it reads the autoprocess.ini from the main convert folder. (Don’t need to put another copy of this in the SABnzb+ scripts folder) Everything Sonarr sends SABnzb+ is flagged with a TV Shows category and I have the script set up to run automatically on all TV Shows.

As I told you on another forum, the problem I had was if one show finished downloading, and called the script, it would start conversions. If another show finished downloading, it would then call the script and another conversion would start. At one point I had 16 different ffmpeg instances running at one time, and none of them were getting much done, as they all kind of walked all over each other. I had to put in an error checking routine to stop the script from starting if it was already running.

I’m not sure how to do this in Windows, but this is how I did it in Linux:

#!/bin/sh
SERVICE='ffmpeg'

if ps ax | grep -v grep | grep $SERVICE > /dev/null
then
    echo "$SERVICE service running, everything is fine"
else
    echo "$SERVICE is not running"
    /volume1/misc/Convert/manual.py -a -i "/volume1/Download/SABnzbd/complete"
fi

exit 0

This looks for ffmpeg running, and if it is, exits out. If it’s not running, it starts against any media it finds in the SABnzbd/complete folder and all sub folders. The autoprocess.ini file points the output to the Sonarr “nzbdrone” folder and Sonarr looks there for files periodically. Sonarr’s “Completed Download Handling” switch is disabled, so it doesn’t try to move a file until it has been converted. It always only takes the converted files from the drone folder and only when the conversion is completed, since the conversion locks the file from other processes. Plex always has the right type of media for Direct Play to most of my client devices after this conversion, and automatically adds the media as Sonarr moves the files… :slight_smile:

Perhaps @cayars can look at the shell script above and work something into a batch file for windows, but as I don’t use Plex for windows, I’ve never looked into it.

The tags mentioned that get cleared are the metadata that is embedded in the file. These scripts strip out any embedded metadata, so there isn’t any problem with naming conventions, etc. As these files are converted before Sonarr renames/moves them, they are converted with the downloaded file names. So trying to embed those names into the file would just give you problems at some point…

HTH

13

@MikeG6.5

Thanks Mike. I totally understand how cayar’s script is implemented in your setup as we seem to use both SAB and Sonarr for TV Shows. One thing though: what happened to the FFMPEG instances when you encountered that 16-instance issue? Did they just eventually stop and nothing was converted?

I would think the error checking cron job can be implemented with an equivalent Powershell and schedule running the ps1 script using Task Scheduler in Windows.

Does the stripped metadata include the media information like codec, container, bitrate, etc.?

@cayars

The 16-FFMPEG instance explained by Mike above is exactly what I was asking you about. I guess he explained it better. I just want to know in-depth knowledge about the script and how to modify it.

I also want it to strip out the subtitle but NOT make a sidecar SRT file. I’m not sure which param in the autoprocess.ini to modify as it seems there’s none.

14

@kevindd992002 said:
@MikeG6.5

One thing though: what happened to the FFMPEG instances when you encountered that 16-instance issue? Did they just eventually stop and nothing was converted?

I would think the error checking cron job can be implemented with an equivalent Powershell and schedule running the ps1 script using Task Scheduler in Windows.

OK, did a bit of research on this, to see if I couldn’t come up with a solution for you. Give this powershell script a test. Not sure If this will work 100% as I haven’t tested it. It should work for you, though:

$ffmpegrunning = (Get-Process 'ffmpeg' -ea SilentlyContinue)
if ( $ffmpegrunning -eq ""){
echo "$ffmpegrunning service running, everything is fine"
exit
}
else
{
C:/Convert/manual.py -a -i c:/Convert/Process
}

@cayars may have some input into it, as well, as he likely knows a lot more about powershell scripting than I do. I’m not sure if you can call the script from SAB. Probably not, so you’re going to need to set this with Task Scheduler to some time frame to check if the files are present at some interval. I do NOT promise this will work, so if not, you’re going to need to do the leg work to find out why not.

I also want it to strip out the subtitle but NOT make a sidecar SRT file. I’m not sure which param in the autoprocess.ini to modify as it seems there’s none.

I think you can just remove any languages in the autoprocess.ini file for the line that reads: _subtitle-language = _ Again, @cayars would have to verify it, or you can test it yourself.

15

Ok, I’ll test these out and will report back.

I’m still wondering though: what happened to the FFMPEG instances when you encountered that 16-instance issue? Did they just eventually stop and nothing was converted? Or did it corrupt anything?

16

I rebooted the NAS, so those files it was working on were redone once I got the scripts working so it never called ffmpeg more than once at a time.

It didn’t corrupt anything, as it rewrites the old file when the job kicks off the second time for that source file.

17

How often does your cron job run to make sure that ffmpeg won’t run more than once? Wouldn’t there still be a time where the script is initiated by the post-processing of SAB more than once before the cron job runs making ffmpeg run more than once again?

If the source file is within a subfolder inside the convert sorce folder, is there a way to delete that subfolder after the source file is converted and transferred to the Drone folder?

18

As I told you on the other forum, the cron for TV shows runs every 15 minutes. The one for movies runs every 20 minutes. As they are dumped into different folders, and I do my file structure differently, I have 2 sets of scripts that run, based on the type of media.

Both of these sets of scripts incorporate the check to ensure that ffmpeg isn’t running, so it never runs more than one instance. And the set of scripts that run during an SAB conversion also include the ffmpeg check, it never calls another instance either. (Not sure how you would do this within SAB, the example I gave you probably won’t work with that. I don’t know if a powershell script can be called by SAB’s post processing system.)

As far as the folders, I manually go in and delete them about once a week or so. I have to do the same thing with other folders, so this is a weekly I do for other apps too.

There are some things working with these scripts you are just going to have to work out on your own, to see how well they are going to work for you, and to build your own workflow. I suggest you try them for a while, see how things go, and then come back and ask questions after you have found out a few things. There are things that each of us that use them do differently.

For instance I know one guy that doesn’t build the folder structures I do for movies, so he has CouchPotato search the same “drone” folder that Sonarr uses, too. He calls the same script via cron for both TV Shows and Movies, since he doesn’t need to seed, etc. He doesn’t use Sonarr scripting to process the files SAB downloads, as his cron job handles everything for him.

19

I get what you’re saying now. I originally thought that the script you made is different from the original bat/sh file that cayar created, hence my confusion. I just realized that you replace cayar’s script with the error-checking script you layed out. I have a general view of what’s happening now and I’m off to testing my setup with it. Thanks.

Do you also have Plex configured to automatically update library? I’m just thinking that if I do that it could lead to “software locking” issues during the exact time when Sonarr or CouchPotato is processing the file.

EDIT: Oh and by the way, aren’t the actions in the if and else statements that you have above interchanged? I was thinking that if ffmpegrunning is blank (""), then it should run the script, else it just exits.

20

I’ve modified the error-checking script and this is what worked for Windows:

$ffmpegrunning = (Get-Process 'ffmpeg' -ea SilentlyContinue) if (!$ffmpegrunning) { Write-Host "The FFMPEG service is not running" Set-Location "C:\Convert" .\manual.py -a -i E:\raw est } else { Write-Host "FFMPEG service running, everything is fine" }