I’ve read through whole thread, but I can’t figure out what I’m doing wrong. I run comchap manually (to test). It runs all the way though the file, says it finds commercials and then ends with this:
Commercials were found. /usr/bin/comchap: line 126: mytvfile.edl: No such file or directory No Commercials found, and not saving to new location. Nothing to do.
I’m manually running the command as the plex user.
Any thoughts?
Looks like for whatever reason the .edl file is not being created. My guess is it’s not creating the .comskip.ini file with output_edl=1 in it and thus it does not write to the .edl file.
Try specifying the comskip.ini file manually.
first try:
/usr/bin/comchap mytvfile.mkv --comskip-ini=/path/to/empty/file
if that don’t work then try:
echo "output_edl=1" >> /path/to/comskip.ini
/usr/bin/comchap mytvfile.mkc --comskip-ini=/path/to/comskip.ini
I’m suspecting a bug was introduced when updating the script to recognize the --comskip-ini argument. I’m guessing I haven’t noticed it since I already have the “output_edl=1” in ~/.comskip.ini.
@Bollar, I added a --work-dir=/path/to/work/dir command line argument to comcut just for you. This will pass the --output argument to comskip as well as write all temp files to the specified path. I’ve done some limited testing and things seem ok. Give it a shot.
@bjs1400 said:
@Bollar, I added a--work-dir=/path/to/work/dircommand line argument to comcut just for you. This will pass the --output argument to comskip as well as write all temp files to the specified path. I’ve done some limited testing and things seem ok. Give it a shot.
Cool, but not quite: With the path “–work-dir=/tmp”, I get:
/comchap/comcut: line 126: /tmp//media/The Real Housewives of Orange County (2006) - S11E16 - Bringing Up Old Ghosts.ffmeta: No such file or directory
The files are created in the right directory, but not deleted.
Ah. I know what the problem is. I’ll get to it as soon as I can find a laptop.
Here’s a fancier version of my script that logs each processed file and how long it takes to complete the scan.
#!/bin/csh
# Add bash path to Plex path.
set path = ($path /usr/local/bin)
# Sleep for a pseudorandom period (up to 10 seconds) to limit the number of instances that start at once
# $$ = process % = modulo
sleep `echo $$%900 | bc`
set now = `date`
set starttime = `date +%s`
echo "Processing: $now $1" >> /tmp/comchap.log
# Everything betwen here and the next # needs to be on one line! Make sure there's no linewrap.
/comchap/comcut --ffmpeg=/usr/local/bin/ffmpeg --comskip=/comskiplinux/Comskip-master/comskip --lockfile=/tmp/comchap.lock --comskip-ini=/comskiplinux/Comskip-master/comskip.ini "$1"
#
set endtime = `date +%s`
@ timediff = $endtime - $starttime
echo "Complete: $timediff seconds $1" >> /tmp/comchap.log
To be safe and avoid any permissions issues, I’d create /tmp/comchap.log myself.
touch /tmp/comchap.log
chown plex /tmp/comchap.logSo @bjs1400 , I’m using comcut and having the output file go as .mkv. But I’m not receiving this error:
[matroska @ 0x555ae2eedec0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[matroska @ 0x555ae2eedec0] Can’t write packet with unknown timestamp
av_interleaved_write_frame(): Invalid argument
Do you know what it could be?
Also, do you know what format do I have to use to prevent Android TV (Nvidia Shield) from having Plex to transcode the files?
Thanks!
Using …ts as the outfile it works fine, but I see a lot of these messages during the conversion:
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127020, current: 40123266; changing to 40127021. This may result in incorrect timestamps in the output file.
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127021, current: 40124017; changing to 40127022. This may result in incorrect timestamps in the output file.
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127022, current: 40124767; changing to 40127023. This may result in incorrect timestamps in the output file.
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127023, current: 40125517; changing to 40127024. This may result in incorrect timestamps in the output file.
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127024, current: 40126267; changing to 40127025. This may result in incorrect timestamps in the output file.
[mpegts @ 0x5555accf12c0] Non-monotonous DTS in output stream 0:0; previous: 40127025, current: 40127017; changing to 40127026. This may result in incorrect timestamps in the output file.
The show I’m trying to process is Designated Survivor, from ABC. Recorded from OTA HDRH.
Any thoughts?
@bjs1400 : Thanks for the useful bash script. I’ve been reading/understanding it the last couple of days and was trying it out for the first time today.
I ran into a bit of a perplexing question I couldn’t figure out. The .ffmeta file looks correct and is being read in by ffmpeg correctly from what I can determine from the ffmpeg output when it inserts the chapter information. However, when I read the chapter information with ffprobe after the file is created it is all messed up. Excerpts below, any ideas?
Correct Chapter Markers during ffmpeg Chapter Insertion:
Input #1, ffmetadata, from 'file.ffmeta': Duration: 00:30:59.00, start: 0.000000, bitrate: 0 kb/s Chapter #1:0: start 0.000000, end 0.000000 Chapter #1:1: start 0.000000, end 8.000000 Metadata: title : Commercial Chapter #1:2: start 8.000000, end 83.000000 Chapter #1:3: start 83.000000, end 243.000000 Metadata: title : Commercial Chapter #1:4: start 243.000000, end 781.000000 Chapter #1:5: start 781.000000, end 1062.000000 Metadata: title : Commercial Chapter #1:6: start 1062.000000, end 1514.000000 Chapter #1:7: start 1514.000000, end 1653.000000 Metadata: title : Commercial Chapter #1:8: start 1653.000000, end 1843.000000 Chapter #1:9: start 1843.000000, end 1859.000000 Metadata: title : Commercial Chapter #1:10: start 1859.000000, end 1859.000000
What I get when I look at the file with ffprobe:
Chapter #0:0: start 0.000000, end 83.000000 Metadata: title : Commercial Chapter #0:1: start 83.000000, end 781.000000 Metadata: title : Commercial Chapter #0:2: start 781.000000, end 1514.000000 Metadata: title : Commercial Chapter #0:3: start 1514.000000, end 1843.000000 Metadata: title : Commercial Chapter #0:4: start 1843.000000, end 1859.000000 Metadata: title : Commercial Chapter #0:5: start 781.000000, end 1062.000000 Metadata: title : Commercial Chapter #0:6: start 1062.000000, end 1514.000000 Metadata: title : Chapter #0:7: start 1514.000000, end 1653.000000 Metadata: title : Commercial Chapter #0:8: start 1653.000000, end 1843.000000 Metadata: title : Chapter #0:9: start 1843.000000, end 1859.000000 Metadata: title : Commercial Chapter #0:10: start 1859.000000, end 1860.125000 Metadata: title :
I think I may have found the issue.
Here’s the original metafile created by the script (note the lack of end time for the last chapter):
;FFMETADATA1 [CHAPTER] TIMEBASE=1/1 START=0 END=0.00 [CHAPTER] TIMEBASE=1/1 START=0.00 END=8.88 title=Commercial [CHAPTER] TIMEBASE=1/1 START=8.88 END=83.78 [CHAPTER] TIMEBASE=1/1 START=83.78 END=243.74 title=Commercial [CHAPTER] TIMEBASE=1/1 START=243.74 END=781.15 [CHAPTER] TIMEBASE=1/1 START=781.15 END=1062.13 title=Commercial [CHAPTER] TIMEBASE=1/1 START=1062.13 END=1514.78 [CHAPTER] TIMEBASE=1/1 START=1514.78 END=1653.89 title=Commercial [CHAPTER] TIMEBASE=1/1 START=1653.89 END=1843.04 [CHAPTER] TIMEBASE=1/1 START=1843.04 END=1859.86 title=Commercial [CHAPTER] TIMEBASE=1/1 START=1859.86
Here’s a modified version I created where I cleaned up first and last chapter:
;FFMETADATA1 [CHAPTER] TIMEBASE=1/1 START=0 END=8.88 title=Commercial1 [CHAPTER] TIMEBASE=1/1 START=8.88 END=83.78 title=Noncommercial1 [CHAPTER] TIMEBASE=1/1 START=83.78 END=243.74 title=Commercial2 [CHAPTER] TIMEBASE=1/1 START=243.74 END=781.15 title=Noncommercial2 [CHAPTER] TIMEBASE=1/1 START=781.15 END=1062.13 title=Commercial3 [CHAPTER] TIMEBASE=1/1 START=1062.13 END=1514.78 title=Noncommercial3 [CHAPTER] TIMEBASE=1/1 START=1514.78 END=1653.89 title=Commercial4 [CHAPTER] TIMEBASE=1/1 START=1653.89 END=1843.04 title=Noncommercial4 [CHAPTER] TIMEBASE=1/1 START=1843.04 END=1859.86 title=Commercial5
Maybe the edl file created by comskip was slightly different than what your script expected:
0.00 8.88 0 83.78 243.74 0 781.15 1062.13 0 1514.78 1653.89 0 1843.04 1859.86 0
I think the problem lies in the last chapter not having an end time. The only other item of substance I changed was eliminate the first chapter of zero length.
Using my modified version it works correctly.
I can add simple logic to not add any chapters which have a duration of zero. No big deal there.
When I originally wrote the script it made a call to ffprobe to determine the total duration of the video for writing the final chapter end time. However I decided to remove that for some reason or another as I found that it was unnecessary (at least in my testing). I think I removed it as I didn’t like having to make a call to ffprobe and didn’t want to make another command line argument to specify the path to ffprobe in cases like @Bollar’s where the path to ffmpeg (and presumably ffprobe’s as well) needs to be specified.
If the total duration can be determined using other means (such as using ffmpeg alone) I’m open to adding it back in.
Thanks for your (and anyone else reading this) interest in comchap and comskip. I really didn’t expect it to become much of any interest to anyone except for myself.
I’d like to move issue tracking to Github (https://github.com/BrettSheleski/comchap/issues). So if anyone finds any issues, please report them there.
It’s very useful and appropriately timed with the new DVR functionality being added. At first I was intending to simply read it and understand it to write my own script. But as I finished understanding what you had written I decided it was well written and essentially did everything I would have done.
The only essential feature missing (which I’ll need to add if you aren’t game to add it) is some type of quiet switch. I run this post-processing in a cron job at night so if it runs successfully I’ll need it to be completely silent. Only if it runs into problems should it print out anything which by default gets e-mailed to me to resolve.
It was late last night when I was looking at this and I actually made three changes to your scripts meta file, but I only mentioned two. I did more testing and found that it was actually the change I didn’t mention which actually made the difference.
1st Change: I erased the zero length first chapter. Mostly cosmetic in its effect.
2nd Change: I erased the last chapter without an end time. I thought this was the issue but I was incorrect.
3rd Change: I added a title statement to each of the chapters (script only named the chapters). In my more detailed testing, this ended up being the actual problem.
I’m curious why this wasn’t seen by anyone else but me. It seems to mangle the chapters quite a bit if you don’t name each of them. I can only think of two reasons, either our ffmpeg versions are slightly different, or I ran into a particular case you haven’t seen or it’s particularly noticeable in my case.
My ffmpeg version was compiled from their code in October, so fairly current. My guess is that you have this issue as well, but it wasn’t noticed because of the particular cases you have been using.
I’ll upload this to the git site as well. Thanks again for your work on the script.
I have noticed the 3 changes you mentioned and came to the same conclusion that the chapter naming is what is causing ffmpeg to garble the chapters. I just committed a change to resolve this. My Plex box is currently down (I think the power supply took a dump on me) so I’m not really able to test. It’s a simple change and shouldn’t be a big deal.
As to why nobody else is coming across this issue: who knows. Maybe they are, but just dont realize it. Also, I think more people are using comcut than comchap, so the importance of proper chapters just isn’t very high for others.
For use in your cron job I would suggest having your script look at the exit status of comchap to determine whether or not to email you. I did some effort in returning a non-zero exit code on error, but it is not fully implemented.
Otherwise I could add something like a --quiet command line argument to prevent echoing anything to stdout. Then you could just redirect stderr to your email.
Thanks for making the change for the chapter title. Don’t worry about the quiet item. I’ll brainstorm something myself.
This is something I need to get back to. My trouble at the moment is the installation or update of ffmpeg and other required apps. For some reason my Ubuntu 16.x server isn’t cooperating. Even using the guides mentioned here I’m not seeing some of the items after compile or install.
Hopefully will get back to this. My trouble will be the amount of shows I’ll need to run this against 
Or, I can wait until I finish next server build / upgrade.
I did want to mention though thanks for still checking the thread and updates!
I just did a fresh install of Ubuntu Server in a new VM. Here’s what I needed to run in order to get Comskip and Comchap/Comcut working:
#update system
sudo apt-get update
sudo apt-get upgrade
#install ffmpeg and git
sudo apt-get install ffmpeg git
#install stuff needed by Comskip
# (this was determined by the output of ./autogen.sh and ./configure lines below for Comskip)
sudo apt-get install build-essential autoconf libtool libargtable2-dev libsdl1.2-dev libavutil-dev libavformat-dev libavcodec-dev
# compile Comskip
cd
git clone https://github.com/erikkaashoek/Comskip.git
cd Comskip
./autogen.sh
./configure
make
# install Comskip
sudo make install
# get Comchap/Comcut
cd
git clone https://github.com/BrettSheleski/comchap.git
# test comchap on a test file (no commercials, just making sure nothing blows up)
cd comchap
wget http://techslides.com/demos/sample-videos/small.mp4
./comchap --verbose small.mp4
I personally like having comchap/comskip executable without needing to prefix the path to execute them. I do so by cloning the repository to /var/lib instead and symlink comchap and comcut to /usr/local/sbin.
# clone the repo to /var/lib/ instead
cd /var/lib
sudo git clone https://github.com/BrettSheleski/comchap.git
sudo ln -s /var/lib/comchap/comchap /usr/local/sbin/
sudo ln -s /var/lib/comchap/comcut /usr/local/sbin/
#now you can execute comchap and comcut without needing to specify the path
# test it out
cd
wget http://techslides.com/demos/sample-videos/small.mp4
comchap --verbose small.mp4
There are perhaps better way of doing this part, but it works for me.
I wrote a functional Bash function using comskip to strip out commercials. It works well, the only flaw being that the commercial detection is quite flaky. I’m more than happy to share my function, but I’m probably going to skip commercial skipping.
My script also does transcoding, subtitle extraction, queries against Sonarr (to update it), and proper file naming.
The one thing that surprises me is the lack of comskip.ini files. Everyone always posts to the comskip forum and I find it lacking. At least it is to say that comskip.ini files are scattered at best.
But on that note, has anyone found a way to use comchap/comcut and specify specific comskip.ini files depending on the station? I have a couple stations that mine works really well with and others it just does a horrible job.
I’m still using comchap for now as cutting would be too unreliable at this point.
@robertclemens
There’s a --comskip-ini=PATH command line argument in both comchap and comskip. Use it to specify your own comskip.ini file. Both comchap and comcut will add the necessary output_edl=1 to the comskip.ini if it is not already in the file.
Example Usage
comchap --comskip-ini=/path/to/my/special/comskip.ini /path/to/my/recording.mkv
I think I see what you were actually asking. You want to have some list of comskip ini files tailored for specific TV stations from which a recording is recorded from. Then have a script automatically select the appropriate comskip.ini and pass it on to comchap/comcut.
I’ve done some tinkering before and may have a possible solution one could implement.
One way I could see doing this is querying the SQLite database that Plex uses. I’ve found that all recordings are stored in a table and stores information such as which channel the recording was recorded from as well as a bunch of other stuff. What’s nice is that this information is pretty easy to query given a “Grab ID.”
You may have noticed that recordings are stored in a .grab directory with a subdirectory with a seemingly random folder name. This folder name is actually the same name as the Grab ID stored in the database. This way you can retrieve the what station the recording was recorded from.
The problem I see is that, in my testing, there is no record in the database for the DVR recording until AFTER post processing is ran. This would mean that there’s no way of running the query to determine the TV Station a recording was done in a post-processing script. I’m unaware of a way of a good way of getting the Grab ID after post processing either.
It may be possible for a post processing script to fork a new process passing the Grab ID with a delay of some time (5 seconds?). This would allow for the post-processing script to complete and have Plex update the database with the required information. From which the script could run and get the needed information to pick the correct comskip.ini file and pass it along to comchap/comskip/whatever.
I’d be interested to see if anyone actually implements something like this.
