Linux commercial detection

General Discussions Plex Features
41

@bjs1400 said:
Have you tried manually calling comcut with some test video file? What I did for testing was copy some video file from my library to some temp directory (not scanned by Plex) and run it through comcut.

eg:

/path/to/comcut /path/to/testfile.mkv

I’m assuming comcut is erroring out on you. Running it manually should clue you in on what it is.

It (and your script) work correctly when I call it from the command line.

[root@plexmediaserver_1 /comskip]# ./comtemp.sh “/media/dvr/Star Trek_ Enterprise (2001)/Season 3/Star Trek_ Enterprise (2001) - S03E18 - Azati Prime.ts”
Comskip 0.81.092, made using ffmpeg
Donator build
Mpeg: /media/dvr/Star
Exe Z:\comskip\comskip.exe
Logo: /media/dvr/Star.logo.txt
Ini: /comskip/comskip.ini

Detection Methods to be used:

  1. Black Frame
  2. Logo - Give up after 2000 seconds
  3. Resolution Change
  4. Aspect Ratio

ComSkip throttles back from -0001 to -0001.
The time is now 0803 so it’s full speed ahead!

comskip.exe since I’m using wine.

42

comskip.exe since I’m using wine.

It may well be a problem with wine. I’ll try to compile comskip for FreeBSD and report back.

43

You may want to check differences in your environment variables between when you manually run the script vs when it gets ran by plex.

In my trials I found that when ran by Plex, Plex sets the LD_LIBRARY_PATH environment variable to /opt/plexmediaserver/something which messes up ffmpeg. You may have something similar going on. I would guess it might be some environment variable that Wine uses.

To determine this I had Plex call a post processing script which just emailed me the output of env so I could compare it to when I’m manually running the script. It’s kind of a backwards way of debugging, but it worked.

44

comskip.exe since I’m using wine.

Why in the world are you using comskip for windows under wine on a linux system?

45

@tvinhas said:

comskip.exe since I’m using wine.

Why in the world are you using comskip for windows under wine on a linux system?

I’m using FreeBSD, not a linux distro, and have been having difficulty compiling comskip because of missing dependencies and the compiler FreeBSD defaults to. I’m working through those issues now.

46

Why don’t you set up a linux box? I don’t understand why would you go thru so much trouble just to use BSD.

47

@tvinhas said:
Why don’t you set up a linux box? I don’t understand why would you go thru so much trouble just to use BSD.

With that logic, why don’t we all just buy TiVo’s? :stuck_out_tongue:

I’m sure he has his reasons for using FreeBSD.

48

@tvinhas said:
Why don’t you set up a linux box? I don’t understand why would you go thru so much trouble just to use BSD.

[Sorry to hijack the thread.]

Well, I already have Plex running on a BSD file server and think it’s awesome. If I can figure this part out, so much the better.

In the meantime, with help, I have found the workarounds on how to compile comskip in BSD: Crash on FreeBSD - Comskip Support Forum

The script now works from the command line, but doesn’t remove commercials when Plex runs it. From watching the files, it appears that the EDL file isn’t created - I wonder if it’s a permissions thing where comskip can’t write the .EDL file?

49

You actually have to specify an ini file to use with comskip which specifies to output to edl file.

50

More specifically, you need to put output_edl=true in an ini file and tell comskip to use that ini file. Why it isn’t a command line argument you can specify directly is beyond me. Oh well.

51

@bjs1400 said:
More specifically, you need to put output_edl=true in an ini file and tell comskip to use that ini file. Why it isn’t a command line argument you can specify directly is beyond me. Oh well.

Sorry, I wasn’t clear - the .edl file is created when I run from the command line, but not when I run from Plex. That could also suggest a different comskip.ini file is being called, but I don’t think that’s the problem, as I expressly called the full path.

52

@bjs1400 said:
I’ve added a new script to the github repo called comcut to be used for commercial removal.

It too should be able to be used directly as a post processing script for Plex to do commercial detection and removal.

GitHub - BrettSheleski/comchap: Commercial detection script to add chapters into video file

comcut:
comchap/comcut at master · BrettSheleski/comchap · GitHub

I’m trying to run it here, tried two recordings at the same time. One copied only the .ts from the .grab to the show’s directory. The other only copied the .edl. Do you have any idea why?

Funny thing is that it took a few minutes after the script ended to copy these files from .grab. Both shows were marked as Error on the UI, saying something was wrong with the date.

53

@Bollar said:

@bjs1400 said:
More specifically, you need to put output_edl=true in an ini file and tell comskip to use that ini file. Why it isn’t a command line argument you can specify directly is beyond me. Oh well.

Sorry, I wasn’t clear - the .edl file is created when I run from the command line, but not when I run from Plex. That could also suggest a different comskip.ini file is being called, but I don’t think that’s the problem, as I expressly called the full path.

Sounds like a permissions issue to me. In your manual testing are you su-ing as the plex user?

Both comcut and comchap scripts look for a .comskip.ini file in the current user’s home director, creating one if necessary.

54

I’m doing some other testing to be sure.

Another thing is, I have the generated .ts file with the .edl in the same directory but I couldn’t see any chapters available on Plex for Android TV. Did I miss something?

55

@tvinhas said:
I’m doing some other testing to be sure.

Another thing is, I have the generated .ts file with the .edl in the same directory but I couldn’t see any chapters available on Plex for Android TV. Did I miss something?

Make sure you grabbed the latest version of comcut. I had a bug in there where it was putting the wrong timestamps for commercials attempting to make chapters at negative timestamps. That should be fixed now.

I’m not familiar with the Android TV. For testing I just bring up the Plex web app on a laptop and press the [ and ] keys.

56

I think I figured it out. The environment path didn’t include ffmpeg in the default directory /usr/local/bin/ - I added the complete path and it appears to be working. If it does, I’ll document how I did it.

Thanks much for your script! I wouldn’t have been able to do it with out it. I forked the FreeBSD specific changes here: https://github.com/rbollar/comchap/blob/master/comcut

I also added a way to force deletion of all of Comskip’s work files. I think that will address something I saw where Plex grabbed a random work file instead of the .ts

57

Awesome. I’m glad you figured it out and pleased that my little script was of use to you.

58

Hello,
I used this to get comcut installed.

cd /var/tmp
mkdir ComSkipWork
wget http://www.xilka.com/xilka/source/comskip-0.93i.tar.xz
tar xf comskip-0.93i.tar.xz
cd comskip-0.93i
./configure --prefix=/usr
make -j8
sudo make install

Is this the latest version of comcut? if not where do I get it from? I can not find a linux source.

I ask because I used your script on a hdhomerun dvr file, that Plex saved as a mpeg2 file in mkv.
I have run the script twice and both times it cut the 30 minute show to 17 and dropped the last 3rd of the show.
The first commercial break it did not remove all the commercials, but remove about 2 minutes of the 4 mins of commercials.

SCREEN SHOWED THIS WHEN i RAN IT.
Input #0, ffmetadata, from ‘bonetest.ffmeta’:
Duration: 00:13:10.00, start: 0.000000, bitrate: 0 kb/s
Chapter #0:0: start -30.000000, end 355.000000
Chapter #0:1: start 265.000000, end 570.000000
Chapter #0:2: start 501.000000, end 790.000000
Input #1, mpegts, from ‘concat:./chapter-1-rZwm3asU.ts|./chapter-2-VlnsV8e8.ts|./chapter-3-BdGKEkR3.ts’:
Duration: 00:04:49.06, start: 1.400000, bitrate: 31899 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #1:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, max. 25000 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #1:1[0x101]: Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 192 kb/s
[matroska @ 0x2091680] Codec for stream 0 does not use global headers but container format requires global headers
[matroska @ 0x2091680] Codec for stream 1 does not use global headers but container format requires global headers
Output #0, matroska, to ‘bonetestcom.mkv’:
Metadata:
encoder : Lavf56.40.101
Chapter #0:0: start 0.000000, end 355.000000
Chapter #0:1: start 265.000000, end 570.000000
Chapter #0:2: start 501.000000, end 790.000000
Stream #0:0: Video: mpeg2video (mpg2 / 0x3267706D), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, max. 25000 kb/s, 29.97 fps, 29.97 tbr, 1k tbn, 90k tbc
Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, stereo, 192 kb/s
Stream mapping:
Stream #1:0 -> #0:0 (copy)
Stream #1:1 -> #0:1 (copy)
frame=29272 fps=1665 q=-1.0 Lsize= 1040581kB time=00:16:19.68 bitrate=8701.2kbits/s
video:1017151kB audio:22965kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.044708%

Any idea were to troubleshoot?
What is the script using to detect commercials?

thanks in advance
Haeffnkr

59

Comskip is the program that actually does the commercial detection. It looks like you were able to compile and install it correctly.

Comchap and Comcut are two scripts I developed.

Comchap is a script that uses Comskip to to do the commercial detection writing the output to an EDL file. The EDL file is then processed to generate a metadata file ffmpeg can use to modify the video file and just mark chapters for both commercials and non-commercials. This allows for easy skipping of commercial breaks detected by comskip without actually removing any content from the file.

Comcut actually removes the commercials marked in the EDL file as well as marking chapters of the remaining content.

The latest version of both Comchap and Comcut can always be found on my GitHub page:

Personally, I would clone the GitHub repository and add symlinks to the scripts to a directory found in your PATH.

You could do something like the following:

cd /var/lib
git clone https://github.com/BrettSheleski/comchap.git
ln -s /var/lib/comchap/comchap /usr/bin/comchap
ln -s /var/lib/comchap/comcut /usr/bin/comcut

Doing so will keep a directory (/var/lib/comchap) that will hold a clone of the GitHub repo. It could always be updated to the latest (assuming you didnt make any of your own changes) by doing the following

cd /var/lib/comchap
git pull

With all that said, it looks like you got everything working correctly, but it seems like Comskip had a problem with correctly detecting commercials. I’m not much of an expert on configuring Comskip, unfortunately. I just wrote some helper scripts.

Both scripts are written such that if an EDL file already exists, it does not call Comskip. This way you could manually create your own EDL file if you wanted. It shouldn’t be tough. Tedious, but not tough.

60

I am trying it on a h264/mp4 file now and seeing if that makes a difference.

I see this in the log file - Could not find the logo file.

On the screen I see this -
[h264 @ 0x20ca7c0] gray decoding requested but not enabled at configuration time
[h264 @ 0x20c73c0] gray decoding requested but not enabled at configuration time

Are either of these an issue?

thanks Haeffnkr