Post-processing for DVR Tutorial

General Discussions Plex Features
1

Sharing what I’m currently doing. Hopefully it’s helpful to those needing to do something similar. I pieced this together from many different sources, so it will be good, I think, to get it all in one location.

My setup:

  • Atom (Braswell) mini-ITX Server running Ubuntu 16.04
  • Clients: Roku 5 and two Windows 10 machines.

In general when a new TV show is recorded I want to accomplish the following:

  1. Extract out the Closed Captions (CC) as a .srt file.

  2. Encode in H.264 (@ 720p).

  3. Mark the commercials as chapters

  4. Hard link duplicate shows watched by both my wife and me.

  5. Closed Captions: I like to have the CC playing normally as sometimes the actors mumble and you can’t quite catch what they are saying. The .ts file has this information in it, but Plex will not play the CC if you play back the .ts file. For a while I tried to have ffmpeg do the extraction, and it does work, to a degree, but the text is mangled in various ways and it was a lot easier and faster processing to go with ccextractor.

ccextractor.org/doku.php

Downloaded it from the site (above) and then used the ‘using_cmake_build.txt’ from the ‘docs’ directory to compile.

  1. Encode in H.264 @720p: For a while I played around with different resolutions (original vs. 720p) different CODECs (H.264 vs. H.265). I really like H.265, but compatibility is somewhat tricky on all devices and it takes quite a while to encode on my Atom server. I ended up doing some testing to help guide my decision:

264:ultrafast: 128 fps / 64mb=>35.8mb 264:superfast: 79 fps / 64mb=>27.2mb 264:veryfast: 60 fps / 64mb=>15.4mb 264:faster: 34 fps / 64mb=>19.0mb 265:ultrafast: 21 fps / 64mb=>6.4mb 265:superfast: 14 fps / 64mb=>8.0mb 265:veryfast: 9.9 fps / 64mb=>9.7mb

I encoded a segment over and over with h.265 and h.264 (above). I found 720p to be a very acceptable resolution and combined with the h.264 ‘veryfast’ setting, I could encode a TV show in about 1 hour. I ended up only using h.265 for my home videos where we are 100% always playing them in VLC and compatibility and encode time aren’t an issue. Using these settings my shows go from 4-5 GB to 500-700 MB or roughly 1/10 in size.

For encoding I use ffmpeg. The below link has a complete guide to compiling on Ubuntu:
trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

I think you can simply install with apt-get if you don’t want h.265, but since I’m using h.265 for home video compression I needed to compile myself.

  1. Marking Commercials: Fast-forwarding commercials with the Roku was the pits. Windows media center and Plex Home Theater did a much better job IMO than the Roku implementation. To make it easier for the Roku, I mark the commercials with comskip/ffmpeg. Comskip runs on the .mp4 and creates a .edl file which is then run through ffmpeg to create chapters at the commercial breaks.

Comskip:
github.com/erikkaashoek/Comskip

I found it’s instructions were almost correct for Ubuntu 16.04, but I had to make some minor alterations (below):
apt-get install -y git build-essential libargtable2-dev libsdl1.2-dev apt-get install -y libavutil-dev libavformat-dev libavcodec-dev cd ~ git clone git://github.com/erikkaashoek/Comskip cd ~/Comskip ./autogen.sh ./configure make

I the comchap script from Brett Sheleski (@BrettSheleski) to do the comskip/ffmpeg legwork.
github.com/BrettSheleski/comchap

Command to download:
git clone http://github.com/BrettSheleski/comchap

The comchap script runs comskip and then reformats the .edl file into a metadata file then loads it into ffmpeg to create the chapters.

  1. Tying it all together and hard links. I created a Perl script to tie this all together and to solve a long-standing issue with viewing/deleting shows both my wife and I watch. Previously for shows we both watch I’d ask her if she had watched a show. She’d think and then ask me what it was about, then after a lot of checking we’d come to an understanding of what she’d watched and I’d manually delete the shows off that we’d both watched. Then it occurred to me that I could use hard links in Linux to solve our challenge. Any show only I watch goes into my TV directory. Same with shows only my wife watches, it goes into her directory. But for shows both of us watch, they are recorded into a separate ‘Shared TV’ directory. When I do my post-processing at night any files in the ‘Shared TV’ directory has two hard links created, one to my directory and one to my wife’s directory. I then delete the original. Because of the magic of hard links we now both appear to have our own copy without any extra disk storage space. She can delete her files anytime she finishes watching it and I can do the same for mine. The physical data is unaffected by either deletion until the final one is deleted at which point it is gone for good.

github.com/Reed97123/encode_movies

I decided not to post-process everything immediately. I didn’t want to tie-up the CPU with intensive processing right when another TV show might be recording or when me or my wife might be trying to stream a program to a client. I wanted the system to be as responsive as possible during the hours when anyone might be using it. Instead I kick-off the post-processing at 11:30pm with:

crontab -e
# Start every night at 11:30pm 30 23 * * * /root/bin/encode_movies

It can then suck up lots of resources without anyone noticing or caring.

2

This is great info…wondering since they have updated to have comskip now if you have any suggestions for a srt extraction and handbrake encode post processing script - i have some experience writing scripts but not for those two

3

you can always skip using comskip by plex and still do it yourself