DVR Post Process Transcoding MPEG2/TS to h264/MKV w/ NVENC Acceleration

Server Version#: 1.19.2.2737
Player Version#: 4.32.2

The Plex (Pass) DVR allows for recording of OTA broadcasts via various tuner devices. In my experience (in NA), these devices simply capture the ATSC MPEG2 stream and drop it on the disk in a TS file format (some tuner devices may support their own real-time transcoding – not mine). MPEG2/TS is a pretty fat format and takes up a good amount of disk space (11 Mbps for some ATSC channels).

Goal: Upon recording completion, automatically convert the MPEG2/TS file(s) to a more efficient format (h264/MKV in this case) and remove the original TS file(s).

Plex’s built-in “optimize” feature does half the job. It will convert files once they are added to the library. But it is only additive. It does not have an option for deleting the original.

POSTPROCESS in the DVR settings allows for a user-provided script to run as the last step prior to the file(s) being added to the library. This seems like a good place to do some work.

Solution: Utilize the same command that “optimize” uses to convert the captured MPEG2/TS to an h264/MKV. As part of the POSTPROCESS script, we can then remove the original after transcode is complete.

NVENC acceleration note: The script below uses NVENC (nvidia) acceleration – just like “optimize” does in a supported environment. In my configuration (GTX 970, Intel Core i7 930), FFMPEG encodes at 500+ fps via NVENC (on the GPU) vs. 70+ fps without acceleration (on the CPU). Any quality offset due to h264_nvenc is worth the CPU time saved (opinion).

Script:

Limitations: No additional software may be installed on the Plex server (docker container in this case). FFPROBE is not available.

(I’m no BASH expert. I welcome any advice for making this script more robust.)

#!/bin/bash

# These PATHS are in the Plex Transcoder process environment when it runs for "Optimization"
# The FFMPEG library path looks like it could change between releases
#    so we determine it based on the mpeg2video library location.
export FFMPEG_EXTERNAL_LIBS="$(find ~/Library/Application\ Support/Plex\ Media\ Server/Codecs/ -name "libmpeg2video_decoder.so" -printf "%h\n")/"
export LD_LIBRARY_PATH="/usr/lib/plexmediaserver:/usr/lib/plexmediaserver/lib/"

# Grab some dimension and framerate info so we can set bitrates
HEIGHT="$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Stream #0:0" | perl -lane 'print $1 if /, \d{3,}x(\d{3,})/')"
FPS="$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Stream #0:0" | perl -lane 'print $1 if /, (\d+(.\d+)*) fps/')"

# Default (max) bitrate vlaues. Assuming 1080i30 (ATSC max) has same needs as 720p60
ABR="4M"
MBR="8M"

# Use awk to do some number comparison and set some flags
PROFILE=$(echo - | awk "{if ($HEIGHT < 720) {print \"LOW\"} else if ($HEIGHT < 1080) {print \"MEDIUM\"}}")
SPEED=$(echo - | awk "{if ($FPS < 59) {print \"LOW\"} else {print \"HIGH\"}}")

if [ "$PROFILE" == "LOW" ];
then
        # This is a pretty deep ATSC sub-channel - not much bitrate
		ABR="1M"
        MBR="2M"
elif [ "$PROFILE" == "MEDIUM" ];
then
        if [ "$SPEED" == "LOW" ];
        then
                # This may be 720@30
				ABR="2M";
                MBR="4M";
        fi
else
        # This may be 720@60
		ABR="4M"
        MBR="8M"
fi

# Plex Transcoder is based on FFMPEG
# Use NVDEC/NVENC support (built into Plex Pass) to transcode the captured MPEG2/TS
# Output an MKV (h264 w/ original audio)
/usr/lib/plexmediaserver/Plex\ Transcoder -y -hwaccel nvdec -i "${1}" \
-c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -b:v "$ABR" -maxrate:v "$MBR" -profile:v high -bf:v 3 \
-c:a copy "${1%.*}.mkv"

# Grab the duration of the original and transcoded files
SRC_DUR=$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Duration"| cut -d ' ' -f 4 | sed s/,// | sed 's@\..*@@g' | awk '{ split($1, A, ":"); split(A[3], B, "."); print 3600*A[1] + 60*A[2] + B[1] }')
DEST_DUR=$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1%.*}.mkv" 2>&1 | grep "Duration"| cut -d ' ' -f 4 | sed s/,// | sed 's@\..*@@g' | awk '{ split($1, A, ":"); split(A[3], B, "."); print 3600*A[1] + 60*A[2] + B[1] }')

# Compare the durations to make sure we got a full transcode (>98% original duration)
CHECK=$(echo - | awk "{if ($DEST_DUR > ($SRC_DUR * 0.98)) print $DEST_DUR}")

# Clean-up the in the event of success or failure
if [ "$CHECK" != "" -a "$DEST_DUR" != "" -a  "$DEST_DUR" == "$CHECK" ];
then
        echo INFO: Transcode complete. Removing source file.
        rm "${1}"
else
        echo ERROR: Transcode complete. Source and destination are not the same length. Preserving source.
        rm "${1%.*}.mkv"
fi

To find a “Plex Transcoder” command line for your environment simply start an “optimize” process in the Plex UI and look at the process running on your server w/ “ps -ef | grep Trans”. Plex creates a very specific command based on the source media. I’ve greatly simplified it above to be general purpose.

I hope this helps somebody looking to save on storage space when recording w/ their Plex DVR.

Docker Addendum: An approach like the one above can be extra useful when Plex is running as a docker container. It can be difficult to introduce custom transcoding executables into such an environment and persist them across Plex upgrades. In general, you don’t want to touch the docker image contents. A single script (like the one above) can easily be mounted into the container as a volume (I mount to /opt/scripts). From Plex UI, I can call the script from “/opt/scripts/myscript.sh”.

1 Like

Cleaned it up a bit, removed some awk in favor of some perl. Come to think of it, this would be better implemented as a Perl or Python script. Live and learn.

#!/bin/bash

# These PATHS are in the Plex Transcoder process environment when it runs for "Optimization"
# The FFMPEG library path looks like it could change between releases
#    so we determine it based on the mpeg2video library location.
export FFMPEG_EXTERNAL_LIBS="$(find ~/Library/Application\ Support/Plex\ Media\ Server/Codecs/ -name "libmpeg2video_decoder.so" -printf "%h\n")/"
export LD_LIBRARY_PATH="/usr/lib/plexmediaserver:/usr/lib/plexmediaserver/lib/"

# Grab some dimension and framerate info so we can set bitrates
HEIGHT="$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Stream #0:0" | perl -lane 'print $1 if /, \d{3,}x(\d{3,})/')"
FPS="$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Stream #0:0" | perl -lane 'print $1 if /, (\d+(.\d+)*) fps/')"

# Bitrate vlaues (Mb). Assuming 1080i30 (ATSC max) has same needs as 720p60
# Assuming 60 fps needs 2x bitrate than 30 fps
MULTIPLIER=$(echo - | perl -lane "if (${FPS} < 59) {print 1.0} else {print 2.0}")
ABR=$(echo - | perl -lane "if (${HEIGHT} < 720) {print 1*${MULTIPLIER}} elsif (${HEIGHT} < 1080) {print 2*${MULTIPLIER}} else {print 4*${MULTIPLIER}}")
MBR=$(echo - | perl -lane "print ${ABR} * 1.5")
BUF=$(echo - | perl -lane "print ${MBR} * 2.0")

# Plex Transcoder is based on FFMPEG
# Use NVDEC/NVENC support (built into Plex Pass) to transcode the captured MPEG2/TS
# Output an MKV (h264 w/ original audio)
/usr/lib/plexmediaserver/Plex\ Transcoder -y -hide_banner -hwaccel nvdec -i "${1}" \
-c:v h264_nvenc -b:v "${ABR}M" -maxrate:v "${MBR}M" -profile:v high -bf:v 3 \
-bufsize:v "${BUF}M" -preset:v hq -forced-idr:v 1 \
-c:a copy "${1%.*}.mkv"

# Grab the duration of the original and transcoded files (ms)
SRC_DUR=$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1}" 2>&1 | grep "Duration" | perl -lane 'print ($4*10 + $3*100 + $2*60*1000 + $1*3600*1000) if /(\d+):(\d+):(\d+).(\d+)/')
DEST_DUR=$(/usr/lib/plexmediaserver/Plex\ Transcoder -i "${1%.*}.mkv" 2>&1 | grep "Duration" | perl -lane 'print ($4*10 + $3*100 + $2*60*1000 + $1*3600*1000) if /(\d+):(\d+):(\d+).(\d+)/')

# Compare the durations to make sure we got a full transcode (>98% original duration)
CHECK=$(echo - | awk "{if ($DEST_DUR > ($SRC_DUR * 0.98)) print $DEST_DUR}")

# Clean-up the in the event of success or failure
if [ "$CHECK" != "" -a "$DEST_DUR" != "" -a  "$DEST_DUR" == "$CHECK" ];
then
        echo INFO: Transcode complete. Removing source file.
        rm "${1}"
else
        echo ERROR: Transcode complete. Source and destination are not the same length. Preserving source.
        rm "${1%.*}.mkv"
fi
2 Likes

Post processing began failing for scheduled recordings as soon as I added two new tuners. Not even getting error output from the script. Will update if I find a fix.

I must have restarted the Docker container and it upgraded Plex. My script location was no longer valid.

Shout out to another script that does transcoding to MKV:

Ah yeah, I noticed that update. I had updated the readme, but yeah a little annoying that they decided to move things on us. Not sure if that means they are going to work on fixing anything.

With some inspiration from @nebhead, I overhauled this post processing script and put it under SCM control here: GitHub - cedarrapidsboy/pms-postprocessing: Post processing scripts to augment the Plex Media Server DVR functionality.

transcode-internal.sh

Converts the captured file (tested w/ MPEG2/AC3) to a lower bit-rate H264/AAC file. The script uses a default bitrate calculation based on the frame size and frame rate. It also encodes the source audio stream into a stereo AAC stream with a DPLII mix (Dolby ProLogic compatible device required for multi-channel sound). NVDEC/NVENC hardware acceleration will be used if supported by the system (if nVidia HW transcoding works in Plex, it will probably work here).

This script uses the PMS built-in Plex Transcoder FFMPEG build. It does not require the installation of any additional video encoders.

1 Like

I am going to give it a try later today. I have some tuners than hand off h.264, and some that do not. Any thoughts of a way to prevent the script from running on files that are already h.264 encoded?

That would be a good feature. Also, pretty straight forward. The command that checks the video dimensions should also be able to determine the source video codec… checking for mpeg2video. I’ll take a quick look.

can this be used to do commercial skip (via chapters) instead of hard removal (plex comskip) ?

Added a quick check for “mpeg2video” as the source video format. I tested against a recorded .TS file and one of my own .mp4 files. Does what it says on the box! I am now using that version… so if it fails, I’ll know pretty soon.

Use the environment variable: ONLYMPEG2=true (see README.md)

Added the new version to the repo.

1 Like

This mechanism (postprocessing) can be used to do any number of things to the video, including comskip. But, my implementation does not do any of that and I haven’t looked into what Plex is doing for commercial skipping.

On the topic of comskip, I haven’t spent much energy on it because all of the methods have their own strengths and weaknesses. So, I feel if you want to do it well, you have to commit to really dialing-it-in to your own specific needs.

1 Like

Your script is totally awesome. Thanks for adding the MPEG2 functionality, this functions exactly how I needed it to, which prevents by Plex server from getting crushed by software transcoding. Hopefully one of these days Plex’s DVR transcode will be able to leverage NVENC.

1 Like

Thanks for the kind words. WRT NVENC and DVR transcode, I’m aware of a couple of places where Plex (for me) effortlessly uses NVDEC/NVENC (HW decode/encode) to process video:

  • Live transcoding from server library for device playback
  • Library optimization (creating additional versions of videos)

However, one “transcode” option doesn;t appear to do anything:

  • I have one tuner, an old PCIe card, that has an option for transcoding during recording. As near as I can tell, though, it’s GNDN. It too just throws down an MPEG2 stream.

I’ve learned a few things about transcoding and Plex that have surprised me (but make sense):

  1. If you use subtitles, and have not burned them into the video ahead of time, Plex may do a transcode to burn them in before they get to your device (this happens when streaming to Chrome),
  2. Transcoding audio cannot happen on your GPU. So CPU utilization may be high during a GPU transcode because Plex is converting the audio to AAC – you can actually see that in the current version of this script.
  3. Video filtering may not happen in the GPU, either. I could not find a way to do yadif deinterlacing in the GPU hardware, so it happens in SW – this causes higher CPU utilization during the postprocessing transcode.

I may consider adding back in an option to copy the audio stream (instead of convert) and to not deinterlace. If it doesn’t do these two things, the 1440x720 throughput on my GTX970 is >600 FPS (>1000 FPS for 704x480)!

Have fun and feel free to branch!

I have noticed errors that occur about 5% of the time.

Logs show unable to copy the transcoded file.
May 15 20:29:55 Plex01 Plex Media Server[1542]: Source video codec is not MPEG2 and ONLYMPEG2 is defined.
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - Dimensions: 704 x 480
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - Framerate: 29.97
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - De-interlace filter: “-vf yadif=0:1:0”
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - Calculated bitrate: 729
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - Max bitrate (2x): 1458
May 15 20:30:08 Plex01 Plex Media Server[1542]: INFO - Encoding buffer (3x): 2187
May 15 20:30:08 Plex01 Plex Media Server[1542]: Input #0, mpegts, from '/media/PlexMedia1/TVShows/.grab/cb4c5f91b9fc40c194824db62d6697e6d00c871f-c51305b7fd58fd3c0df24350139664739866cda3/America Says - S02E35 - Mabel’s Family vs. Bartenders.mp4": No such file or directory
May 15 20:30:08 Plex01 Plex Media Server[1542]: Duration: 00:29:57.67, start: 1.400000, bitrate: 2774 kb/s
May 15 20:30:08 Plex01 Plex Media Server[1542]: Program 1
May 15 20:30:08 Plex01 Plex Media Server[1542]: Metadata:
May 15 20:30:08 Plex01 Plex Media Server[1542]: service_name : Service01
May 15 20:30:08 Plex01 Plex Media Server[1542]: service_provider: FFmpeg
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, smpte170m, bottom first), 704x480 [SAR 40:33 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, $
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:10x101: Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), s16p, 384 kb/s
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:20x102: Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, mono, s16p, 96 kb/s
May 15 20:30:08 Plex01 Plex Media Server[1542]: Only ‘-vf yadif=0:1:0’ read, ignoring remaining -vf options: Use ‘,’ to separate filters
May 15 20:30:08 Plex01 Plex Media Server[1542]: Only ‘-af aresample=matrix_encoding=dplii’ read, ignoring remaining -af options: Use ‘,’ to separate filters
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream mapping:
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_nvenc))
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
May 15 20:30:08 Plex01 Plex Media Server[1542]: Press [q] to stop, [?] for help
May 15 20:30:08 Plex01 Plex Media Server[1542]: [aac @ 0x1cc0840] Using a PCE to encode channel layout “5.1(side)”
May 15 20:30:08 Plex01 Plex Media Server[1542]: Output #0, mp4, to ‘/media/PlexMedia1/transcode/transcode.0dFnRGMf.mp4’:
May 15 20:30:08 Plex01 Plex Media Server[1542]: Metadata:
May 15 20:30:08 Plex01 Plex Media Server[1542]: encoder : Lavf58.27.104
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:0: Video: h264 (h264_nvenc) (High) (avc1 / 0x31637661), yuv420p, 704x480 [SAR 40:33 DAR 16:9], q=-1–1, 729 kb/s, 29.97 fps, 30k tbn, 29.97 tbc
May 15 20:30:08 Plex01 Plex Media Server[1542]: Metadata:
May 15 20:30:08 Plex01 Plex Media Server[1542]: encoder : Lavc58.52.100 h264_nvenc
May 15 20:30:08 Plex01 Plex Media Server[1542]: Side data:
May 15 20:30:08 Plex01 Plex Media Server[1542]: cpb: bitrate max/min/avg: 1458000/0/729000 buffer size: 2187000 vbv_delay: -1
May 15 20:30:08 Plex01 Plex Media Server[1542]: Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1(side), fltp, 192 kb/s
May 15 20:30:08 Plex01 Plex Media Server[1542]: Metadata:
May 15 20:30:08 Plex01 Plex Media Server[1542]: encoder : Lavc58.52.100 aac
May 15 20:32:51 Plex01 Plex Media Server[1542]: [35.6K blob data]
May 15 20:32:52 Plex01 Plex Media Server[1542]: frame=53875 fps=328 q=24.0 Lsize= 209030kB time=00:29:57.61 bitrate= 952.6kbits/s dup=507 drop=0 speed=10.9x
May 15 20:32:52 Plex01 Plex Media Server[1542]: video:165167kB audio:42139kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.831437%
May 15 20:32:52 Plex01 Plex Media Server[1542]: [aac @ 0x1cc0840] Qavg: 702.713

May 15 20:32:53 Plex01 Plex Media Server[1542]: mv: cannot move ‘/media/PlexMedia1/transcode/transcode.0dFnRGMf.mp4’ to “/media/PlexMedia1/TVShows/.grab/cb4c5f91b9fc40c194824db62d6697e6d00c871f-c51305b7fd58fd3c0df2435013966473986cda3/America Says - S02E35 - Mabel’s Family vs. Bartender.mp4”: No such file or directory
May 15 20:32:53 Plex01 Plex Media Server[1542]: 20200515-203253 ERROR: 1 : Error moving temporary file. Source preserved.

@TeknoJunky - Regarding the commercial skip chapters, please see my newest release - v2. It optionally runs a comskip pass and puts the chapter marks in the video. They do show up in the Plex Web player as chapters. However, I can’t see a way to use the chapters in my Roku Plex player. VNC and mpv both recognize the chapters in the mkv and mp4 container formats.

@heimdm - that one has me perplexed. From the lines above, it looks like you are possibly recording MP4 files with mpeg2video content. This is compliant with mp4, but surprising. Either way, though, the script shouldn’t care about the source file extension. I’ll take a closer look. In the meantime… can you find your Plex Media Server log for this time frame (and post it at as a new issue at the github site)? Also, I’ve refactored the script. See v2 - it will behave differently (but perhaps won’t change your issue).

1 Like

I pulled down the latest update, and switched to mkv, and will post to github, if failures show up. :). Thx again.

On the hardware/software discussion, it appears the comskip that comes with Plex does not include the option for hardware. It does include --hwassist, but not the ability to for nvenc/nvdec,

Agreed. I looked into this. It’s possible there is a branch of comskip that has NVDEC support, but that is not the one packed with PMS.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.