Hello all,
I’m back with a brand new version of my post-processing script. Since the original thread has been locked for a long time, and this version is a significant upgrade, I’m making a new post.
UPDATE 2022-10-06: I fixed some typos and made the script default to keeping backups of all video files in temporary storage for 7 days or until the next reboot.
FEATURES:
- Full hardware decode and encode using VAAPI for Intel CPUs.
- Works on all video files (previous version was limited to .ts files)
- Keep closed captions as a subtitle track
- Closed captions are cleaned up to remove formatting tags.
- First audio stream copied, the rest ignored. There are settings to change this to AAC or FLAC in the script.
- Video encoding uses QP 25, this is easily changed to whatever number you want. You can also use matching to change QP based on the title or stream height.
- Auto-crop, and only crop if it’s over a certain threshold
- For safety, all original video files are kept in temporary storage for 7 days or until the next reboot, whichever comes first. If you uncomment the entire section, it will only move originals that have been cropped, and it will delete others immediately. CAUTION: If you run this script on an existing file older than 7 days, any backups of the original video file will be deleted on next run as this decision is based on creation date.
- Garbage collection for temporary files - all temp files are stored in /tmp so they are cleared on reboot. Additionally, each time the script runs, it deletes temp files older than 7 days. This means any original video files too.
- Adaptive deinterlace for 1080 and 480 content.
- Forces NTSC frame rate, this is easily changed to PAL or one could omit the variable entirely to use original framerate. This is because I don’t need 60fps content.
- Ability to create encoding profiles based on show name or stream properties.
- Error-checking that will perform cleanup operations and leave original video file intact if any ffmpeg processes fail along the way
- Job queue and per-job logging with lots of helpful information (adapted and expanded from plex-tools/plex.sh at master · HolyRoses/plex-tools · GitHub)
- Can be run on the command line with lots of helpful console output. I linked mine to /usr/local/bin so I can call it from anywhere.
- Multi-user friendly, but make sure every user that needs to use the script is added to “render” group (see below). One single queue is kept for all users’ jobs.
- Well documented internally with major variables exposed for easy customization
REQUIREMENTS:
- Intel CPU with QuickSync (tested as far back as 3rd gen). May be able to work with AMD but I don’t have a card to test with.
- Tested on Ubuntu 22.04, YMMV on other versions or distributions.
- Any user that wants to use this script needs to be part of the “render” group. Check group membership with “groups {username}” If you need to, add users like this:
For user johndoe:
sudo usermod -a -G render johndoe
Then reboot.
INSTRUCTIONS
Copy script to new text file, make it executable, and drop it in the Plex Media Server script directory, then let Plex know about it in Settings. More info on this is in the Plex documentation.
OK, here’s the script:
#!/bin/bash
#
# Plexpost 2022
#
## Job Variables
# Set temporary directory here
temp_dir="/tmp/plexpost"
[ -d $temp_dir ] || mkdir $temp_dir
[ -d $temp_dir/jobs ] || mkdir $temp_dir/jobs
touch $temp_dir/tickets.txt
# Directory to which original file is moved after encode:
originals="$temp_dir/originals/"
[ -d "$originals" ] || mkdir "$originals"
# Make sure anyone can use temp files
chmod 777 -R $temp_dir
job_id=$$
job_wait_timer=5
tickets=$temp_dir/tickets.txt
results_file=$temp_dir/jobs/results-$job_id.txt
>$results_file
echo $job_id >>$results_file
## Greetings
echo
echo "*** Welcome to EncodeBot. ***" | tee -a $results_file
echo "I'll have your file ready in just a little while." | tee -a $results_file
echo | tee -a $results_file
## Setup File Names
filename=$(basename "$1")
fullname=$(readlink -f "$1")
extension="${fullname##*.}"
filepath=$(dirname "$fullname")
# New file should just swap extensions, unless it's already MKV
if [ "$extension" = mkv ] ; then
outfile="${1//.$extension/-processed.mkv}"
else
outfile="${1//.$extension/.mkv}"
fi
subfile=$temp_dir/jobs/$job_id.srt
echo "Processing $filename, in directory $filepath" | tee -a $results_file
echo | tee -a $results_file
## Create symlink to avoid problems with apostrophes in file names
templink="$temp_dir/jobs/$job_id.$extension"
echo "Creating link with safe filename at $templink." | tee -a $results_file
ln -s "$fullname" "$templink"
echo | tee -a $results_file
echo "$@" >> $results_file
echo "$1" >> $temp_dir/post_processing_files.txt
## Job queueing and tracking based on code found at
## routine to queue jobs
queue_job () {
echo "Queueing job $job_id" | tee -a $results_file
if [ ! -f "${tickets}" ] ; then
echo $job_id >>"${tickets}"
#set initial mode of file
chmod 777 "${tickets}" >>$results_file 2>&1
else
echo $job_id >>"${tickets}"
fi
current_job=$(head -1 "${tickets}")
while [ $job_id -ne $current_job ] ; do
depth=0
echo "Job ${job_id} is not ready. Job ${current_job} is running" >>$results_file
while IFS= read -r ; do
ticket=$REPLY
let depth++
if [ ${job_id} -eq ${ticket} ] ; then
position=${depth}
#break
fi
done < "${tickets}"
#depth=$(wc -l $tickets | awk '{print $1}')
echo "Job ${job_id} queue position ${position} of ${depth}." >>$results_file
echo "Sleeping ${job_wait_timer} seconds" >>$results_file
sleep ${job_wait_timer}
current_job=$(head -1 "${tickets}")
done
echo "Job ${job_id} is ready!" | tee -a $results_file
}
## Routine to remove jobs
remove_job () {
echo "Removing job ${job_id} from ${tickets}" >>$results_file
# this step actually creates a new file with new ownerships, but maintains previous mode
sed -i "/^$job_id$/d" "${tickets}"
}
## Routine to cleanup before exiting
cleanup_job () {
# Fix permissions again just in case
chmod 777 -R $temp_dir
# Remove job from queue
remove_job
echo "Job Removed From Queue." | tee -a $results_file
echo | tee -a $results_file
echo "Removing temporary file link." | tee -a $results_file
rm $templink >>$results_file 2>&1
echo | tee -a $results_file
echo "Removing temporary .srt file." | tee -a $results_file
rm $subfile >>$results_file 2>&1
echo | tee -a $results_file
# Cleanup Temporary Storage
echo "Deleting temporary files older than 7 days..." | tee -a $results_file
find "$temp_dir" -mtime +7 -print | tee -a $results_file
deleted=$(find "$temp_dir" -mtime +7 -print -delete)
if [ -n "$deleted" ] ; then
echo "Deleted." | tee -a $results_file
else
echo "Nothing to delete." | tee -a $results_file
fi
echo | tee -a $results_file
timestamp=$(date +"%Y-%m-%d %T")
echo "Job Finished at $timestamp." | tee -a $results_file
echo
}
## Get in line
queue_job
## Start Processing
echo | tee -a $results_file
timestamp=$(date +"%Y-%m-%d %T")
echo "Processing started at $timestamp." | tee -a $results_file
## Discover Video Height
echo | tee -a $results_file
echo "Getting Video Height..." | tee -a $results_file
height=$(/usr/bin/ffprobe -hide_banner -i $templink 2>&1 | egrep " Video: " | sed 's#.* \([[:digit:]]\+x[[:digit:]]\+ [SAR [[:digit:]]\+:[[:digit:]]\+ DAR [[:digit:]]\+:[[:digit:]]\+]\).*#\1#' | sed "s#[[:digit:]]\+x\([[:digit:]]\+\) .*#\1#")
echo $height | tee -a $results_file
## Detect crop
echo | tee -a $results_file
echo "Detecting Crop..." | tee -a $results_file
# Set crop with ffmpeg, adjust cropdetect settings here.
crop="$(/usr/bin/ffmpeg -i $templink -t 300 -vf cropdetect=24:16:1350 -f null - 2>&1 | awk '/crop/ { print $NF }' | tail -1)"
# Check to see if crop detect finished.
if [ -n "$crop" ] ; then
echo "Crop detection successful."
echo $crop | tee -a $results_file
else
echo "Crop detection failed, cleaning up and exiting."
cleanup_job
fi
# Set do_crop variable for ffmpeg command - just adds a preceding comma so we can drop it in the middle of our -vf options.
do_crop=,"$crop"
# Don't crop below a certain threshold. This should cover most cases for video 540 through 1080.
nocrop="crop=1920:1080|crop=1920:1078|crop=1920:1076|crop=1920:1074|crop=1920:1072|crop=1920:1070|crop=1280:720|crop=1280:718|crop=1280:716|crop=1280:714|crop=1280:712|crop=1280:710|crop=1280:708|crop=1280:706|crop=1280:704|crop=704:480|crop=720:480|crop=720:478|crop=704:478|crop=720:476|crop=704:476|crop=720:474|crop=704:474|crop=720:472|crop=704:472|crop=704:540|crop=720:540|crop=720:528|crop=704:528"
if echo $crop | egrep "^(${nocrop})" >/dev/null 2>&1 ; then
crop=""
fi
if [ -n "$crop" ] ; then
echo "Will crop." | tee -a $results_file
else
echo "No crop." | tee -a $results_file
fi
## Closed Captions
# Read Closed Caption data in video stream and create .srt file
echo | tee -a $results_file
echo "Converting closed captions to $subfile..." | tee -a $results_file
/usr/bin/ffmpeg -hide_banner -y -f lavfi -i "movie=$templink[out0+subcc]" -map s $subfile >>$results_file 2>&1
# Correct placement of subtitles
echo "Setting subtitle position..." | tee -a $results_file
sed -i 's/an7/an2/g' $subfile
# Remove tags from subtitles
echo "Removing formatting text from subtitles..." | tee -a $results_file
echo " Tags..." | tee -a $results_file
sed -i 's/<[^>]*>//g' $subfile
echo " Done."
echo " {\an2}..." | tee -a $results_file
sed -i 's/{\\an2}//' $subfile
echo " Done."
echo " \h ..." | tee -a $results_file
sed -i 's/\\h//g' $subfile
echo " Done." | tee -a $results_file
echo "Subtitles converted." | tee -a $results_file
echo "Done." | tee -a $results_file
## ffmpeg Settings
# Set VAAPI device here. The default should work on any supported Intel built-in graphics.
vaapi_device=/dev/dri/renderD128
LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/
# Default ffmpeg statement. Sets up VAAPI hardware acceleration. Generally best to leave this alone.
ffmpeg="ffmpeg -hide_banner -y -hwaccel vaapi -hwaccel_device $vaapi_device -hwaccel_output_format vaapi -vaapi_device $vaapi_device -i"
# Encode Settings
qp="25"
rate="ntsc"
# Audo Settings
# Set Default Here
audio_default="-c:a copy"
audio=$audio_default
# Set Format - Accepts aac or flac. Any other value will copy the audio stream.
format=""
# Format options
if [ "$format" = "aac" ] ; then
audio="-af aresample="matrix_encoding=dplii" -ac 2 -c:a aac -b:a 192k"
else
if [ "$format" = "flac" ] ; then
audio="-af aresample="matrix_encoding=dplii" -ac 2 -c:a flac"
fi
fi
# Set video parameters based on crop or no crop
if [ -n "$crop" ] ; then
video="-vf deinterlace_vaapi,hwdownload,format=yuv420p$do_crop,format=nv12,hwupload -c:v h264_vaapi -qp $qp -r $rate"
else
video="-vf deinterlace_vaapi -c:v h264_vaapi -qp $qp -r $rate"
fi
# Process based on height - Skip deinterlacing 720p content
if [ "$height" = "720" ] ; then
if [ -n "$crop" ] ; then
video="-vf hwdownload,format=yuv420p$do_crop,format=nv12,hwupload -c:v h264_vaapi -qp $qp -r $rate"
else
video="-c:v h264_vaapi -qp $qp -r $rate"
fi
fi
# Process based on names - No Crop
shows="NOVA|It's Always Sunny"
if echo $filename | egrep "^(${shows})" >/dev/null 2>&1 ; then
echo "Name matched for No Crop - enforcing no-crop option." | tee -a $results_file
crop=""
fi
# Process based on names - Original Quality
shows="The Shivering Truth|Rick and Morty|Off the Air"
if echo $filename | egrep "^(${shows})" >/dev/null 2>&1 ; then
echo "Name matched for Original Quality - setting to stream copy." | tee -a $results_file
crop=""
video="-c:v copy"
audio="-c:a copy"
fi
## Deinterlacing alert
# Notify if deinterlacing is on or off. Based on height at this point, will move to separate variable later.
if [ "$height" = "720" ] ; then
echo "Deinterlacing off."
echo | tee -a $results_file
else
echo "Deinterlacing on."
echo | tee -a $results_file
fi
## Final Encode
echo "Encoding to new file $outfile." | tee -a $results_file
echo | tee -a $results_file
echo "Complete ffmpeg command:" | tee -a $results_file
echo "$ffmpeg "$templink" -f srt -i "$subfile" $video $audio -c:s copy "$outfile"" | tee -a $results_file
echo | tee -a $results_file
$ffmpeg "$templink" -f srt -i "$subfile" $video $audio -c:s copy "$outfile" >>$results_file 2>&1 && encode=complete
echo | tee -a $results_file
## Ensure ffmpeg finished without errors, then cleanup.
if [ -n "$encode" ] ; then
echo "Encode complete. Cleaning up." | tee -a $results_file
echo | tee -a $results_file
cleanup_job
echo | tee -a $results_file
# Original File Handling - If Cropped, Move Original File To Temporary Storage, Otherwise Delete It
# if [ -n "$crop" ] ; then
# Move original to temp storage
mv "$1" "$originals" >>$results_file 2>&1
echo "Original file moved to temporary storage at $originals." | tee -a $results_file
# else
# # Delete original file
# rm "$1" >>$results_file 2>&1
# echo "Original file deleted." | tee -a $results_file
# fi
else
echo "Encode failed. Original file will not be altered and new file will be removed. Please read the log at $results_file for more information." | tee -a $results_file
echo | tee -a $results_file
# Remove new file
rm "$outfile"
cleanup_job
fi
TO-DO:
Add NVIDIA and AMD support with automatic card detection
Switches to control encoding without having to change script file
Add optional volume boost back in for instances when audio is re-encoded
Enjoy! Let me know if this helps you or if you make any interesting/useful/etc. additions or changes.
