Cayars - Setup walk through and some tips and tricks

@cayars said:

On another note: Today’s my birthday. Just turned the big five-oh and I don’t know if it’s in my head or not but woke up with a sore jaw, aching back and sore foot. Guess I’m getting old. LOL

Reminds me of an Arlo and Janis cartoon from a while back.

Arlo is standing before in front of a mirror shaving and he says to Janis: “If I sleep well I wake up sore. If I don’t sleep well I wake up sore. I’m beginning to notice a disquieting trend.”

Of course 50 is just a beginning of a good life. “I intend to live forever - So far so good.”

And I will leave you with a birthday wish that my ex wife gave me: “You have just turned 50. You will never have fun again for the rest of your life.” She was wrong and the fun started with the divorce.

Happy birthday. >:) Keep up the good work.

@benamira:

First, just want to say this is all @cayars and @MovieFan.Plex work, they were gracious enough to help me out.

MP4 Converter Details and How to Install

autoProcess.ini
you can read more about this here sickbeard_mp4_converter, but the only settings in this tutorial that matters are:

output_directory =

  • blank will place it in the same location the file is found
  • fill this out if you want it to be a different location

delete_original =

  • True/False

audio-language =

  • blank will keep all audio formats
  • 3 letter country code, eng by default means it will strip out ALL OTHER AUDIO other than English.

Performance Tweaking
under Convert\Converter\ffmpeg.py change to the following:

  • cmds.extend([’-crf’, ‘18’])
  • cmds.extend([’-threads’, ‘1’])
    this will increase the quality and also only allow it to run on one core per process, again helping with quality.

SQL Scripts and Creating your batch files to run the process

  • First and foremost you are using your plex database, and even though it is just select statements, ALWAYS make a backup or just run the scripts against a backup like I did.
  • Download and install SQLite3 binaries onto your machine from here. You will want SQLite3 and all the files below in the same directory.

Remember, the below scripts don’t do the work, they just create batch files for you to run that will do the work. Depending on what OS/how your structure is setup you might need to modify the output of these to run. For example, Plex stores a record of the file I need like ‘/tvseries/Adventure Time/Season 01/Adventure Time - S01E04 - Tree Trunks.mp4’, I needed to find and replace the slashes and also add an ip to the beginning. Nothing major but just something to notice before you kick it off.

This will list all movies without English Audio, meaning no language is listed as well
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “1_Non-English_Audio_Files.bat”
select mp.file from media_parts as mp inner join media_items as mi on mi.id = mp.media_item_id inner join metadata_items as mdi on mdi.id = mi.metadata_item_id where mp.id not in (SELECT media_part_id from media_streams where stream_type_id = 2 and language = “eng”) and mdi.metadata_type < 5 order by mp.file;

OR This will list all movies without English Audio but also not list no language, assuming these are english, your choice.
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “2_No_Default_Or_English_Audio_Files.bat”
select mp.file from media_parts as mp inner join media_items as mi on mi.id = mp.media_item_id inner join metadata_items as mdi on mdi.id = mi.metadata_item_id where mp.id not in (SELECT media_part_id from media_streams where stream_type_id = 2 and (language = “eng” or language = “”)) and mdi.metadata_type < 5 order by mp.file;

See Autoprocess.ini section for language changes

MP4’s that aren’t “web optimized for streaming”, MOOV_ATOM
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “3_Web_Optimize_Files.bat”
SELECT ‘[PATH_TO_PYTHON]\Python [PATH_TO_MP4_CONVERTER]\manual.py -a -i "’ || file || ‘"’ FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where container=‘mp4’ and optimized_for_streaming <> 1
order by file;

Find 10 Bit Videos to fix using straight up ffmpeg instead of the MP4 Converter script
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “4_10Bit_Files.bat”
select ‘[PATH_TO_MP4_CONVERTER]\ffmpeg -i "’ || file || ‘" -c:v libx264 -c:v libx264 -preset slow -crf 18 -acodec copy -map 0 -profile:v high -level 4.0 "C:\convert\done’ ||
replace(file, rtrim(file, replace(file, ‘’, ‘’ ) ), ‘’) || ‘"’
FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where media_parts.extra_data like ‘%AVideoProfile=high%%2010%’
order by file;

Make a list of all non-MP4’s.
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “6_Complete_File_List.bat”
SELECT ‘[PATH_TO_PYTHON]\Python [PATH_TO_MP4_CONVERTER]\manual.py -a -i "’ || file || ‘"’ FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where container!=‘mp4’
order by file;

1 Like

@benamira said:
happy birthday cayars!! And remember that fifty-oh are the new forty-oh!! :slight_smile:
now one question., of course i don’t expect an answer today…
I’m thinking in converting my library using your scripts (oh my god!!, tons of movies…), do you already recommend this for a large library?? Or better wait for future add-ons to “optimize” in Plex settings?
Most of my library is MKV (around 3000 movies).

Personally I’d do it outside of Plex so you can have much more control over the process and get far better results. I’ve posted most of the scripts to use in this thread. don.alcombright just gave you the overall best way to go about this for items ALREADY in your library.

I would however add one additional step in there before step 5. That would be to process MP4s in your library that have the wrong level or are 10 bit video (often causes transcodes).

@MikeG6.5 You’re right. A lot has changed. I noticed one quote JFK made your list but it was a couple of years before my time. 1963 vs 1966. It was still on everyone’s heart and minds. You also missed one of the biggest events in my lifetime “One small step for man…” which happened just about a year after the assassination of Martin Luther King, Jr.

I hadn’t really thought much about this until reading what you wrote. From the hippies of the Woodstock/Vietnam era to today. I can remember as a small guy having two water fountains and bathrooms for the "white and “black” (wonder if this was along the lines of what was edited from your post, if so I see no need as it’s history and you can’t “delete” history). Thankfully our country has come a long way on this front but we still have a ways to go. Thankfully with each new generation, we are one step removed from the ways of the past.

@Plexhilarated I never saw the song that way before. Very cute. I sung along to it. :slight_smile:

@vanstinator you got the “title” for oldest fart in the thread. :slight_smile: I’m surprised how old a bunch of us our here in this thread. I’m so used to most people on the techie side being 1/2 to 1/3 our age.

@vanstinator you got the “title” for oldest fart in the thread. :slight_smile: I’m surprised how old a bunch of us our here in this thread. I’m so used to most people on the techie side being 1/2 to 1/3 our age.

Definitely meant I’m 25 younger. So I’m one of those 1/2 your age folks :wink:

Ahh, read that wrong when you said you had us beat by 25 years. :slight_smile:

@don.alcombright said:
@benamira:

First, just want to say this is all @cayars and @MovieFan.Plex work, they were gracious enough to help me out.

MP4 Converter Details and How to Install
https://forums.plex.tv/discussion/comment/931888/#Comment_931888

autoProcess.ini
you can read more about this here sickbeard_mp4_converter, but the only settings in this tutorial that matters are:

output_directory =

  • blank will place it in the same location the file is found
  • fill this out if you want it to be a different location

delete_original =

  • True/False

audio-language =

  • blank will keep all audio formats
  • 3 letter country code, eng by default means it will strip out ALL OTHER AUDIO other than English.

Performance Tweaking
under Convert\Converter\ffmpeg.py change to the following:

  • cmds.extend([‘-crf’, ‘18’])
  • cmds.extend([‘-threads’, ‘1’])
    this will increase the quality and also only allow it to run on one core per process, again helping with quality.

SQL Scripts and Creating your batch files to run the process

  • First and foremost you are using your plex database, and even though it is just select statements, ALWAYS make a backup or just run the scripts against a backup like I did.
  • Download and install SQLite3 binaries onto your machine from here. You will want SQLite3 and all the files below in the same directory.

Remember, the below scripts don’t do the work, they just create batch files for you to run that will do the work. Depending on what OS/how your structure is setup you might need to modify the output of these to run. For example, Plex stores a record of the file I need like ‘/tvseries/Adventure Time/Season 01/Adventure Time - S01E04 - Tree Trunks.mp4’, I needed to find and replace the slashes and also add an ip to the beginning. Nothing major but just something to notice before you kick it off.

This will list all movies without English Audio, meaning no language is listed as well
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “1_Non-English_Audio_Files.bat”
select mp.file from media_parts as mp inner join media_items as mi on mi.id = mp.media_item_id inner join metadata_items as mdi on mdi.id = mi.metadata_item_id where mp.id not in (SELECT media_part_id from media_streams where stream_type_id = 2 and language = “eng”) and mdi.metadata_type < 5 order by mp.file;

OR This will list all movies without English Audio but also not list no language, assuming these are english, your choice.
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “2_No_Default_Or_English_Audio_Files.bat”
select mp.file from media_parts as mp inner join media_items as mi on mi.id = mp.media_item_id inner join metadata_items as mdi on mdi.id = mi.metadata_item_id where mp.id not in (SELECT media_part_id from media_streams where stream_type_id = 2 and (language = “eng” or language = “”)) and mdi.metadata_type < 5 order by mp.file;

See Autoprocess.ini section for language changes

MP4’s that aren’t “web optimized for streaming”, MOOV_ATOM
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “3_Web_Optimize_Files.bat”
SELECT ‘[PATH_TO_PYTHON]\Python [PATH_TO_MP4_CONVERTER]\manual.py -a -i "’ || file || ‘"’ FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where container=‘mp4’ and optimized_for_streaming <> 1
order by file;

Find 10 Bit Videos to fix using straight up ffmpeg instead of the MP4 Converter script
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “4_10Bit_Files.bat”
select ‘[PATH_TO_MP4_CONVERTER]\ffmpeg -i "’ || file || ‘" -c:v libx264 -c:v libx264 -preset slow -crf 18 -acodec copy -map 0 -profile:v high -level 4.0 “C:\convert\done' ||
replace(file, rtrim(file, replace(file, '', ‘’ ) ), ‘’) || '”’
FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where media_parts.extra_data like ‘%AVideoProfile=high%%2010%’
order by file;

Make a list of all non-MP4’s.
.open ‘[PATH_TO_PLEX_DB]\com.plexapp.plugins.library.db’
.output “6_Complete_File_List.bat”
SELECT ‘[PATH_TO_PYTHON]\Python [PATH_TO_MP4_CONVERTER]\manual.py -a -i "’ || file || ‘"’ FROM media_parts join media_items
on media_parts.media_item_id=media_items.id
where container!=‘mp4’
order by file;

Awesome!!
Let me check in detail your suggestions and i will prepare the best plan for my particular case.
Thanks!

@cayars said:

On another note: Today’s my birthday. Just turned the big five-oh and I don’t know if it’s in my head or not but woke up with a sore jaw, aching back and sore foot. Guess I’m getting old. LOL

It’s not just “in your head”, it’s real. :stuck_out_tongue:
Happy birthday.

@cayars Happy Birthday! :slight_smile:

@cayars said:
Ahh, read that wrong when you said you had us beat by 25 years. :slight_smile:

Puh!
That mean I don’t have to change my username right away :slight_smile:

(Oldest until proven otherwise…)

@cayars said:

It will pull out any english subtitles and create SRT files from them.  If you want to pull out other languages then modify the subtitle-language setting in the ini file.  Multiple languages are supported by commas

The completed file will not have subtitles (we pulled them out)

It will clear all tags in the mp4 file so Plex doesn’t pick up and use these tags instead of the proper meta-data

It will create an AAC track as the first track upto 256K in size.

It will remove all audio tracks not in english.  If you want other languages included modify audio-language setting (ini file) and include any languages you want kept in the MP4 file.  Multiple languages are supported by commas

I tried to modify the ini file… I need to add subtitles and audio tracks. Tried to write the languages in different ways… but ended up with the bat closes…

How do I set norwegian as default audio language, and create aac stereo stream and 5.1 stream of both norwegian and english?

And how to do I set it to extract Nor subtitles, swe subtitles and eng subtitles? and set nor as default…?

Grab the 3 letter country codes from here: http://www.worldatlas.com/aatlas/ctycodes.htm

Just use those after the = sign
seperate country codes by a comma

Found out now… My bad… I opened to edit in notepad… and everything was on one line…

Yea, notepad is a b*itch for this type of thing. Notepad++ or similar tool is much better.

The ini file is much cleaner looking and easier to figure out when you can actually read and edit it. :slight_smile:

Hmm… I did set the default language to Nor… and now it changes unknown languate to nor when it convert avi’s :stuck_out_tongue: is there a way to change the setting so it wont change to nor? I do have to remove the default again I guess?

And is there an easy way change the language data in a mp4 file?

If the track isn’t marked we only have a couple of choices.
We can skip it since it doesn’t exactly match what you specify.
We can include it hoping it’s what.

They are really the only two choices we have and you can do it either way based on your ini settings.

Well… I want to include it. But I know it’s in eng… and the convert process make ist nor. I guess this might only be the issue with only a few of my files… So is there a way to change it quickly afterwars?

And also… Is there a way to keep the other sub formats? like dvdsub? as it’s setup now it removes the sub files completely… only srt’s is copied…

leave it blank after the = sign for the default. If you assign it a value then anything not marked will be treated as the default. That is what that setting is. “the default to set unknown items to”.

It only works for srt files.

@cayars said:
I’ve decided it’s time to add a 1 TB SSD drive to my system exclusively for the MetaData and Transcoding folders.

I am considering doing this with my 245GB of Plex Metadata, did you notice a big difference?

@neebski said:

@cayars said:
I’ve decided it’s time to add a 1 TB SSD drive to my system exclusively for the MetaData and Transcoding folders.

I am considering doing this with my 245GB of Plex Metadata, did you notice a big difference?

A bunch of us have noticed a huge difference when serving the metadata off of an SSD. I made the move a few months ago and haven’t looked back

^^ That ^^

512GB Samsung x2