Is there anyway to get Cayars script to embed the movie art into the file like Sickbeard MP4 automator does, but still keep it simple and portable?
Thought I previously posted this but guess I didnāt.
Download CayarsBeta1.zip from ftp://ayars.tv
username and password are both guest
Unzip to C: drive. You will end up with C:\Convert
There are two directories (can be changed) inside this folder that are used for the media
Process is where you put any files you want to convert
Done is where the files will end up after processing
You can run this multiple ways but the easiest is to use the Run.bat file.
If you edit this file you will notice itās one line that reads: c:\python27\python manual.py -a -i c:\convert\process
If you install python to a different folder then change the python directory at the beginning of this line.
If you want to process files from a different location then c:\convert\process then just change this in the batch file.
If you want to change the final location of where the final files are stored then edit autoProcess.ini file and change the output_directory setting.
You will need Python2.7x installed.
in browser goto https://www.python.org/downloads/windows/
scroll down to Python 2.713 and download/install Windows x86-64 MSI installer
install Python2.7x into c:\python27 directory for all users
in browser goto https://pypi.python.org/pypi/setuptools#installation-instructions
download and install setuptools-34.0.1-py2.py3-none-any.whl (md5)
open powershell and cut and paste the following:
cd c:\python27
pip install requests
pip install requests[security]
pip install requests-cache
pip install babelfish
pip install āguessit<2ā
pip install āsubliminal<2ā
pip install stevedore==1.19.1
pip install python-dateutil
pip install deluge-client
pip install qtfaststart
exit or close powershell
As with previous script modify run.bat to match your directories if needed. At present it will look in c:\convert\process for files that need to be processed.
The above instructions are included in install.txt that will be in the convert directory.
In this version of the scripts Iāve tried to keep in closer to the original scripts so that you can use all of the original functionality but itās tuned to run better with Plex in our environment. Iāve also made a few changes to settings. For example it will now use hardware QuickSync encoding and seems to work well. If you donāt want to use hardware encoding open autoProcess.ini and change
use-qsv-decoder-with-encoder = True to use-qsv-decoder-with-encoder = False
Iāve also changed the max level from 4.0 to 4.1. If you would prefer 4.0 level then just change this line in the ini
h264-max-level =4.1
Those are the two options that are different than before. Of course there are also a lot of changes under the hood so let me know how it works for you.
This includes the latest ffmpeg files instead of using slightly older versions as before.
Let me know how things work and keep in mind this is a beta. I know of a couple of things I want to change.
Carlo
You know I use the older version in my workflowā¦ 
Iāll be testing this over the next few days and post any problems I encounter. And I will be making the required edits for Linux I did last time, for anyone else interested in the port. Refer to the link in my signature for any changes I had to make for full functionality in Linux.
And thanks Carlo. Been hoping this would pop up here!
Iāve been using it for about 2 months pretty steady and have ran over 1000 files through it. There are still a couple of little changes I want to make but they will be minor. Just havenāt gotten to them yet. The changes mostly concern writing meta-data back into the files.
@MikeG6.5 never thought to mention it before but once you have them updated let me know. I can host them for you on my FTP server if you want (up to you). How you did the last version is just fine as well.
@cayars said:
Thought I previously posted this but guess I didnāt.Download CayarsBeta1.zip from ftp://ayars.tv
username and password are both guest
Unzip to C: drive. You will end up with C:\Convert
There are two directories (can be changed) inside this folder that are used for the media
Process is where you put any files you want to convert
Done is where the files will end up after processing
You can run this multiple ways but the easiest is to use the Run.bat file.
If you edit this file you will notice itās one line that reads: c:\python27\python manual.py -a -i c:\convert\process
If you install python to a different folder then change the python directory at the beginning of this line.
If you want to process files from a different location then c:\convert\process then just change this in the batch file.
If you want to change the final location of where the final files are stored then edit autoProcess.ini file and change the output_directory setting.You will need Python2.7x installed.
in browser goto Python Releases for Windows | Python.org
scroll down to Python 2.713 and download/install Windows x86-64 MSI installerinstall Python2.7x into c:\python27 directory for all users
in browser goto setuptools Ā· PyPI
download and install setuptools-34.0.1-py2.py3-none-any.whl (md5)open powershell and cut and paste the following:
cd c:\python27
pip install requests
pip install requests[security]
pip install requests-cache
pip install babelfish
pip install āguessit<2ā
pip install āsubliminal<2ā
pip install stevedore==1.19.1
pip install python-dateutil
pip install deluge-client
pip install qtfaststart
exit or close powershellAs with previous script modify run.bat to match your directories if needed. At present it will look in c:\convert\process for files that need to be processed.
The above instructions are included in install.txt that will be in the convert directory.
In this version of the scripts Iāve tried to keep in closer to the original scripts so that you can use all of the original functionality but itās tuned to run better with Plex in our environment. Iāve also made a few changes to settings. For example it will now use hardware QuickSync encoding and seems to work well. If you donāt want to use hardware encoding open autoProcess.ini and change
use-qsv-decoder-with-encoder = True to use-qsv-decoder-with-encoder = FalseIāve also changed the max level from 4.0 to 4.1. If you would prefer 4.0 level then just change this line in the ini
h264-max-level =4.1Those are the two options that are different than before. Of course there are also a lot of changes under the hood so let me know how it works for you.
This includes the latest ffmpeg files instead of using slightly older versions as before.
Let me know how things work and keep in mind this is a beta. I know of a couple of things I want to change.
Carlo
Been using the old release for months. Followed these instructions and installed the beta. Woke up this morning to find lots of movies ruined. They display a thumbnail and refuse to play on any players. I donāt even know how bad the damage is yet, but Iām guessing I lost a lot.
Beware.
Edit:
I lost 52 movies from what I can tell. 500+ GB. Most of them downloaded within the last couple months. All of them had been processes by the original script in the past. On the beta script I used default settings. I turned off āprocess MP4āsā. So the only thing I can tell is that it messed up the files while writing tags, since it didnāt actually process the videos.
Ouch, I hate to say it but with anything BETA you should always test things before putting them in production.
With that said itās strange as Iāve now put over 2000 (at least) files through it running on 4 different machines. Iām using it with the DVR. TS files, mkv, avi, etc
Can you upload one of your ruined files? Iād like to take a look at it. Upload to the Media folder on my FTP server ftp://ayars.tv
username and password both āguestā.
Thanks,
Carlo
Iāve already pruned them out and deleted them. In process of redownloading them now. I disabled writing tags and tested the script again, and that seems to have fixed the problem. All of the files that got ruined had already been converted with the older script you posted, and the only thing the new script did was write tags to them. So not sure if there is some correlation there or not.
On the ruined files, they would just open up and display the downloaded Poster in the middle of the of the media player window. Skipping forward or backward didnāt change anything. All that the files would do was display the Poster. Trying to play them via Plex would just return an error. Whatās weird is that 100ās of other movies that had been converted months ago all work fine still. So no idea what the issue could be. Iām just gonna keep testing with tagging off and see if all is good.
Edit:
Also, I have a Fiber connection so I took the risks. Take me about an hour to download 500 GB worth of movies. My warning is for those who donāt have a super fast connection.
If you can see if you can reproduce the error on a ānon productionā file. If you can do me a favor and upload both the original and messed up version to my FTP server. That will enable me to get a look at it and see if I can figure out what went wrong.
Iām on a Verizon FIOS connection so it shouldnāt take to long to upload. Alternately use a file sharing site and PM the d/l location.
Just so you know Iām not happy with the tagging ability in this new version and think it needs tweaking. If it canāt connect to pull down meta info it does some weird things but Iāve never seen it mess up the file.
Thanks,
Carlo
Just wanted to say a big thank you Cayars for your script - beers on me if youāre ever down Hampshire way in the UK. Iām not completely incompetent with Powershell but Python is well out of my league!
Iām using it with another mishmash of borrowed and edited code to get utorrent working correctly with both rarād and flat files - wish I could credit the original owners but Iām now not sure where I got the code from, Iāve been through that many pages looking for inspiration.
Pasting it here if itās of use to anyone. utorrent uses the ārun programā functionality to call this as a batch file:
set fromdir=%1
set name=%2
set label=%3
set kind=%4
set filename=%5
set todir=z: orrents\extracted\
if %kind%=="single" goto copyfile
if exist %fromdir%\*.rar goto extractrar
if exist %fromdir%\*.mkv goto copyfile
if exist %fromdir%\*.avi goto copyfile
if exist %fromdir%\*.mp4 goto copyfile
GOTO:EOF
:copyfile
xcopy %fromdir%\*.mkv %todir% /S /I /Y
xcopy %fromdir%\*.mp4 %todir% /S /I /Y
xcopy %fromdir%\*.avi %todir% /S /I /Y
GOTO runtranscode
:extractrar
echo Extracts all rars in %fromdir% to %todir%.
rem Requires WinRar installed to c:\Program files
if not exist %todir% mkdir %todir%
"c:\program files\winrar\winrar.exe" x -ibck -inul %fromdir%\*.rar *.* %todir%
GOTO runtranscode
:runtranscode
cd\
c:
cd convert
c:\python27\python manual.py -a -i Z: orrents\extracted
GOTO:EOF
So:
uTorrent pulls the file.
Script moves it if itās an mkv, avi or mp4 to the āextractedā folder. If itās a .rar, winrar takes over and extracts to the same folder.
Cayars script then runs against the āextractedā folder and outputs to the āconvertedā folder.
Sonarr then pulls the final file in to the relevant library location.
Carlo, thank you for everything and welcome back! This thread has been an incredible source for information. Had two questions:
(1) Are you still using SnapRAID as a backup? Have any of your drives failed (if so, how many?) and have you been able to restore them successfully?
(2) Did you eventually backup everything to Amazon Drive? Are you using Amazon Drive as a mapped network drive for Plex? If so, do your movies stream on Plex as well as if it were streaming off a local drive?
Yes still using SnapRAID. Iāve had the fun of restoring from it because of user goof. I accidentally deleted the wrong directories, etc. Iāve not had a hardware failure.
Back when I was testing it I pulled and replaced 2 drives to restore to make sure it actually worked correctly and it did.
I no longer backup to Amazon. I never really used Amazon to stream from but did test it and it sort of worked sometimes. Some days it worked perfectly and other days it seemed like the Amazon pipe was ācloggedā.
Thanks for the update! Now that I know SnapRAID is āCarlo Approvedā I will continue to use the software. Did Amazon ever give you a hard time, or question, utilizing a decent amount of their resources (bandwidth and/or space)?
Have you added more hard drives to your environment? I would love to add a few more to mine, but my wife would like me to justify the additional space. I currently have 30 TBs and she believes that is enough, the nerve of that woman! Curious to see where you are at since the picture you took of the WD external drives.
I pulled the drives from the USB enclosures and put 8 of them in one of these https://www.newegg.com/Product/Product.aspx?Item=9SIA1UH3H76495&cm_re=8_bay_hard_drive_enclosure--17-707-367--Product and the rest inside my main server and two in my 2nd Plex server.
Iām currently running two Plex servers. The first is my old reliable Dell i7 first gen running at 2.8 GHz with an AMD 280x video card. This is running the Hardware encoding BETA Plex software.
The 2nd is an old Dell T5500 XEON 4 core (slow) running the Plex DVR Beta. This machine is used strictly for recording TV, then running my post processing scripts which cut commercial, convert to MP4 and then through my normal scripts. This machine has an nVidia GTX 750 graphic card for NVENC hardware encoding to help speed up the process. This machine was and is still used for additional storage (file server) and is plenty fast for this purpose.
So at present to run both hardware encoding and DVR I need to machines.
Iām getting great results using hardware encoding.
I recently moved my servers from one side of my basement to the other side of my basement. Once side is āfinishedā and one side is not. The ānot sideā has the heater, hot water heater, washer and dryer as well as what was a bunch of servers. Wasnāt the best place for them.
The finished side has steps sink, gas stove/oven on the left wall (1). Circuit Box, Cables sat feeds, telephone wiring and FIOS as well as my Half fridge, 57" TV with xBox One on wall 2. On wall 3 is my loading table/bench with computer benches right next to it. Half the time my computer bench is full of ammo. Right now itās got a couple thousand 12 gauge rounds Iāve loaded.
PS I shoot trap, skeet, 5 stand and sporting clays (shotgun) as well as small and large caliber handgun and small and large rifle shooting competitions. Usually shoot twice a week in some type of formal shoots plus practice so I go through lots of rounds a year thus the loading bench and presses. I clean, process and make my own bullets and pellets as well as reload. I go through 40 to 50 lb of lead each week. Luckily I find it enjoyable to do so it saves a lot of money and gives me great quality control on my ammo. before anyone asks no lead is ever melted inside. That is all well ventilated outside work:)
Anyway, itās an extreme āman caveā if you will. Only problem is both my dad and son will sometimes āhogā it.
@cayars said:
I pulled the drives from the USB enclosures and put 8 of them in one of these https://www.newegg.com/Product/Product.aspx?Item=9SIA1UH3H76495&cm_re=8_bay_hard_drive_enclosure-_-17-707-367-_-Product and the rest inside my main server and two in my 2nd Plex server.Iām currently running two Plex servers. The first is my old reliable Dell i7 first gen running at 2.8 GHz with an AMD 280x video card. This is running the Hardware encoding BETA Plex software.
The 2nd is an old Dell T5500 XEON 4 core (slow) running the Plex DVR Beta. This machine is used strictly for recording TV, then running my post processing scripts which cut commercial, convert to MP4 and then through my normal scripts. This machine has an nVidia GTX 750 graphic card for NVENC hardware encoding to help speed up the process. This machine was and is still used for additional storage (file server) and is plenty fast for this purpose.
So at present to run both hardware encoding and DVR I need to machines.
Iām getting great results using hardware encoding.
I recently moved my servers from one side of my basement to the other side of my basement. Once side is āfinishedā and one side is not. The ānot sideā has the heater, hot water heater, washer and dryer as well as what was a bunch of servers. Wasnāt the best place for them.
The finished side has steps sink, gas stove/oven on the left wall (1). Circuit Box, Cables sat feeds, telephone wiring and FIOS as well as my Half fridge, 57" TV with xBox One on wall 2. On wall 3 is my loading table/bench with computer benches right next to it. Half the time my computer bench is full of ammo. Right now itās got a couple thousand 12 gauge rounds Iāve loaded.
PS I shoot trap, skeet, 5 stand and sporting clays (shotgun) as well as small and large caliber handgun and small and large rifle shooting competitions. Usually shoot twice a week in some type of formal shoots plus practice so I go through lots of rounds a year thus the loading bench and presses. I clean, process and make my own bullets and pellets as well as reload. I go through 40 to 50 lb of lead each week. Luckily I find it enjoyable to do so it saves a lot of money and gives me great quality control on my ammo. before anyone asks no lead is ever melted inside. That is all well ventilated outside work:)
Anyway, itās an extreme āman caveā if you will. Only problem is both my dad and son will sometimes āhogā it.
Sounds like an awesome setup.
I see youāre using both and and Intel for HW transcoding (in different settings). Anything to say about the differences? Did you notice one giving better quality output?
What happened to #71 - Storage Spaces? I have a Windows Server 2012 R2 with and Areca RAID controller with 2x RAID 5 with different size disc today. Iām thinking about upgrading to Windows Server 2016, buying some new discs and going for storage spaces. Iām thinking that the advantage will be more flexibility as I can use different size discs, and if I add a cheap 512 GB SSD I can get greater speed, and Iāll cut power usage since I can sell the RAID controller. Itās more future proof as well as I can remove old smaller disks and and bigger new ones. But if thereās something written regarding Storage Spaces and Plex Iād love to read about it.
Storage spaces turned out to be a very bad idea on a Plex server. Main problem is everything needs to be done in striped sets. So for example imagine having just set up 8 4 TB disks which is roughly 32 TB or raw disk (actually smaller) and works out to be something like 25 TB of usable space. Now you want to go and add a single 6 TB disk because you need more storage. What do you do?
Youāre already maxed out at 8 drives for the āpoolā so this new drive wonāt be part of any striping. Thatās just the beginning of the problems youāll experience with RAID or Storage Spaces in general.
The far simpler and easier setup I found was to use Stablebit DrivePool to combine any amount of drives into a pooled drive letter for easy Plex use. You can add/remove drive at will and can pull a drive and mount it in any other computer if needed. Drivepool doesnāt care about the size of your discs at all (unlike storage spaces).
Then use SnapRAID for parity/backup. This will give you a very flexible storage system that is really hard to beet for Media use.
I do use RAID for other purposes but not for Plex. BTW, save the SSD for plex use and maybe your operating system if itās big enough. Putting the plex meta data on an SSD drive is one of the things you can do to keep things snappy. You donāt need to ācacheā your media files with the use of the SSD.
Hope that helps,
Carlo
@cayars said:
The far simpler and easier setup I found was to use Stablebit DrivePool to combine any amount of drives into a pooled drive letter for easy Plex use. You can add/remove drive at will and can pull a drive and mount it in any other computer if needed. Drivepool doesnāt care about the size of your discs at all (unlike storage spaces).
Hope that helps,
Carlo
What would be the point of using this kind of solution vs JBOD considering once the folders are pointed to in your Plex libraries it all becomes transparent to the end user?
I could see it if they integrated their Clouddrive and made it possible to backup your drivepool encrypted in ACD or GSuite, but otherwiseā¦ I donāt see it?
Simplicity.
As an example here are a few drives:
D:
3D Movies
4K Movies
Movies #-A
NFL
Screeners
E:
Movies A-D
G:
Movies D-H
H:
Movies H-M
M:
Movies M-R
N:
Movies R-T
O:
Movies T
P:
Movies T-Z
TV Shows/Ended
1,000 Places to See Before You Die (2007) to 100 (2014)
Q:
TV Shows/Ended
100 (2014) to La Femme Nikita (1997)
R:
TV Shows/Ended
Las Vegas (2003) to The West Wing (1999)
S:
TV Shows/Ended
The West Wing (1999) to Zero Hour (2004)
TV Shows/Ongoing
2 Broke Girls (2011) to Law & Order, Special Victims Unit (1999)
T:
TV Shows/Ongoing
Law & Order, Special Victims Unit (1999) to Z Nation (2014)
V:
Home Movies
Music
W:
Educational
Now instead of having to track where everything is and setup multiple library paths to each Plex library i can use Stablebit Drive pool and just put to one of these:
F:\3D Movies
F:\4K Movies
F:\Educational
F:\Home Movies
F:\Movies
F:\Music
F:\NFL
F:\Screeners
F:\TV Shows\Ended
F:\TV Shows\Ongoing
This makes life much easier to manage. I can also share my pooled drive F over the network so I can copy/move files easily to the server. It makes setting up my FTP much easier as well as I donāt need several movie or several TV show ādirectoriesā. It just makes life a lot easier to manage files.
Plus as already mentioned I can add a new drive at will to the system and then just add it to DrivePool and I have more instant storage available.
I do however setup SnapRaid to create parity across the individual drives and not the pooled drive of course.
If you only have a few drives then Drivepool might not be needed/wanted but when you start to get a decent amount of drives it makes things easier.
What I also like is the fact that Plex always sees my media coming from drive F regardless of what drive itās on. So if I have to reorganize drives it doesnāt affect Plex. No more having to rescan, delete optimize libs so plex can find files when I move them among disks.
Hope that helps,
Carlo
BTW, DrivePool has many other features that can be used that I didnāt mention. It can make duplicate copies (or triplets or 4 copies, etc) of any directories if you wish. So if you have a document or home movies directory Drivepool can make sure to keep copies on separate disks. I donāt use this for media in general except for my home movies.
It can do a lot of other things as well.
SnapRAID is open source and Drivepool is cheap so it was well worth the investment in my book!
Carlo
@cayars said:
Simplicity.As an example here are a few drives:
D:
3D Movies
4K Movies
Movies #-A
NFL
ScreenersE:
Movies A-DG:
Movies D-HH:
Movies H-MM:
Movies M-RN:
Movies R-TO:
Movies TP:
Movies T-Z
TV Shows/Ended
1,000 Places to See Before You Die (2007) to 100 (2014)Q:
TV Shows/Ended
100 (2014) to La Femme Nikita (1997)R:
TV Shows/Ended
Las Vegas (2003) to The West Wing (1999)S:
TV Shows/Ended
The West Wing (1999) to Zero Hour (2004)
TV Shows/Ongoing
2 Broke Girls (2011) to Law & Order, Special Victims Unit (1999)T:
TV Shows/Ongoing
Law & Order, Special Victims Unit (1999) to Z Nation (2014)V:
Home Movies
MusicW:
EducationalNow instead of having to track where everything is and setup multiple library paths to each Plex library i can use Stablebit Drive pool and just put to one of these:
F:\3D Movies
F:\4K Movies
F:\Educational
F:\Home Movies
F:\Movies
F:\Music
F:\NFL
F:\Screeners
F:\TV Shows\Ended
F:\TV Shows\OngoingThis makes life much easier to manage. I can also share my pooled drive F over the network so I can copy/move files easily to the server. It makes setting up my FTP much easier as well as I donāt need several movie or several TV show ādirectoriesā. It just makes life a lot easier to manage files.
Plus as already mentioned I can add a new drive at will to the system and then just add it to DrivePool and I have more instant storage available.
I do however setup SnapRaid to create parity across the individual drives and not the pooled drive of course.
If you only have a few drives then Drivepool might not be needed/wanted but when you start to get a decent amount of drives it makes things easier.
What I also like is the fact that Plex always sees my media coming from drive F regardless of what drive itās on. So if I have to reorganize drives it doesnāt affect Plex. No more having to rescan, delete optimize libs so plex can find files when I move them among disks.
Hope that helps,
Carlo
Makes sense, though like you said I donāt think I need it with the storage I have right now.
Iām more trying to use cloud storage and stream over netdrive, seems to be working well on new rig so far.
Last question, when using pool drive, can you still access the original drive by its letter or do they all ādisappearā behind the one new drive letter?
Thanks
