That’s what I’m currently using for USB enclosure. Works well with ESATA or USB3 and a good cost. Doesn’t support RAID but I don’t want that. Each drive in the enclosure is a separate drive letter to the operating system.
I’m using the NAS boxes mostly to just write parity to from SnapRAID so I just set them up as SMB links.
I use iSCSI for SAN mounts however.
SnapRAID will write parity to a network drive but a network drive can’t be a “source” drive. For SnapRAID to work it needs to have a “local” drive letter. That’s why I use iSCSI for the media drives but not for the parity drives.
I’m using a couple of the cheap WD NAS boxes that are super cheap (think Walmart). Think USB3 drive but with Ethernet connection vs USB3. Not really much different than plugging in a USB 3 drive to your router. Works great for parity drives.
Carlo,
What are you using as a host card / connector for that external enclosure? How are the write speeds? I have four 6TB, 2 5TB, and 2 2TB drives that would be great to put into a single enclosure instead of spreading them out across multiple PC’s. But, I’m concerned about the write speeds, and about what happens to reading from disks while writing to others if everything is in a single chassis like that. My network is gigabit, so it “shouldn’t” cause issues… but, I always try and think things through before jumping in a particular direction.
That’s what I’m currently using for USB enclosure. Works well with ESATA or USB3 and a good cost. Doesn’t support RAID but I don’t want that. Each drive in the enclosure is a separate drive letter to the operating system.
I’m using the NAS boxes mostly to just write parity to from SnapRAID so I just set them up as SMB links.
I use iSCSI for SAN mounts however.
SnapRAID will write parity to a network drive but a network drive can’t be a “source” drive. For SnapRAID to work it needs to have a “local” drive letter. That’s why I use iSCSI for the media drives but not for the parity drives.
I’m using a couple of the cheap WD NAS boxes that are super cheap (think Walmart). Think USB3 drive but with Ethernet connection vs USB3. Not really much different than plugging in a USB 3 drive to your router. Works great for parity drives.
Thank you for your great answer. Also interested in your Connector and Speed.
What WD Nas Boxes you mean? WD Mycloud? Cant think of any Cheap 1bay Nas here
@ember1205 said:
What are you using as a host card / connector for that external enclosure? How are the write speeds? I have four 6TB, 2 5TB, and 2 2TB drives that would be great to put into a single enclosure instead of spreading them out across multiple PC’s. But, I’m concerned about the write speeds, and about what happens to reading from disks while writing to others if everything is in a single chassis like that. My network is gigabit, so it “shouldn’t” cause issues… but, I always try and think things through before jumping in a particular direction.
Right now a USB3 port. With USB3 I get 120MB+ on transfers to the box from a drive not in the box. If I copy from drive to drive in the box it’s about 60Mb+. So basically the box itself when connected via USB3 gives you similar speed to a normal WD USB3 external drive. Plenty fast enough for Plex use and has never caused a problem.
@ember1205 said:
What are you using as a host card / connector for that external enclosure? How are the write speeds? I have four 6TB, 2 5TB, and 2 2TB drives that would be great to put into a single enclosure instead of spreading them out across multiple PC’s. But, I’m concerned about the write speeds, and about what happens to reading from disks while writing to others if everything is in a single chassis like that. My network is gigabit, so it “shouldn’t” cause issues… but, I always try and think things through before jumping in a particular direction.
Right now a USB3 port. With USB3 I get 120MB+ on transfers to the box from a drive not in the box. If I copy from drive to drive in the box it’s about 60Mb+. So basically the box itself when connected via USB3 gives you similar speed to a normal WD USB3 external drive. Plenty fast enough for Plex use and has never caused a problem.
Since a lot of my media is ripped from one system and then transferred across the network to the Plex system, those speeds might be a big step down from what I get currently. I’ll have to do some file transfers and measure actual timings to see how much of a difference it would be.
One of the things that I’ve found with Plex is that you need to copy the media file onto the disk into a directory that is NOT part of a library first, then “move” it to a directory that is in a library. Otherwise, you either have to manually tell Plex to scan library files or wait for an update interval before the media shows up in the library. And that’s with the reasonably good transfer speeds I currently get…
So for example if you use the 8TB model you can use it as 1 parity. On the back of this drive is a USB expansion port so you can plug another 8TB USB3 drive into it and have a 2nd parity drive.
These NAS boxes don’t have fast CPUs so you aren’t going to run Plex on them or anything like that but they work really well for normal storage that is very reliable. Since I had out growed them for storage I found SnapRAID can write to them so they made a lot of sense to use for Parity (for me).
From the standpoint of PARITY and SnapRAID storing this data over a gigabit network is fine. I prefer to keep local drives for storage so they can be accessed quickly and network drives for parity.
I was in need of another drive and the local Walmart had them on sale so I picked one up to test. I figured if it didn’t work well I could always pull the drive from it like I’ve done with more than a dozen USB3 WD drives. From day one the drive worked well for parity so I’ve just left it that way. I’ve got a couple of them I use this way.
If I had to go cheap on network drives I’d rather do this than use a 4 or 8 bay NAS since each drive this way get’s it’s own Ethernet 1Gb connection (assuming I didn’t want raid 5 which I don’t). However you can use SnapRAID to backup these types of drives since they aren’t local. But they work great for parity drives which is how I use them.
I never thought about it until right now but SnapRAID could probably be changed to backup network drives since it’s open source. I might take a look at this now that I think about it. Could be useful.
@kevindd992002 said: @cayars I have the old version of your “Convert” script as I’ve started using it since early this year (2017). I see that the last modified date, in your FTP server, of Convert 2.2 is 4/16/2017. How do I upgrade my existing Convert (which is running fine btw, except for the fact that I still don’t know how to run it silently, I hope you can still help me with this) to the latest version? Do I just overwrite the files over?
Another issue that I have is with Plex for Roku. My media files are converted by your script and creates the AAC Stereo track as the first track properly. But for some reason, Plex for Roku always plays the AC3 5.1 (2nd track) by default. When I manually change the audio stream, it goes into transcode mode instead of Direct stream. Is this something to be expected?
I don’t think you’ll get it to run silently without a lot of work. It was just never designed to run this way. I just leave it running minimized with the 60 second check so it just keeps processing any new files it finds.
Version 2 fixed a problem in the original (now ver 1) where it never marked the 1st audio track as the default. That’s probably what is causing you issues on the Roku. After I created version 2 I ran 50K+ files back through it just to mark the audio tracks correctly on my system. It’s does them pretty fast because it’s just a remux. Basically as fast as files can be read, written.
Hope this is ok to ask here, since this might also apply to your re-encoded versions:
If I have two different versions of e.g. the same episode: Does Plex always pick the best for the client or does this heavily depend on the client itself? I know OpenPHT asks which version I want to play while e.g. the web app does not ask.
I am curious because recently an LG smart TV was added to my list of clients which seems not to like my files to Direct Play. I am thinking to re-encode at least some of my library and probably keep the original files side by side.
I’d suggest just getting the script working at the quality you like (defaults should work) and use it then replace the originals. If you have things setup correctly you should have a very hard time telling the original from the converted video. I can’t tell the difference on my 75" TV.
Keep in mind if the source file has the video portion in the correct format it will just copy it without re-transcoding it which is fast and prevents any degradation.
@cayars you are probably spot on about the quality. I guess I will try it with some shows/episodes and see how that works on the respective LG device. If it does not Direct Play anyways, there is no need to re-encode everything.
I have to adapt it for Linux first, but that shouldn’t be to complicated I guess. Thanks already for your effort.
@Coxeroni said: @cayars you are probably spot on about the quality. I guess I will try it with some shows/episodes and see how that works on the respective LG device. If it does not Direct Play anyways, there is no need to re-encode everything.
I have to adapt it for Linux first, but that shouldn’t be to complicated I guess. Thanks already for your effort.
I’m using Handbrake on a Linux host to do the video transcoding and forced subtitle inclusion (where necessary). In the process, I’m converting from MKV to MP4. When complete, I pass the resultant file through the v2.2 scripts from Carlo to modify the audio and to relocate the MOV atom (and convert to M4V). The high quality preset I’m using with Handbrake keeps the image quality such that I can’t tell a difference from the lossless original on a 75" 4K panel and I have zero issues playing any of the files through a Roku TV or the native Plex app on my various devices (including Android TV).
The reason that I’m doing it the way that I am as opposed to just using the scripts from Carlo is because Handbrake actually compresses more than the FFMPEG that Carlo’s scripts use as their base. I routinely see 50-75% compression with Handbrake while Carlo’s scripts are generally in the 30-50% area.
And, again, I saw zero difference in PQ among lossless, FFMPEG transcoded, and Handbrake transcoded. So, I went with the one that gave me the best compression overall. I spent weeks of time allowing my process to munge its way through about 750-1000 movies, but all -I- had to do in the process was to move the uncompressed files out of the library and into a working folder. The control script that I wrote took care of all of the processing and moved the new files into a temp directory where a subsequent cron job would identify the presence of new files and then import them into the Plex library automatically.
So for example if you use the 8TB model you can use it as 1 parity. On the back of this drive is a USB expansion port so you can plug another 8TB USB3 drive into it and have a 2nd parity drive.
These NAS boxes don’t have fast CPUs so you aren’t going to run Plex on them or anything like that but they work really well for normal storage that is very reliable. Since I had out growed them for storage I found SnapRAID can write to them so they made a lot of sense to use for Parity (for me).
From the standpoint of PARITY and SnapRAID storing this data over a gigabit network is fine. I prefer to keep local drives for storage so they can be accessed quickly and network drives for parity.
I was in need of another drive and the local Walmart had them on sale so I picked one up to test. I figured if it didn’t work well I could always pull the drive from it like I’ve done with more than a dozen USB3 WD drives. From day one the drive worked well for parity so I’ve just left it that way. I’ve got a couple of them I use this way.
If I had to go cheap on network drives I’d rather do this than use a 4 or 8 bay NAS since each drive this way get’s it’s own Ethernet 1Gb connection (assuming I didn’t want raid 5 which I don’t). However you can use SnapRAID to backup these types of drives since they aren’t local. But they work great for parity drives which is how I use them.
I never thought about it until right now but SnapRAID could probably be changed to backup network drives since it’s open source. I might take a look at this now that I think about it. Could be useful.
Thank you for your answer - too bad the “normal” Price of the Boxes is just too high (200€+ for 6TB)
Will go for a USB3 Enclosure for now I think - too bad the Rackmount ones are so expensive
Thanks @ember1205 for sharing your thoughts. Gotta figure out how I do this most efficiently on my VPS, since my media files are sitting at google drive but for the re-encoding I would need to move them first to my VPS drive. Additionally I would want to incorporate the re-encoding in the future directly into sickrage but I am also not sure at which point and how to do it. First world problems
@Coxeroni said: @cayars you are probably spot on about the quality. I guess I will try it with some shows/episodes and see how that works on the respective LG device. If it does not Direct Play anyways, there is no need to re-encode everything.
I have to adapt it for Linux first, but that shouldn’t be to complicated I guess. Thanks already for your effort.
I’m using Handbrake on a Linux host to do the video transcoding and forced subtitle inclusion (where necessary). In the process, I’m converting from MKV to MP4. When complete, I pass the resultant file through the v2.2 scripts from Carlo to modify the audio and to relocate the MOV atom (and convert to M4V). The high quality preset I’m using with Handbrake keeps the image quality such that I can’t tell a difference from the lossless original on a 75" 4K panel and I have zero issues playing any of the files through a Roku TV or the native Plex app on my various devices (including Android TV).
The reason that I’m doing it the way that I am as opposed to just using the scripts from Carlo is because Handbrake actually compresses more than the FFMPEG that Carlo’s scripts use as their base. I routinely see 50-75% compression with Handbrake while Carlo’s scripts are generally in the 30-50% area.
And, again, I saw zero difference in PQ among lossless, FFMPEG transcoded, and Handbrake transcoded. So, I went with the one that gave me the best compression overall. I spent weeks of time allowing my process to munge its way through about 750-1000 movies, but all -I- had to do in the process was to move the uncompressed files out of the library and into a working folder. The control script that I wrote took care of all of the processing and moved the new files into a temp directory where a subsequent cron job would identify the presence of new files and then import them into the Plex library automatically.
You can easily get more compression out of my scripts if you want. Just change the video-crf to a higher number. Also something to keep in mind. My scripts won’t touch the video if it’s already in the correct format so this won’t get you additional compression.
The scripts as setup (default values) is to get them direct playable at the highest quality. They can be reconfigured to always compress and if using a higher CRF then to save space as well.
But if you have something that works, it’s best to leave well enough alone.
@Coxeroni said:
Thanks @ember1205 for sharing your thoughts. Gotta figure out how I do this most efficiently on my VPS, since my media files are sitting at google drive but for the re-encoding I would need to move them first to my VPS drive. Additionally I would want to incorporate the re-encoding in the future directly into sickrage but I am also not sure at which point and how to do it. First world problems
You don’t want to have to copy the files down to your local machine to process then to move back. You want to do the conversions on the machine hosting your files.
@Coxeroni said:
Thanks @ember1205 for sharing your thoughts. Gotta figure out how I do this most efficiently on my VPS, since my media files are sitting at google drive but for the re-encoding I would need to move them first to my VPS drive. Additionally I would want to incorporate the re-encoding in the future directly into sickrage but I am also not sure at which point and how to do it. First world problems
You don’t want to have to copy the files down to your local machine to process then to move back. You want to do the conversions on the machine hosting your files.
Exactely, I don’t want to. But maybe I have to Working with rclone mounts has not been super stable for me. Will test it nevertheless anytime soon.
You can easily get more compression out of my scripts if you want. Just change the video-crf to a higher number. Also something to keep in mind. My scripts won’t touch the video if it’s already in the correct format so this won’t get you additional compression.
The scripts as setup (default values) is to get them direct playable at the highest quality. They can be reconfigured to always compress and if using a higher CRF then to save space as well.
But if you have something that works, it’s best to leave well enough alone.
I had tried different settings with your scripts to bring down the CRF and such, but Handbrake was giving me the kind of quality AND compression that I wanted. So, as you said, I stuck with what was already working instead of doing a ton of customization to your scripts to “get me back to where I was” (if you will). End result is that, for me, I have what I feel is a solid setup that is easy for me to use. I’m pretty sure no one else would agree because there are three or four different machines that are involved when ripping BluRay’s before they end up in the library, and there are manual steps.
@kevindd992002 said: @cayars I have the old version of your “Convert” script as I’ve started using it since early this year (2017). I see that the last modified date, in your FTP server, of Convert 2.2 is 4/16/2017. How do I upgrade my existing Convert (which is running fine btw, except for the fact that I still don’t know how to run it silently, I hope you can still help me with this) to the latest version? Do I just overwrite the files over?
Another issue that I have is with Plex for Roku. My media files are converted by your script and creates the AAC Stereo track as the first track properly. But for some reason, Plex for Roku always plays the AC3 5.1 (2nd track) by default. When I manually change the audio stream, it goes into transcode mode instead of Direct stream. Is this something to be expected?
I don’t think you’ll get it to run silently without a lot of work. It was just never designed to run this way. I just leave it running minimized with the 60 second check so it just keeps processing any new files it finds.
Version 2 fixed a problem in the original (now ver 1) where it never marked the 1st audio track as the default. That’s probably what is causing you issues on the Roku. After I created version 2 I ran 50K+ files back through it just to mark the audio tracks correctly on my system. It’s does them pretty fast because it’s just a remux. Basically as fast as files can be read, written.
Carlo
Carlo,
Yes, I understand this and I’m already past this point of questioning This is the latest post that you might’ve missed:
@kevindd992002 said:
Did you receive my PM regarding the CP/Sonarr/Sab/Convert script configuration though?
Yes, just haven’t responded yet. Will get to it. Got a list of PMs so I take them in order and a bit backed up.
One more thing that I noticed when I tried Convert 2.2 with CPU encoding. The output file has an extension of .mp4.original. Why is it appeding the .original there?
I’ve never seen that before. Are you using the original ini file?
@kevindd992002 said:
Did you receive my PM regarding the CP/Sonarr/Sab/Convert script configuration though?
Yes, just haven’t responded yet. Will get to it. Got a list of PMs so I take them in order and a bit backed up.
One more thing that I noticed when I tried Convert 2.2 with CPU encoding. The output file has an extension of .mp4.original. Why is it appeding the .original there?
I’ve never seen that before. Are you using the original ini file?
Ok, cool!
Yes, I’m using the original INI file. To be specific, here’s a copy of it if you want to check for yourself:
With USB3 I get 120MB+ on transfers to the box from a drive not in the box. If I copy from drive to drive in the box it’s about 60Mb+. So basically the box itself when connected via USB3 gives you similar speed to a normal WD USB3 external drive. Plenty fast enough for Plex use and has never caused a problem.
Working with rclone mounts has not been super stable for me. Will test it nevertheless anytime soon.