Trying to figure out a new storage solution and not quite sure what direction to go in. My current setup has been very simple. I have a a dedicated media server PC running windows 10 and plex server. I use the same machine for ripping and any manipulation of video files that may be needed. I have 2 portable seagate 5TB USB drives attached and run synctoy periodically to keep identical copies. I lost a disk in the past and had to re-rip everything. Don’t want to do that again. With UHD rips a reality now I’m eating through space with about 1TB left. Looking at NAS, DAS, RAID, online backup solutions, etc. I don’t need server/transcoding abilities as I will continue to use the PC server for that. Having the ability to do MacOS time machine backups would be a nice bonus but not a necessity. I was really temped to buy a Synology DS418 or DS418J, adding 3 or 4 drives, and running some form of RAID. That seems pretty popular and very easy to upgrade storage over time keeping a single volume that plex sees. But that still seems to be risky even if I do RAID 6 or SHR-2 without another backup solution which starts to get pricey adding everything up. Online backups for that amount of data seem to be pretty expensive too. So then I thought of just a simple 4 bay enclosure with a couple large drives and do something similar to what I’m doing now with synctoy. That’s the cheapest and simplest but not as easy as a RAID system to add/expand and keep all media on a single volume that Plex sees. So I’m just looking for any recs for a easy to use/expand but safe storage solution without breaking the bank.
You have a ton of options and there is no right or wrong way to do it. Well there are bad ways such as using RAID 5 and I might argue RAID 6 as well. 
Going with a NAS is a popular way to expand storage. Adding additional internal drives or even external USB3 drives can be quite effective and maybe all you need.
I have and do use multiple NAS devices, multiple servers, multiple external USB3 drives as well as multiple DAS drive enclosures. Check my sig and you’ll see I’ve got a decent size collection of media.
First a couple of questions. You mention you have two 5TB drives. Are you sure these are 5TB and not 4 or 6TB?
Regardless, are you syncing the contents of drive one over to drive two just to make sure you don’t lose any data?
Are you looking for better $ per TB storage (assuming you are doubling your storage just to have a copy)?
There are quite a few really good ways achieve what you are looking for.
Do you have an idea how big your library will be in 1 or 2 years storage wise?
Give us some additional info to work with to help you.
Carlo
@cayars said:
You have a ton of options and there is no right or wrong way to do it. Well there are bad ways such as using RAID 5 and I might argue RAID 6 as well.Going with a NAS is a popular way to expand storage. Adding additional internal drives or even external USB3 drives can be quite effective and maybe all you need.
I have and do use multiple NAS devices, multiple servers, multiple external USB3 drives as well as multiple DAS drive enclosures. Check my sig and you’ll see I’ve got a decent size collection of media.
First a couple of questions. You mention you have two 5TB drives. Are you sure these are 5TB and not 4 or 6TB?
Regardless, are you syncing the contents of drive one over to drive two just to make sure you don’t lose any data?Are you looking for better $ per TB storage (assuming you are doubling your storage just to have a copy)?
There are quite a few really good ways achieve what you are looking for.
Do you have an idea how big your library will be in 1 or 2 years storage wise?Give us some additional info to work with to help you.
Carlo
Thanks. Yes right now I have 2 5TB portable usb 3 drives. Pretty cheap actually at $130 each.
https://www.amazon.com/dp/B01LZP2B23/ref=cm_sw_r_cp_api_BrqCAbDE68VSF
I use synctoy as a scheduled task to duplicate my media folder from primary drive to backup drive. Little hard to guess on storage needs going forward but could be 1-2TB per year with all these uhd rips. Right now I’m using 4tb so was thinking to at least expand to 8tb. I don’t have a particular budget but when I started pricing out a Synology NAS plus 4 drives for raid 6 and then some sort of back up for that the cost was pretty steep. I like the idea of a nas running raid 6 or Synology’s shr-2 but not sure in reality I need to do all that. I do have 2 extra 2tb portable usb drives used before the 5TB drives). I could just get a 4 bay enclosure, get 2.5” to 3.5” drive adapters, remove bare drives from enclosures (may need to destroy enclosure) and put all 4 drives in enclosure. That would give me 7TB each for use and back up spending the least amount for now. What ever I do I do like the idea of having all the drives in an enclosure rather than loose connected to a powered usb hub. Too much clutter for me.
I know this mostly comes down to my own preferences and needs but I’m sure there are lots of users here who have tried many options and found pros and cons I’m not thinking of. Also happy to hear others back up solutions. Are those with huge 10-20TB libraries really duplicating the whole library for back up.
I have about 23 tb of media and I do duplicate all the media.
My setup is a computer that acts as a file server (Plex runs on a different computer) that has about 50 tb of external USB drives attached that are pooled using StableBit’s DrivePool. I use DirvePool’s duplication feature for redundancy that is almost as good as a true backup for my needs but I also backup to another much slower computer with 25tb of external drives also pooled but with no duplication.
I find that setup very reliable and easy to both maintain and expand as needed. The only real negative is clutter. Each external drive needs its own external power supply and then there are all the USB cables. But, for me, that matters very little as I have it all mounted to a rolling wire shelf system that holds my file server and my plex server and my modem and my router and a good sized UPS.All that lives in a back room well out of sight and generally can be maintained by access using a VNC setup. My backup computer is only turned on once a week and it lives inside a fire proof cabinet in the same room. That backup is not perfect but it is close enough for me. The cabinet gets opened and the computer gets turned on once a week for a couple of hours and, during that time the whole system is vulnerable to a disaster like a fire.
My setup is not for everyone but I have yet to run into any real problems.
BTW: DrivePool stores all files in native Windows format so, should I loose all my computers at once the drives can still be accessed and read when attached to any other computer.
@cayars said:
You have a ton of options and there is no right or wrong way to do it. Well there are bad ways such as using RAID 5 and I might argue RAID 6 as well.
What would you think is a good RAID? Or do you not believe in them?
Honestly though. Not trying to be facetious.
RAID 5 is old technology that isn’t ideal for big drives like we have these days. RAID 5 basically allows you to lose 1 drive but if you lost/lose a 2nd drive all is lost.
RAID 6 allows you to lose 2 drives without loss of data but if you lose the 3rd drive all is lost. Both of these may not sound bad but what happens when you have a drive go bad and then are in the rebuild phase with a new drive and the extra work of the rebuild causes another drive to fail? This happens a lot more then people realize.
RAID 5/6 have their place in business when working with small files like documents, Xcel files and business apps since the overall storage doesn’t need to be big (rebuilds can be fast). This changes when you are basically trying to build storage solutions.
The problem with RAID 6 for the OP is that he’s thinking of adding a 4 bay NAS. Well if 2 bays are needed for RAID 6 that only leaves 2 bays for data. He might as well just mirror the data and be done with it.
RAID systems make a bit more sense when you have 8 drive bays as that leaves more drives for data. RAID systems can come with limitations that cause many people untold issues when they try to expand storage as they need to rebuild the arrays which takes longer and longer the bigger you get.
I’ve been down this road to many times and now just prefer KISS (Keep It Simple Stupid). I run windows and mount drives (local, USB3, NAS or File Server) as local drives on the Plex Server. So with say 20 drives if I loose 1 drive I’ve lost 1/20th of my storage and never all data at once. I use DrivePool to group multiple drives into one drive letter (makes everything look like 1 giant drive like RAID can) which makes plex setup easy as well as FTP servers or just accessing my media over the network.
For backup or redundancy I use the free opensource SnapRAID (not raid at all) to create parity of all my drives (local or network). For me this works awesome as I can add drives any time I want and size of drives doesn’t matter. I can pull a drive and move it to any other computer and access it normally since it’s not a proprietary drive format but just good old NTFS…
With SnapRAID I can have from 1 to 8 drives used for Parity and I can add a new parity drive anytime I want for additional redundancy.
So the OP could purchase a 4 bay drive NAS or just a direct attached storage and populate it as needed with additional drives one at a time and keep using the 2 USB3 drives he already has for maximum flexibility in drive space. He could start with 1 drive parity and then add a second drive later when ever he wanted.
Again, like I said in my first post there isn’t a best way but there are many ways but RAID 5 is just bad for media files and arguable RAID 6 is bad as well or just not as ideal.
Thanks again. I think I do prefer the simplest approach which would be DAS, no RAID. Less to configure. Less to go wrong. Drivepool does sound like a good app with many happy users. I’m likely going to purchase one of the mediasonic 4 bay non-raid enclosures. $100 on Amazon. For now will use the 4 2.5” usb drives I have but extract them from their enclosures and buy adapters to fit in the mediasonic 3.5” enclosure (only $10) and then start to replace with 3.5” drives when I run out of space. As I mentioned I have 2 5TB drives in use and 2 2TB drives not currently in use. I can create 2 7TB pools with one for backup. Remaining question is how best to do backups. I can continue with scheduled synctoy runs. I have also read than drivepool can create duplicates. Not yet sure how that works. Is that just as safe? Will have to read about snapraid. Sounds interesting too. If I’m keeping a copy of my media with synctoy or drivepool I’m not sure if snapraid would have a use. Would it be usefull instead of those options for backup or not as safe?
I have not used the Seagate USB drives you have but did convert about a dozen WD USB3 drives to internal drives to put them into the server and an 8 bay DAS box. I’m now back with 4 or 5 USB3 attached drives again as needed additional storage.
With the WD drives you can’t just pull the drives and use them but need to format them before using them since the little USB3 to SATA board does encryption of the drive. No idea if the Seagates do anything like this or not. But it’s something to check or be on the lookout for.
If you aren’t having trouble using the drives as USB attached drives there might not be a reason to gut them to put them into a DAS box. For me with 20 drives or so the wiring was becoming a nightmare. But I never had a single issue using the drives via USB.
So you could keep those drives external and then just populate the DAS box with new drives. There is no right or wrong way here.
Carlo
@cayars said:
I have not used the Seagate USB drives you have but did convert about a dozen WD USB3 drives to internal drives to put them into the server and an 8 bay DAS box. I’m now back with 4 or 5 USB3 attached drives again as needed additional storage.With the WD drives you can’t just pull the drives and use them but need to format them before using them since the little USB3 to SATA board does encryption of the drive. No idea if the Seagates do anything like this or not. But it’s something to check or be on the lookout for.
If you aren’t having trouble using the drives as USB attached drives there might not be a reason to gut them to put them into a DAS box. For me with 20 drives or so the wiring was becoming a nightmare. But I never had a single issue using the drives via USB.
So you could keep those drives external and then just populate the DAS box with new drives. There is no right or wrong way here.
Carlo
So right now I have 2 mirrored 5TB drives with all my media and 2 2TB unused drives. Lets say for now I keep them as external drives on a hub. What is the simplest approach for transitioning all 4 drives to drivepool and using drivepool’s 2x duplication as my backup plan? It seems like there maybe a lot of upfront copying/duplicating files with the way drivepool works.
I have been using Linux’s software raid mdadm for years without any issues. Currently using a RAID-5 on 4 3TB disks. I’ve had disks die and replaced without issues. mdadm is great! Biggest thing to remember is RAID is not a backup. Therefore I use a cronjob to backup my RAID-5 to another internal 8TB disk nightly. Worst case scenario, if I lose 2 disks in the raid, I at least have a backup within 24 hours.
Monthly I also backup the RAID-5 to an external drive which I encrypt and keep in my desk drawer at work. I use rdiff-backup which retains versions and deleted files in case of accidental deletion etc… Most importantly, it gives me easy offsite backup in case of fire, theft, etc…
I’ve used this setup for years and works great. Maybe overkill for some, but all depends on how important you feel your data is.
@cardsdoc said:
@cayars said:
I have not used the Seagate USB drives you have but did convert about a dozen WD USB3 drives to internal drives to put them into the server and an 8 bay DAS box. I’m now back with 4 or 5 USB3 attached drives again as needed additional storage.With the WD drives you can’t just pull the drives and use them but need to format them before using them since the little USB3 to SATA board does encryption of the drive. No idea if the Seagates do anything like this or not. But it’s something to check or be on the lookout for.
If you aren’t having trouble using the drives as USB attached drives there might not be a reason to gut them to put them into a DAS box. For me with 20 drives or so the wiring was becoming a nightmare. But I never had a single issue using the drives via USB.
So you could keep those drives external and then just populate the DAS box with new drives. There is no right or wrong way here.
Carlo
So right now I have 2 mirrored 5TB drives with all my media and 2 2TB unused drives. Lets say for now I keep them as external drives on a hub. What is the simplest approach for transitioning all 4 drives to drivepool and using drivepool’s 2x duplication as my backup plan? It seems like there maybe a lot of upfront copying/duplicating files with the way drivepool works.
Stop using 2x duplication. Instead go with drivepool and ADD SnapRaid. SnapRAID allows you to create a parity disc that can work across multiple discs. So instead of needing to dedicate 1/2 your space to duplication you can only give up 1/8 or so (when you get more discs).
So with that I’d add the two 2TB drives to the new 4 bay box. Keep one of the 5TB drives as a data drive. So that’s 3 DATA drives (5TB+2TB+2TB=9TB) and then convert the other 5TB drive to use for parity with SnapRAID.
You can then add an additional 2 drives to the 4 Drive bay enclosure and still use just the one parity drive. The only thing to think about or keep in mind is that the parity drive needs to be the biggest drive in the system since it needs to mirror the other drives.
If you wanted to avoid a conversion down the road you could also purchase a larger drive like 8, 10 or 12TB and make that the parity drive right from the beginning. This is what I would do if $ allow it. The parity drive is actually just a large file on that disc. It will only be as big (roughly) as the consumed space on the largest data drive. So if you were to pickup a 10TB drive to use for Parity this drive would still have roughly 5TB open on it which could also be used for duplication if needed.
The duplication of course is optional but if you also store important docs (taxes, financial, xcel, word docs, etc) on your “NAS” then you can have these “backed up” by the parity (SnapRAID) as well as duplication from (DrivePool).
This type of setup is very, very flexible. One of the cool things with a setup like this is that SnapRAID (parity) can “backup” both local and network drives as well as write the parity to a networked parity drive. Very flexible.
Carlo
I use Unraid on a dedicated machine for my server. It’s great if you’re a hobbyist and not an expert because the WebUI, docker integration and file system makes it really easy to do just about anything you’d want your Plex server to do without typing a single line of code.
@cayars said:
@cardsdoc said:
@cayars said:
I have not used the Seagate USB drives you have but did convert about a dozen WD USB3 drives to internal drives to put them into the server and an 8 bay DAS box. I’m now back with 4 or 5 USB3 attached drives again as needed additional storage.With the WD drives you can’t just pull the drives and use them but need to format them before using them since the little USB3 to SATA board does encryption of the drive. No idea if the Seagates do anything like this or not. But it’s something to check or be on the lookout for.
If you aren’t having trouble using the drives as USB attached drives there might not be a reason to gut them to put them into a DAS box. For me with 20 drives or so the wiring was becoming a nightmare. But I never had a single issue using the drives via USB.
So you could keep those drives external and then just populate the DAS box with new drives. There is no right or wrong way here.
Carlo
So right now I have 2 mirrored 5TB drives with all my media and 2 2TB unused drives. Lets say for now I keep them as external drives on a hub. What is the simplest approach for transitioning all 4 drives to drivepool and using drivepool’s 2x duplication as my backup plan? It seems like there maybe a lot of upfront copying/duplicating files with the way drivepool works.
Stop using 2x duplication. Instead go with drivepool and ADD SnapRaid. SnapRAID allows you to create a parity disc that can work across multiple discs. So instead of needing to dedicate 1/2 your space to duplication you can only give up 1/8 or so (when you get more discs).
So with that I’d add the two 2TB drives to the new 4 bay box. Keep one of the 5TB drives as a data drive. So that’s 3 DATA drives (5TB+2TB+2TB=9TB) and then convert the other 5TB drive to use for parity with SnapRAID.
You can then add an additional 2 drives to the 4 Drive bay enclosure and still use just the one parity drive. The only thing to think about or keep in mind is that the parity drive needs to be the biggest drive in the system since it needs to mirror the other drives.
If you wanted to avoid a conversion down the road you could also purchase a larger drive like 8, 10 or 12TB and make that the parity drive right from the beginning. This is what I would do if $ allow it. The parity drive is actually just a large file on that disc. It will only be as big (roughly) as the consumed space on the largest data drive. So if you were to pickup a 10TB drive to use for Parity this drive would still have roughly 5TB open on it which could also be used for duplication if needed.
The duplication of course is optional but if you also store important docs (taxes, financial, xcel, word docs, etc) on your “NAS” then you can have these “backed up” by the parity (SnapRAID) as well as duplication from (DrivePool).
This type of setup is very, very flexible. One of the cool things with a setup like this is that SnapRAID (parity) can “backup” both local and network drives as well as write the parity to a networked parity drive. Very flexible.
Carlo
Very interesting. Thanks again. Seems like a pretty slick setup this way. Will think about this. Using drivepool alone with duplication while it requires more disc space is just very simple and all GUI driven. I think I will start this way but as I get closer to filling up space think about adding snapraid and getting rid of duplication. Also for now I think I’ll just go with a powered USB hub and keep drives external until I need more space then go with a DAS enclosure and 3.5" drives.
I lost a R6 array the other day. One drive popped, two more popped during the rebuild. This was mid-range enterprise class hardware too.
FWIW my personal implementation of my PMS storage has been RAIDZ2, then RAIDZ1 (and two servers with an identical copy each) but now BTRFS in RAID10.
Some great solutions offered in this thread. I started evaluating my options too, as my storage needs continue to grow rapidly with 4k. I’m up to about 14 TB on an internal PC drive enclosure with 4, 10 TB hard drives (Seagate Barracude Pro) in a Windows storage space with parity. I prefer keeping everything on 1 PC, just to keep it clean, but unless 20 TB hard drives come out soon I can see running out of space quickly. A separate DAS seems to be the best approach using one of the software “raid” tools mentioned.
Couple of questions though:
Is Windows Storage Spaces a feasible alternative to the other “raid” tools mentioned? I have noticed file copies to the storage space maxes utilization on the “drive”, but I don’t notice any issues with playing back media.
If a DAS becomes the only option I have for increased storage, what should I be looking for? I use a 4-bay MediaSonic for periodic full backups of my data, but not sure that would be the best solution for the active Plex library. And if I’m using a DAS, should the connection be USB, eSATA, or does it even matter? At least on my PC it seems like bandwidth is shared by all eSATA/SATA and USB ports, which doesn’t quite make sense to me and seems like it increases the possibility of slowness when Plex scans the library.
If your Internet connection is half ways decent then forget about sinking money into hardware that will fail & become redundant & never keep pace with your needs. G Suite for Business costs $10/€8/£6.60/month & gives you unlimited storage on Google Drive. Mount the Google Drive on your local machine with rclone or Netdrive & it operates just like a local drive. If you have remote users then use some more of the money you saved to rent a VPS then your whole Plex infrastructure can live out in the Cloud. Just follow this excellent guide https://www.techperplexed.ga/p/index.html
@nigelpb said:
If your Internet connection is half ways decent then forget about sinking money into hardware that will fail & become redundant & never keep pace with your needs. G Suite for Business costs $10/€8/£6.60/month & gives you unlimited storage on Google Drive. Mount the Google Drive on your local machine with rclone or Netdrive & it operates just like a local drive. If you have remote users then use some more of the money you saved to rent a VPS then your whole Plex infrastructure can live out in the Cloud. Just follow this excellent guide https://www.techperplexed.ga/p/index.html
Is this what you did? or are you just providing more options?
@NewPlaza said:
@nigelpb said:
If your Internet connection is half ways decent then forget about sinking money into hardware that will fail & become redundant & never keep pace with your needs. G Suite for Business costs $10/€8/£6.60/month & gives you unlimited storage on Google Drive. Mount the Google Drive on your local machine with rclone or Netdrive & it operates just like a local drive. If you have remote users then use some more of the money you saved to rent a VPS then your whole Plex infrastructure can live out in the Cloud. Just follow this excellent guide https://www.techperplexed.ga/p/index.htmlIs this what you did? or are you just providing more options?
Yes, have you used this solution? And how did you get $10 per month for unlimited? The G suite pricing on the web site is $10 per month, but is capped at 1 TB per user if under 5 users. From the web site, “accounts with fewer than 5 users get 1TB per user”. So it would basically cost $600 per year to get unlimited.
I’m not sure it would work for me anyways since my internet connection is 100 Mb. It would work for most content, but not 4k when the internet connection is being shared. So I think I’m still searching for that local option.
I recommend just buying an off-the-shelf unit like Synology or QNAP and being done with it. Some of the pricer models you can run plex on, or just get a cheap one and store data on it.
@NewPlaza said:
@nigelpb said:
If your Internet connection is half ways decent then forget about sinking money into hardware that will fail & become redundant & never keep pace with your needs. G Suite for Business costs $10/€8/£6.60/month & gives you unlimited storage on Google Drive. Mount the Google Drive on your local machine with rclone or Netdrive & it operates just like a local drive. If you have remote users then use some more of the money you saved to rent a VPS then your whole Plex infrastructure can live out in the Cloud. Just follow this excellent guide https://www.techperplexed.ga/p/index.htmlIs this what you did? or are you just providing more options?
Along with many others I use Google Drive for my Plex library. I mostly use Plex Cloud but also have a VPS & a local PMS mounting the library. There is a long discussion in this thread on using a VPS https://forums.plex.tv/discussion/262896/plex-cloud-vs-vps-why-we-went-with-a-vps#latest
