4-Drive Setup Questions

General Discussions
1

Hi everyone, noob here.

I currently have an 8TB hard drive that is just about full. I have 2 10TB hard drives on the way, for a total of 28TB. I believe additional video files will yield approximately 5TB, for a total of 13TB used of 28TB total. I also have a 4-drive Probox on the way. My end goal is to stream movies via Plex (on a Roku TV) in my apartment.

I would appreciate feedback on the best way to setup the drives and to protect the files from drive failure. As far as setting up, I was contemplating parity via Raid 5, but Iā€™ve read that reconstructing a failed driver can lead to additional drive failure and permanent loss of data. @cayars recommended using DrivePool to group the drives into one and SnapRAID for redundancy: here. That was almost two years ago, and Iā€™m unclear if this is still the best route to follow and if so, how to set these drives up in DrivePool so that they are grouped together but not via RAID (like what settings would I be looking for), and then how to integrate SnapRAID into the process. I would ideally like one massive drive to work from, the date of which is reasonably protected should drive failure occur.

Last, Iā€™m not sure of the best way to backup the files. Is mirroring recommended, or the duplicate option in DrivePool? Or should I just upload everything to an online service - if so, is any particular one recommended?

Really appreciate the help!

2

I use UnRaid myself.

you would take 1 drive (needs to be as big as your biggest data drive) so in your case 10T and make it a parity drive. the others would be data drives so you would have 18T available and either drive can fail and you can access your data during failure and go and buy a replacement drive and have it rebuild your drive.

I have used this for about 5 or 7 years now and have lost drives with no data loss or down time.

the free version lets you do what I mentioned above with a limited amount of data drives. you can get a paid for version and have multiple parity drives so you can lose more than 1 drive at a time and also additional features like cache drive and more data drives.

what cayars mentioned is also a valid way of still doing this.
there is also a product called FreeNas which does softraid like unraid.

EDIT - my bad there was a ā€œfreeā€ unraid version back in the day but now there is only a paid for version starting at $59. They do have a free 30 day trial but it was free for up to 3 drives for a long time.

3

There are proā€™s and conā€™s to every storage type. NO storage type is a substitute for backups of important data.

Backups = multiple copies of the same data, best on different types of storage, and ideally in multiple physical locations (think fire/flood/theft).

Whether audio/video media counts as important, is entirely up to you.

Audio/video on dvd may be re-rippable, but not necessarily for home videos and pictures, or saved documents and files.

Storage solutions typically provide ways to protect your data from 1 or more drive failures, either via redundancy (ie raid 5/6 and similar) or mirroring (real time disk duplication) or duplication (copying data to multiple other disks/destinations).

While storage is used by pretty much anyone using PMS, this forum is not really the most appropriate place for storage discussion, there are many other websites and forums dedicated solely to storage and how to decide what is best for your situation and budget.

Keep in mind, storage costs can quickly multiply as your library grows over time.

IMO, the best solution for anyone who is not, or does not want to learn the details/pros/cons of storage is best served by a getting a name brand nas of the size and cost they can afford.

These nas typically have dedicated forums and websites for support and updates.

Depending on nas, some can run plex directly on the nas, others should be used only for media storage (plex run on a different computer).

You can certainly save money by ā€˜rolling your ownā€™, but by doing so you also accept responsibility for all maintenance/updates/minutia that is involved with keeping your storage working.

In the end, the only ā€˜correctā€™ solution is something only YOU can decide based on your budget, tech knowledge, and TIME.

Use google to research extensively.

Some things like different raid softwares, you can try out without committing, as long as you have sufficient extra drives available.

1 Like
4

@buddyboy101
As a long time DrivePool user nothing could be simpler.
That said, as alluded to how much time do you have on your hands. Unraid is fine is you want to get away from Windows and have the time to learn something new. Be that UnRaid, FreeNAS or anything else outside of Windows.

So back to DrivePool.
Install just like anything else in Windows.
You are immediately offered the option to create a pool.
Select your drive or multiple drives or any other drives in the future.
It doesnā€™t matter how much data is already on the drives you lose no data.

Creating a pool from empty drives takes around 15 seconds.
Creating a pool from drives full of data takes around 15 seconds.
Moving existing data inside the pool (which isnā€™t done by default) and takes around 2 seconds for your existing 4TB of data.
From that point you can choose to duplicate some ā€œpreciousā€ stuff, everything or nothing at all.
Once itā€™s all installed you are using nothing other than Windows as you always have.
Forget DrivePool is installed. Unless you want to change settings or create multiple pools you are done. There is no need ever to even look at the DrivePool UI again if you donā€™t want to.

So Iā€™m assuming here you are using Windowsā€¦ do you want to learn something new like UnRaid or just be done within 5-10 minutes?

One other thingā€¦ If a drive dies (and you havenā€™t setup something like SnapRaid) you lose that data. Drivepool will tell you but will carry on regardless. All data on the other drives is still accessible.

For me Drivepool is your obvious choice.
Tech knowledgeā€¦ Well like I said do you want to learn a new OS?
Timeā€¦ Do you have time to learn a new OS?
Budgetā€¦ Assuming you have enough Sata/USB ports on your existing system then a NAS would be the one totally NON viable choice.
Having totally missed your Probox purchase until just now then DrivePool becomes an even better option.

6

Hey buddy, thanks so much for the very practical advice. After reading some other threads and your response, I think DrivePool is the clear winner. Seems like a great application that gets the job done without much fuss. The duplication tool should be good enough for me (as far as redundancy), but Iā€™ll still figure out a good backup method. I just had a baby (which is part of the reason why Iā€™m doing this - so my wife and kid can easily queue up any movie we have via Plex), so very little time or patience to get bogged down in excessive tech. Thanks again!!

7

If you have Windows 10, it has drive pooling built in. If not, DrivePool is great.

8

The makers of Drivepool also offer Cloud Drive. That means you can mount any storage provider to your local system (Google offer unlimited storage for around $10 a month via Gsuite.)
To add to that They also offer an option to make that remote storage part of DrivePool.
That way you can have your files stored locally and the duplication all set to your remote storage.
The files are hashed and encrypted though on the cloud storage provider. If you browse those files outside of Drivepool they will just look like nonsense. (for example with Googleā€™s web app or DriveFileStream.)
Within Drivepool they just look like the original locally stored files.
Oh and the duplication happens automatically so no scheduled tasks to run.

One thing though. If you lose your encryption key you lose access to that remote (cloud) copy for good. But generally though your remote files should be safer when encrypted.
That said your local stored files are left un-encrypted.

1 Like
9

Thanks so much for the info. This sounds like a great back-up solution! Just a couple questions from the following advice you provided:

Regarding making the remote storage part of DrivePool, does that mean the virtual drive will be in the same pool as my 4 hard drives, or would the virtual drive be separate? Is there any reason I would or not want the virtual drive in the same pool as the physical hard drives? Iā€™m leaning towards having them as separate to better identify whatā€™s an original vs. a duplicate file, does that make sense?

Regarding setting the virtual/remote drive as the duplication location in DrivePool, how would I go about designating that? I havenā€™t used the software yet - plan on setting it up this week when the drives arrive.

Last how do I go about configuring automatic duplication? Related, if I modify or delete a file on my physical hard drives, will DrivePool automatically modify/delete the dupe as well?

And one other general DrivePool question, I read that one of the limitations with RAID is that you cannot extend the array without starting over. Is the same true for DrivePool, or can you easily add or upgrade drives over time without starting over?

Thanks so much for your help.

10

Sorry for the delay, itā€™s been along shift at work.,

OK for now lets assume you are just doing everything locally.
Drivepool will by default (but by no means restrictively) add new files to the Pool equally so each drive fills at the same rate. That offers the best option to ensure that everything can be duplicated.
To enable duplication its just a tick box in the UI, but this can be set at any level.
Top Folder, Sub Folder or even a single file. Generally though you want want Top Folder if you choose to duplicate everything.

So that aside lets move on to the cloud stuff.
You create your Cloud Drive for Google or whichever provider you prefer.
DrivePool will display that as just another drive letter as far as windows in concerned.
That drive will initially be empty. However that drive can also be added to the pool. Again Windows thinks its just another ā€œlocalā€ drive.

Lets say locally you have drives D, E, F and G on your local system and you create a Pool.
That Pool is assigned the letter P
The Cloud Drive that you added to the Pool you assigned it as X

Drive P and Drive X are in the same Pool and you set duplication which then causes everything in P to be duplicated to X.
X isnā€™t on your local system, itā€™s in the Cloud. Any Time you add something new to your local drive P will be duplicated to X in the background.
If you are ultra paranoid you could even set duplication on your local system AND also duplicate to the cloud. (3 copies).

I want to add here that all your actual physical drives can have there drive letters hidden for less clutter. If like me you still want access to those individual drives you can still remove the drive letter and create mount points. Basically thatā€™s just a drive mounted as a folder with no drive letter. Itā€™s generally not needed but a great option for troubleshooting or if you just really need to know the exact physical drives where a particular file is.

As for adding new drives. Yep easier still. Connect it, and formatted or not DrivePool will see it and the dashboard will offer you the option to add it to the pool. That will take around 10 seconds of your life.:rofl:

What we havenā€™t yet touched on is multiple Pools.
Personally I have separate ones for Movies, TV Shows and my 4K HDR stuff.
None of it is needed but Iā€™m a little OCD.

Finally not wanting to sound like an advert for Stablebit but they also have Stablebit Scanner.
If a hard drive shows signs of failing it will immediately start moving everything off that suspect drive.

Iā€™m sure you asked me stuff that I havenā€™t quite answeredā€¦ or worse still I totally baffled you with some of it.
But I have a lot more free time tomorrow so feel free to keep shooting the questions. Hopefully I will be able to answer quicker tomorrow (Iā€™m UK based.)

One very final thing.
I have been using Stablebit DrivePool and their other products far longer than the 5 or 6 years I have been using Plex. Both just work!

2 Likes
11

I am also a bit OCD and I find that, in my case, that pretty much forces me to have just one large pool withe directories for each different library. That way when one library grows I donā€™t have to worry about adding a drive to the correct pool I just add a drive to the one pool and let DrivePool work it out.

This is my experience as well. I really think DrivePool is the most stable and reliable piece of software I have ever used.

2 Likes
12

Yep for me it all started when it launched as an alpha when Microsoft announced the end of the Windows Home Server family.
Since that first alpha I havenā€™t had a single issue.

I was offered the software for free for testing.
The dev has no facility for donations.
So I purchase a never to be used licence every year to circumvent the inability to donate.

Sometimes devs (especially individual ones) just really deserve the appreciation.

The main thing about DrivePool though is in its simplest form it just works.
For those with varying levels of complex needs (and OCD), it just works.

1 Like
13

Hey man, thanks so very much for your help and apologies for my delay in responding. Two quick follow-up questions.

Regarding duplication, letā€™s assume thereā€™s a file on the P drive that was duplicated on the X drive. If I later modify the name or metadata (e.g. tags) of that file in the P drive, will those changes be detected and then updated automatically on the duplicate file in the X drive? Likewise if I delete the file from the P drive, will it also automatically be deleted from the X drive?

Regarding the pooling of drives, I want to make sure Iā€™m following. Assuming D/E/F/G are individual drives that are pooled as P, will D/E/F/G appear as one consolidated drive or will I still see them as individual drives? I believe the answer is they will appears as one, but if I were to add X to the pool, wouldnā€™t it appear as combined with P as well since itā€™s part of the same pool? Or am I missing something? Sorry for such basic questions. I imagine playing around with it myself will answer some of this!

14

Iā€™m almost certain the answers to both are yes. Duplication is exactly that. Also If you decided that the whole setup wasnt working for you the workflow would be to remove X from the Pool.
Do not start deleting all the files in X under any circumstances. I would be far happier if you asked this over here though https://community.covecube.com/ just to be on the safe side.
There is also this. Covecube - Wiki

With regards to the second part I probably didnā€™t explain it too well.
DrivePool will (quite rightly) not alter your system.
So if D,E, G and F are in Pool P then Windows will show P as the consolidated drive. But by default Windows will show the individual drives also.
Its safe to simply remove the drive letters for drives D, E, F and G in Disk Management in Windows.
However as I mentioned you still have the option to browse the individual drives via explorer if you use the ā€œMount in the following empty NTFS folderā€. Generally though itā€™s not needed.

15

Thanks again for the help! Iā€™m still stuck on where X fits into the Pool. I imagine the Pool you are describing consists of D/E/F/G + X. How do I tell DrivePool that the contents of DEFG should be duplicated on X and not elsewhere in DEFG? I guess my concern is how do I specify that the dupe files should reside on X, that way if D/E/F/G fail, no dupe files will be lost?

16

it is more like drive X: = d+e+f+g

you only need to access drive X the rest you can ignore

17

I thought P = D+E+F+G? Is X (the google drive where duplicated files should reside) part of that pool or external to it? In other words, can DrivePool duplicate files in a pool to a location outside of the pool?

Also, is there any way to duplicate to avoid Googleā€™s 24 hour ban if you upload more than 750GB in 24 hours? Should I just enable duplication on a folder-by-folder basis (in 750GB groupings) initially to get around this?

1 Like
18

Thatā€™s right @buddyboy101

We are calling our local pool P in my examples.
X does not exists unless you use Stablebit Cloud Drive also.
Whatever storage provider you use it will create a drive letter for the remote mount.
That is X in my examples.
So when that is done and Cloud Drive has created that mount it can also be added to the pool.
To be clear on this itā€™s called hybrid pooling I think. In effect you have a local pool P: and a remote Pool X:.
You can add a pool to a pool and thatā€™s what you do.

Obviously X: is empty as you will have only just created it. You simply set duplication between P (your local pool) and X (your cloud provider mount.)

EDIT: Sorry, we are all cross typing.

The only way you can avoid the upload limit is with team drives in Google (now called shared drives).
Its not something i have ever looked at, at least not on my local Windows system.

19

Oh, well I was using them as examples, and I guess I misunderstood anyway. My X equals your P

1 Like
20

Thank you as always, sir! After reviewing all of the manuals and FAQs, I think I know what I need to do to configure things properly. Regarding setting up the cloud drive, what local cache size do you recommend? I wonā€™t be streaming files from it, just housing the duplicates there. Also have you noticed any performance issues from using cluster sizes larger than 4 KB so that you can have a cloud drive that is larger than 16TB? Thanks!!

21

I only ever used it in a test environment to be honest.
I also have a remote Linux server mounted to Gdrive that was exclusively for my shared users while I used my local one. However after a Windows issue a couple of weeks ago I ā€œtemporarilyā€ switched to using the remote one until I got the local one fixed. But it works too well for me to go back (at least unless Google backtrack on unlimited storage.)

If you arenā€™t streaming then I found cache size didnā€™t matter too much. It definitely did for streaming though.

One other thing whilst I think on. Cloudrive offers a multitude of options for setting the cloud drive size. What actually took me a while to figure out is the fact that it allows you to type in your own custom size that isnā€™t on the list. Iā€™m pretty sure when I tested a set a much higher cluster size.