StableBit's Drive Pool Implementation Questions

@HitsVille said:

@jjrjr1 said:

I must have missed that in the documentation…

Yeah i dont think its actually in the Drivepool documentation because it is kind of a hacky way of doing it.
However it was the one of the Stablebit guys who first mentioned it in their forums.
Hopefully it should save you some time.
:slight_smile:

The procedure is documented here:
http://wiki.covecube.com/StableBit_DrivePool_Q4142489

I do not really agree with what @Wiidesire says about duplication. While duplication does use space it is not wasted. Storage is very cheap these days and duplication is almost as good as a true backup for protecting your data. Of course it is not necessary to turn on duplication for every folder and file but rather it should be on for those folders that would be difficult to restore the files. difficult must be defined by each person.

I duplicate all my media and I also store important documents and other files in the pool with duplication turned on for them.

With DrivePool you can choose which folders are to be duplicated and which are not and, for REALLY important files you can even turn on 3X or 4X duplication. But do keep in mind that duplication is NOT backup and does not protect from manual deletion or the like as a backup does.

I duplicate all my media directories and I backup those that I do not have DVDs for like recorded media or where The DVDs have been damaged or lost or the files that have been transferred from VHS tapes.

How you handle your Pool is quite personal but I find storage quite cheap so I tend to use hardware solutions whenever I can and I try to avoid a situation where the loss of a drive will cause me to even have to restore any media files at all.

Remember if duplication is being used and you loose a drive everything using files in the pool will continue to function correctly. That is if I am playing a file called Video1.mp4 in the pool and I loose a drive containing the copy that is being played then DrivePool will switch to the backup copy and continue playing. Often if that happens Plex will timeout but simply restarting playback will continue playing where it was interrupted.

Snapraid runs great alongside Drivepool. As did FlexRaid but they never seem to know their development path from one year to the next.

Gdrive helps too. :slight_smile:

@Elijah_Baley You must be made out of money?

As @HitsVille says DrivePool + Snapraid and an additional Cloud backup is by far the better solution.

Let’s see. I have ~100TB of storage right now. Turning on DrivePool duplication would cost me around 2500€.
SnapRaid on the other hand, I have 7 data drives and 1 parity drive on one of my pools. Which makes the available capacity 87,5% instead of 50%.

As such I have savings of around 1875€. GSuite costs 96€/year and Backblaze ~45€/year (bi-annual plan). So it would take around 13 years to make it even. Not only 13 years to make even but with GSuite + Backblaze I have a double backup + local data loss prevention with SnapRaid.

Well I am gonna use Duplication
The only problem is Drive Pool does not perform duplication to another pool over the network.
They should implement that…

But I am gonna use a disk sync program that will keep both drives are in sync. I can do this over the network and each locations will be Drive Pool drives.

@jjrjr1 said:
Well I am gonna use Duplication
The only problem is Drive Pool does not perform duplication to another pool over the network.
They should implement that…

But I am gonna use a disk sync program that will keep both drives are in sync. I can do this over the network and each locations will be Drive Pool drives.

So you are going to have two pools that duplicate each other? That is closer to a backup solution than DrivePool’s duplication feature. But since you are going to use something to sync the drives it is still not quite a true backup as it does not protect from accidental deletions.

I use DrivePool’s internal duplication that duplicates within the pool mainly for simplicity of the duplication process. Using that I have to do nothing and I only have one system to maintain.

As I said earlier I also have a separate backup that goes to a different machine and I only backup the files I cannot pretty easily recover by re-ripping or another process. So out of my 50tb system I only actually backup about 3.5tb.

Of course that is one of the many advantages of DrivePool. You can use it it so many different ways that most people can come up with a way of using it that works for them.

@Elijah_Baley
Yup That’s the plan.
2 pools running on separate network connected computers with the pooled drive network shared on each.
The sync software will do 2 way synchronization of both Pools (Drive letters on the network). Thus if a back up drive or a production drive fail, they can be replaced and re synced…

Should work.

The only thing I will lose is the ability to still access files when a drive fails until re-synced

I did notice when you do DP Duplication it appears to make a second pool on that machine to house the duplicates.

Stable Bit should allow the ability to select a pool to duplicate to and be able to do it over the network.

But until they do, I think my sync plan will work…

What do you think?

BTW each of my pools are 40TB… LOL

I tried drivepool a while back, seemed very interesting, but I wish there had been a simple way to sync a pool to GSuite directly, as backup solution, without using it alongside 2 or 3 other software products. In the end, I’ve just uploaded it all to GSuite and mounted my drive locally for my server, been working super well for me, didn’t really add any delays and Google now takes care of duplication and backups for me lol

@jjrjr1 I see no reason why your plan would not work but, to me, it seems overly complex but that is your choice and it is entirely possible that my solution may seem overly complex to you.

I have never seen that DrivePool creates a second pool on the computer where duplication is used. My experience is that duplication simply is achieved by DrivePool maintaining two copies of duplicated files within the same pool while assuring that the copies reside on different physical drives so the loss of a drive will not cause the loss of any file.

As far as duplicating over the network goes I really do not see a need but I am just one little person and others may well feel differently.

I do not see any advantage to setting it up the way you describe but you clearly see it differently.

If I were setting up a system with the hardware you describe I would put all 80tb of drives on one computer and pool them and place Plex of the other computer with the pooled drive on the first computer shared on the network. But I use external drives and that might be harder if you use internal drives.

Edit: You know that having DrivePool installed on two different computers will require that you buy two licenses, correct? You can have as many pools as you want on one computer under one license but each computer you have pools on must have its own license.

@Elijah_Baley
I have 2 licenses.
And yes a couple of the HDDs are internal to the backup machine, Making it VERY difficult to move those drives physically upstairs to the PMS box.

I can’t think of any other way, in this environment, to achieve duplicate/mirrored media drives.

Any other ideas are welcome.

Thanks for all your input!!!

@jjrjr1 My suggestion would be, from what you said, to place all drives on the backup machine and pool all the drives and share that pooled drive on the network then on your PMS box attach a drive letter to the pooled drive and use that drive letter for you Plex media drive.

That is basically what I do. One machine on the network running Plex and another acting as a file server running DrivePool and hosting all my media.

On my file server I have a 1tb drive as a boot drive and a 5tb drive internal and the remaining drives (45tb total) are connected to USB 3.0 ports. The 45tb external drives and the 5tb internal are all pooled for Plex media and shared on my network.

I am not saying my setup is best it is just the way I do it and it works very well for me.

Couldn’t you do this by adding StableBit Cloud Drive and the network share provider? Setup a cloud drive on the secondary machine and then add the Cloud Drive to the primary Pool.

@jjross said:
Couldn’t you do this by adding StableBit Cloud Drive and the network share provider? Setup a cloud drive on the secondary machine and then add the Cloud Drive to the primary Pool.

Yes.

Well Project Done!
Drive Pool implemented and Sync software works great.

Just DP being able to span all the drives and merge content is GREAT. Files all organized and easy to navigate now.
The sync software keeps both 40TB DP drives in sync (Except for deletions), giving me a sort of managed backup.

Only issue is some of the folders drive pool left behind are NON-Removable (Access denied)

Never mind.

I use DrivePool on the same computer running both Plex and Emby. I also use SnapRAID as well to “backup” the individual drives used in the pools.

@Elijah_Baley said:

If I were setting up a system with the hardware you describe I would put all 80tb of drives on one computer and pool them and place Plex of the other computer with the pooled drive on the first computer shared on the network. But I use external drives and that might be harder if you use internal drives.

Edit: You know that having DrivePool installed on two different computers will require that you buy two licenses, correct? You can have as many pools as you want on one computer under one license but each computer you have pools on must have its own license.

My use of plex is similar to yours, I think.

I have around 8-9 tb of video on 4 separate external drives attached to a windows 10 machine, where plex lives on the internal drive. No media lives on the internal drive on the windows 10 machine.

I’ve been interested in drive pool, but when you say I’d need a separate computer, I’m deterred. Can I pool the external drives only, and not include the internal drive where plex lives? I’d prefer to avoid the expense of a second computer.

@brucethevideobug said:

@Elijah_Baley said:

If I were setting up a system with the hardware you describe I would put all 80tb of drives on one computer and pool them and place Plex of the other computer with the pooled drive on the first computer shared on the network. But I use external drives and that might be harder if you use internal drives.

Edit: You know that having DrivePool installed on two different computers will require that you buy two licenses, correct? You can have as many pools as you want on one computer under one license but each computer you have pools on must have its own license.

My use of plex is similar to yours, I think.

I have around 8-9 tb of video on 4 separate external drives attached to a windows 10 machine, where plex lives on the internal drive. No media lives on the internal drive on the windows 10 machine.

I’ve been interested in drive pool, but when you say I’d need a separate computer, I’m deterred. Can I pool the external drives only, and not include the internal drive where plex lives? I’d prefer to avoid the expense of a second computer.

Yes you can do it that way. Many people do it that way without any problems.

I just have had problems with Plex and DrivePool coexisting on the same computer. My problems could be due to the VNC (TightVNC) that I use for local remote access. I just found it easier and better for me to run a computer running DirvePool and a separate computer for Plex. I did mot want to troubleshoot whatever the problem was.

Also I STRONGLY believe that a server should have no other duties if at all possible.

BTW: I have a LOT more storage that you (50tb on 11 drives) and that may also contribute to the problems I had with Plex and DrivePool on the same computer.

As a point of info: The problems I had were not crashing. It was that sometimes DrivePool would not start on the computer I was running Plex on. When I separated the two my problems went to zero. However once the computer and all parts of the system started the system was absolutely stable.

No need for a second computer.
You can pool your external drives with Drive Pool.
It is not advised to include the system drive in the pool as PMS has problems with the database in a pool.

Installing Drive Pool was the best thing I did to increase my PMS’s ease of use and organization of my media.

In fact I have Drive Pool set up on 2 different machines. One machine is where my PMS lives and all my Plex media is pooled there.
A second machine has Drive Pool running and has a mirror copy of my PMS media.
I use software called GoodSync to keep both machines as exact copies of my library.
If a drive fails, I just replace it and GoodSync will make them the same again.

BTW my Plex Media Disk is over 45TB on each machine… (5500 movie titles, 30000 TV Shows, 70000 Audio tracks.

This works great for me and my backups.