Wasabi S3 Cloud Integration

Hello everyone. I have been looking into wasabi cloud service. And it sounds like it may work. But I need some help translating their email. And any idea on how to set it up.

Email:
Joseph,

Thanks for reaching out to Wasabi. To store your media content with Wasabi, Plex will need an app / software layer to point to our S3 based cloud. If your Windows, MAC, or Linux desktop app is already integrated with S3, then it will be no problem. Plex would just point to our http:repository. If Plex is NOT S3 integrated, we have partners that will convert the TCP and enable to the two to speak together. The partners are in expensive and the value add you is you still get the back-end Wasabi data storage price of $4.99. Does Plex need a repository for your media content? Hopefully that helps, I can connect us with our Engineer to elaborate

  • James

Hi There! I am actually the Product Manager for Wasabi. Yes: it’s entirely feasible to run a Plex server via Wasabi. You would need a way to mount the bucket as a local drive. I mostly haven’t done it yet since I have a 16TB server at home but I will play around with it next week and report back/make a KB article on our site.

Also @elan if you guys are interested in any kind of partnership let us know ! :wink:

1 Like

So I got a solution! Utilizing lucidlink, you can stream video files from Wasabi on your local plex media server.

they provide info on that here:

https://lucidlink.freshdesk.com/support/discussions/topics/31000000634

in order to do it presently is a little complicated but they are going to make it simplier in the near future.

For now, you can set it up with wasabi by doing the following

  1. create a File space using AWS provider in Your storage, selecting a close region to the object store
  2. stop at the initialization point in the portal when the File space creation is complete
  3. open the LucidLink client to fire up the backend processes (daemon/service)
  4. run the following command (RUN in widows or Terminal in Mac) supplementing the <> file space name (step 1) and your Wasabi credentials, shared secret, bucket name, etc.

lucid init-s3 --fs --password --https --endpoint s3.wasabisys.com --access-key --secret-key --bucket-name

Hope this helps!

1 Like

thanks Richard - we will have native Wasabi / LucidLink integration in our portal in coming days however since this post has been drawn to my attention i felt it important to provide further colour and context to aid those interested in the meantime.

you need to populate the details between and you need to ensure you create a file space however not complete the initialization from the LucidLink portal, we need to send the manual string to initialize:

lucid init-s3 --fs filespace.name --password sharedsecret --https --endpoint endpoint --access-key accesskey --secret-key secretkey --bucket-name bucketname

you run the above command from the command line/terminal once you have installed your LucidLink OS client (Windows, macOS or Linux) http://lucidlink.com/download and have the client open at the “connect to file space” prompt, leaving it prompting.

example if your file space you created in the portal was called my.filespace and want a shared secret of 12345 with your bucket called myfilespace with access/secret credentials of abc123 and 456xyz the command would look like:

lucid init-s3 --fs my.filespace --password 12345 --https --endpoint s3.wasabisys.com --access-key abc123 --secret-key 456xyz --bucket-name myfilespace

Please note: LucidLink is a distributed log structured file system, which encrypts and compresses data on the client side, streams to and from the object store in a data layout which allows us to cache and prefetch data on demand

Data within the bucket will not be accessible directly from the object store without using our LucidLink OS client installed and your shared secret.

I will update this thread once our native support is live!

David

1 Like

Thank you for posting this! Works like a charm :slight_smile:

What about cost of LucidLink? I see that it is $0.05 per Gig, which is 10x more than Wasabi storage and with minimum commitment of 1TB it will be $600/y just for data management, while actual Wasabi data storage will be just $60.

16TB local NAS would cost you about $800 with drives. Plus about $200 in electricity cost if you run it 24x7 and access one file at the time for 8 hours per day.

This is cost of Wasabi for one year of equal capacity with added convenience that you don’t have to worry about noise and failed drives. I can justify this, but it is hard to justify almost $10K/y in LucidLink cost.

Any comments would be highly appreciated.

Currently I’m testing SDFS with S3 backend against my own hosted “minio” S3 server, https://github.com/opendedup/sdfs But it should be the same on Wasabi

Looks pretty good, however I still need to see how it reacts to access to blocks evicted from the cache. Also since Wasabi has 90 days storage minimum, I’m also looking how much data is being rotated by SDFS on S3 backend.

1 Like

Hi tv_tvman-us

I’m one of the founders of LucidLink. We originally envisioned this as an Enterprise targeted service, and it has been architected with that in mind. On the other hand, I must admit that I am also a Plex user with a couple of TB of media that I store in Wasabi.

If the price were more in line with an end user type of use case, on par with Wasabi pricing, would you find it useful? Do you think other Plex users in this community would be interested in it?

Best,
Peter

2 Likes

Hi Peter

I found this thread very useful and interesting. When I compared the S3 prices from wasabi with the LucidLink price, this is nothing that I would pay as a private end-user: LucidLink is way to expensive for a private use case.

From my point of view, I would pay for one of this models

  1. I’m calculating TCO per TB per year 5-10% for the “software”, the rest for the storage price. Example: Wasabi 10TB = 500$ yearly + 25-50$ for “a software” that connects it.
  2. Onetime payment for a unlimited use of storage. Here I can’t name a price at the moment
  3. A yearly subscription for a unlimited use of storage. Maybe with a cap to make sure it is used for plex like 25TB?

With one of this models, I would pay for a LucidLink license, because the TCO for the solution gives me a great value. I know that 20-50$ per year sounds cheap for an enterprise software. Don’t for get that this use case is aiming for end users looking for cheap Plex storage.

Other solutions from a end-user perspective could be CyberDuck. I use it to accessing my files on S3 from my laptop (free of charge). There is a version called “mountainduck” that integrates S3 directly into windowsexplorer (40$/license). I’ll test the setup with Plex in some weeks.

Thanks for the thoughtful reply SnickSwiss, apologies for not seeing this sooner.

I totally understand the pricing concerns when it comes to personal use case - I would also not spend that kind money myself. The challenge for us would be that it costs that much for a reason - we’ve designed the product from an enterprise usage standpoint, with no shared architecture and performance oriented back-end infrastructure supporting it.

Assuming we could simplify the offering, and offer it at a lower price point would this be valuable for you to use?

1 Like

Hi @obkook, I totally understand the design goals and pricing structure and not saying that it has no value. However here we are talking about niche market and specialized application which does not use all the capabilities of the LucidLink solution. What could be done, if LucidLink could fork the core engine and partner with Plex in order to create an S3 accelrator available to Plex Pass users. Not sure about economics and feasibility of the business case, so this is just an idea.

1 Like

There is a small amount of data rotated during the normal operations of the library, on the scale of a couple of MB per month, so that will not make a dent in the Wasabi bill. With 64kb of sdfs block size, nicely aligned with 4kb billing unit, the response time was within reasonable buffering expectations, almost indistinguishable from local storage. The only drawback was that SDFS uses a lot of RAM and it was a bit a problem on my Plex system.

1 Like

I’m still reading through this thread to see where things stand here in 2019 but yes I’d be totally interested if it’s not there.

1 Like