Cache, Cloud Integration

Is there any plan or goal to do away with what appears to be a straight two layer map/disk cache at ~/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder? Or for that matter the many SQLite databases, although they are atleast optimized a bit.

I’m also interested in a way to integrate a cloud storage provider – not a full fledged service like Dropbox but blob storage with metadata like AWS S3 or Azure Storage… but not like how you do it with either the Cloud Sync or Cloud Server services. You either hook your server into the cloud for backup or possibly serving media but it has to go through your server to go to the viewer, which is suboptimal. The other way, which is kind of cool, is the new instanced Plex servers. I’d rather not get to the point where I have clusters of Plex servers, I deal enough with crap like that as it is, so I’m interested in ways to sort of extend the limits god and Intel placed on the upper limits of storage, concurrency and bandwidth.

So I’d like to keep my server running, but offload some of the work. I’ve been playing with the media.play webhook (wish there was a media.added and media.deleted one) to trigger a download of that same video once nobody is watching, once downloaded it is sent to an Azure Storage collection/bucket (but you could do the same with AWS,etc.) I have a seperate trigger that automatically transcodes any videos that arrive into a widely supported MPEG-4/ AAC LC combination that I feel provides the minimal chance for trancoding on the fly to be necessary and without losing much/any quality.

I could then, in theory, offload this to a CDN (also automatically) and post a API call back to the plex informing it of the availability of this asset. A CDN might not even be necessary, you could serve it straight from the blob container as well and that might be good enough. Then my server could serve a CORS link to stream that video file from the CDN/blob – it might have to take over if they asked for an encoding I didn’t provide for in advance or burn in subtitles, but I’m not sure how often that would happen In any case, all my Plex would have to do is serve the HTML. I realize this hybrid option wouldn’t be the biggest priority with you guys since there is no real service for you to provide and therefore no way to get your beak wet, but I do think it is the coolest way to potentially integrate cloud services with Plex.

At the simplest if you use a container that only Plex has access to you could even cache the metadata Plex-side and have very minimal LIST and PROPFIND requests to the blob container, which is key to affordable blob storage use, not to mention it makes it more snappy. Alternatively you could setup a trigger via either AWS Simple Notification or Azure Queue to inform Plex of a metadata/state change and then the Plex could refresh if you’d like to have concurrent access to other things while still being able to maintain a local cache.
.

Some super interesting ideas in here; I think if I had to summarize, sort of a smart edge CDN for Plex Servers?

Just as a followup question, for the transcode bit, are you positing that building your own encoding pipeline will be much more cost effective (as opposed to e.g. Amazon’s Transcoder which would be like $3/movie)?

Sort of I guess. Mainly as a way to get more storage out of a server that is a thousand miles away, the hourly remote hands rate at my ISP makes Amazon look like a charity. Also, I have a fair bit of credit with Azure, although I actually prefer working with AWS.

As far as the transcoding bit…no, not really. That was just me playing with some of the features of a cloud platform to see the viability of automatically standardizing the encoding of all the videos my wife puts on the plex – It’d please me if they were in a format most people could Direct Play without transcoding at all, but that is optimistic I bet considering subtitles and stuff. I definitely don’t think making your own transcoding pipeline capable of real-time video on demand is at all close to feasible. The amount of overhead you’d need to have… the cloud platforms are kind of purpose built for that kind of bursty spin up an instance for a little while activity – I think your Plex Cloud instances are very innovative that way, and couldn’t imagine the datacenters you’d need to fill to implement a system like that internally.

That said, I DO know how to make a small fortune using AWS… you just have to start out with a very large one! Ahaha, thanks, please tip the waitress.