Distributed transcoding.
Some really useful stuff is being mentioned for sure!
The problem with all of the suggestions we've had so far, (including mine) is if you do something like any of them, we wind up with another un-official third part hack to do what should be part of the PMS tool sets to begin with. (Not trying to insult, Carlo, you know what I mean...)
I'm anxious to see what you are doing, though. While I don't have the user base, I'm getting there with the media base, so I'm sure I'm going to find some use for whatever you have coming along.
GPU transcoding...
Yes agreed, but this isn't as much a "hack" as many other things. Actually if Plex were to incorporate this "feature" into the eco-system they would probably have to do it just like I'm doing it. :)The problem with all of the suggestions we've had so far, (including mine) is if you do something like any of them, we wind up with another un-official third part hack to do what should be part of the PMS tool sets to begin with. (Not trying to insult, Carlo, you know what I mean...)
I'm anxious to see what you are doing, though. While I don't have the user base, I'm getting there with the media base, so I'm sure I'm going to find some use for whatever you have coming along.
GPU transcoding...
That's really a shame but that was your second guess. :)
Actually, somebody did guess it already.
Which of the guesses sounds like something that could be used by bigger Plex systems?
Distributed transcoding
(I know it's been guessed, and this is my second...)
Wait - wait.. no... I got it... it's Transcribulated Decoding... no... it's Defibrillated Hardcoding... no... "Distributed Transcoding."
Yeah... that's it! :)
[Always late to the party]
I’m going to go with some form of Bandwidth Control.
Which would include the ability to limit bandwidth per session, limit number of concurrent streams per user, and force direct play for servers with high upload speeds.
From your posts there is a list of wants you have mentioned, but of those this, for me, would be at the top. So that is why I’m guessing this feature
Sent from my HTC One_M8 using Tapatalk
^^ Doesn't Fragments.Media have a tool that does these things already?
Cayars to your talk about FlexRaid two pages back i can add a bit. Unfortunately the support situation with specific situations on flexraid is not the best, but in my experience only infrequently will you have issues. I have been using Flexraid raid-f for two years...really like the snapshot raid as continuous raid is a bit unnecessary for pretty static media libraries. Its implementation of parity drives is one of the best i have ever seen and was the reason for my purchase. The bad is like you said that the pool is slower than the normal drives...and if you do a lot of transfers/accessing it hits the cpu pretty hard too. Also it doesn't allow for recently added files, so i changed to having the plex library point at the tv or movie folder on each drive...This works really well and still gives you the pool for other uses including transfers. The guy in charge over there also says that in the near future the t-raid pooling system will be ported over to raid-f and it is much much better so that should help that out.
Unfortunately i made the dumb decision to move over to windows 10 and it seems every pooling software is currently broken on windows 10. So that has been quite an issue for the short term.
^^ Doesn't Fragments.Media have a tool that does these things already?
Some of it yes, but the ability to limit concurrent streams isn't there, at least last time I checked
Sent from my HTC One_M8 using Tapatalk
Cayars to your talk about FlexRaid two pages back i can add a bit. Unfortunately the support situation with specific situations on flexraid is not the best, but in my experience only infrequently will you have issues. I have been using Flexraid raid-f for two years...really like the snapshot raid as continuous raid is a bit unnecessary for pretty static media libraries. Its implementation of parity drives is one of the best i have ever seen and was the reason for my purchase. The bad is like you said that the pool is slower than the normal drives...and if you do a lot of transfers/accessing it hits the cpu pretty hard too. Also it doesn't allow for recently added files, so i changed to having the plex library point at the tv or movie folder on each drive...This works really well and still gives you the pool for other uses including transfers. The guy in charge over there also says that in the near future the t-raid pooling system will be ported over to raid-f and it is much much better so that should help that out.
Unfortunately i made the dumb decision to move over to windows 10 and it seems every pooling software is currently broken on windows 10. So that has been quite an issue for the short term.
I may go back and look at it again in the future when he does add this. I'm always open to testing and trying new stuff (actually enjoy it). But for right now I'm happy with DrivePool and SnapRaid combo.
Actually I had posted in another thread I actually just gave both Amazon Drive Cloud and SnapRaid a workout when I thought I had deleted about 300 media files. I used SnapRaid to restore/fix all my movies and used NetDrive2 to access the TV Shows I had uploaded to Amazon Cloud Drive. Both got me all my media back. Then I found I had made a change to the util I was working on and hadn't deleted anything but simple moved the files (Doh moment). So I did a file compare and everything was bit for bit accurate.
Distributed Encoding is the correct answer.
Like I said, I have this working in a prototype mode for Emby which directly uses ffmpeg. Plex uses a modified version of ffmpeg with a different name but should roughly work the same.
In a nutshell it will work similar to Emby and this is the “setup” on that platform at present: To use this presently you would just rename ffmpeg.exe to ffmpegOriginal.exe (or similar) and then drop in a replacement ffmpeg.exe I’ve created. The drop in replacement is basically just a “proxy” that takes the command line options sent to it and hands them off to ffmpeg.exe running on another computer (or the server). It watches stderr, stdin, stdout and shuffles these messages to and from the actual ffmpeg and the drop in Proxy. Plex should not even be aware it’s not “talking” to the real transcoder.
As long as the 2nd, 3rd computer have the same drive mapping as the server to the transcode directory and the media itself everything seems to work. You’ll of course also have to copy the Plex transcoder over to these additional machines from the server.
All of the above presently work (for Emby). Should just need to change names to use the Plex Transcode EXE and it should roughly work (might need a tweak here or there). What I need to do to go from prototype to real working edition is to figure out how I want to monitor and figure out which computer to run the transcode on. This could be as simple as monitoring the CPU use on each computer and using the lowest.
Where it will get interesting is to be able to wake up computers on the LAN via WOL functionality to dynamically bring online additional transcoders when needed.
For Plex this should work for real-time streaming, cloud sync, device sync, Bif/index file creation or any other time the transcoder is called.
I like this approach to scaling out more so then trying to run multiple Plex servers as there is only one DB, one set of meta-data and no need to “sync” data among the servers.
What do you guys think?
Carlo
PS at present it’s only “proto-typed” for Windows machines but i don’t see why I couldn’t do this for Linux and other platforms also. Could be useful for those running Plex on a NAS as they could wake a computer on the LAN to do the transcoding.
I may go back and look at it again in the future when he does add this. I'm always open to testing and trying new stuff (actually enjoy it). But for right now I'm happy with DrivePool and SnapRaid combo.
Actually I had posted in another thread I actually just gave both Amazon Drive Cloud and SnapRaid a workout when I thought I had deleted about 300 media files. I used SnapRaid to restore/fix all my movies and used NetDrive2 to access the TV Shows I had uploaded to Amazon Cloud Drive. Both got me all my media back. Then I found I had made a change to the util I was working on and hadn't deleted anything but simple moved the files (Doh moment). So I did a file compare and everything was bit for bit accurate.
I have started to use netdrive for some media that is less accessed. I am past 20 tb locally and i really don't want to invest anymore in storage. Really liking netdrive just hoping its worth taking the 45 dollar plunge.
Distributed Encoding is the correct answer.
Picky, picky, picky. :)
I’m liking it also but I still get IO errors sometimes when using it with ACD but with a few retries it does work. ACD is still in beta for NetDrive 2 so they are probably still working out some kinks with it. Hopefully it will get better and better.
Carlo
Picky, picky, picky. :)
Actually I meant to type Distributed Transcoding not Encoding. :)
drinehart nailed the answer correctly in any case!
I'm going to go with some form of Bandwidth Control.
Which would include the ability to limit bandwidth per session, limit number of concurrent streams per user, and force direct play for servers with high upload speeds.
From your posts there is a list of wants you have mentioned, but of those this, for me, would be at the top. So that is why I'm guessing this feature
Sent from my HTC One_M8 using Tapatalk
You gave me an interesting thought. I could probably include some of this directly into what I'm working on (Phase 2) since anything about to get transcoded could be stopped outright or changed on the fly to a remux instead of a transcode or could have the bitrate (and resolution) changed. However I don't want to go down that path for a while as that could make debugging issues problematic. I'd have to give this a really good workout before sharing something like this!
I could see how this could be useful for some people however. For example if you had 10Mb upstream and 2 people streaming you could allow 5Mb each. With 4 people you could change the bitrate to 2.5Mb, etc. Don't know how Plex would react to getting a file back different from what was requested of the transcoder but it's something I'll play with down the road. I'll have to definitely put some good logging in at that point.
The bad is like you said that the pool is slower than the normal drives...and if you do a lot of transfers/accessing it hits the cpu pretty hard too.
Why is it slower ? I thought one of the selling points of raid-f was that it ran at disk speed as it did not need to compute the parity in real time.
Why is it slower ? I thought one of the selling points of raid-f was that it ran at disk speed as it did not need to compute the parity in real time.
If you access it through the pool...using scanning on plex it is about 3 times slower than straight disk access
