distribute transcoding/cpu intense tasks to other devices

Feature Suggestions
1

just wondering if it would be a good idea to have an option to distribute transcoding and other heavy tasks to other devices, can be other servers or just plex client app.

 

so with that enabled, the plex server don't have to be powerful, it can a very simple low power nas like wd/seagate ones. all the server does is storing the file and creating sort of index (library), and maybe downloading metadata as well. after all, severing on 'local' hard drive (on the same device) is always more reliable.

 

ok, what above i believe has been mentioned already, what i want to say is, instead of having a 'slave' server, can we distribute those transcoding to other plex clients, such as plex home theater or tv boxes.  it's just like having a build in hidden slave server in every client. those devices are usually quite powerful and should not have any problem transcoding videos. if there's any license issues on android tv boxes for audio, i believe audio transcoding is quite easy and *may* be able to be handled on wd/seagate like Nas's?  plex, the benefit of utilizing tv boxes is that those tv boxes are usually always powered on, and consumes only little energy while idle. and we don't have we build another full blown 'slave' server. 

 

 

 

 

in this way, when we play some video on mobile devices (iOS, web), ideally the wait time for transcoding should be much less, coz more devices are participating the transcoding, i.e., more computational power. and we can have plex server running (by running,  i mean can also support iOS/web transcoded playing) on low cost/power devices.

really, sometimes we are having a much much more powerful tv box than our NAS, and those boxes are just sitting there doing nothing.

 

 

 

 

the few key points of this feature.

1. easy to set up, all functions are build-in, just need to be checked to be enabled.

2. automatically distribution. maybe having 'heartbeat', every clients within LAN reports (together with their cpu ability, so server can know which clients are too slow to distribute transcoding to) to server and register periodically; and server pings every 'fast' client when start playing and transcoding is needed, and ignore high latency clients (e.g., ping > 100), so we don't spend too much time on network transcoding, and also rules out some VPN clients that 'seems' to be within LAN.

3. when transcoded data converges to server, all server does is pack them together and send to the client (ios/web) playing. 

 

2

Already requested. Distributed Transcoding/Scaling

3

Take a look at what Cayars (Carlo) is talking about in this post.

https://forums.plex.tv/topic/131308-cayars-setup-walk-through-and-some-tips-and-tricks/page-9#entry933644

Many of us have been bouncing ideas off him for a week or so now...  (Ok, closer to 2 or even 3 weeks.)  This is the guy with the system that can actually make something like this a reality the soonest, outside of the Plex Dev team and maybe one or two others.  He's got the experience, equipment and the need to make this work.

If it's going to come any time soon, it's going to come through Carlo, I'm sure.  The Dev Team has had their hands full with music and security, lately.

Read through that whole thread.  He has some great info in it, even for a little system sharing out 20-30 movies. 

4

not it's not.   i've read this post already, and the main difference is, instead of having dedicate slave servers, we build them right into certain plex clients. and have all work done automatically. this means the more plex clients (not servers) we have, the more powerful plex network we get.

5

Sorry, missed that part about using the clients to do it.  Nice idea but not very practical.  Plex uses FFMPEG to perform the transcoding.  Since there isn't version of FFMPEG for all clients, this just wouldn't work.  Plex would need to come up with a whole new transcoding system.

6

You wouldn’t want the clients doing the transcoding unless they also were beefy computers or maybe NUCs running PHT with free CPU cycles to spare for this.

7

not it's not.   i've read this post already, and the main difference is, instead of having dedicate slave servers, we build them right into certain plex clients. and have all work done automatically. this means the more plex clients (not servers) we have, the more powerful plex network we get.

Problem is the clients would need to be local to the server (Same network for fast access to the file to transcode it) and most traditional 'clients' would be useless at this, the only clients this could work with would be standard PC clients, which you could just run the transcoding slave plex on that PC too, for the same result.

Clients outside the local network can't help with the transcoding as the network would be a huge bottleneck. And the only useful clients for transcoding besides PCs which is what the linked thread talks about, would be phones. But your phone is

A ) not on the local network usually

B ) in-use or otherwise locked

C ) battery powered, so not wanting to be transcoding stuff a lot

It's a really neat idea, but 90% of plex clients are not fit for transcoding files, and the other 10% are PCs which is what the previous thread suggests.

The slave server idea, isn't as you suggest building whole other slave PCs for plex, but rather just installing something on your usual home pcs that are sitting around idling with spare CPU cycles. Even NAS boxes as you suggest could be running as a slave server. You might have 3 NAS boxes which each could only transcode 1 file max, but then thats 3 files transcoded at the same time plus your main servers amount. It makes these otherwise 'useless' plex boxes add up to be useful.

8

Bingo

9

Sorry, missed that part about using the clients to do it.  Nice idea but not very practical.  Plex uses FFMPEG to perform the transcoding.  Since there isn't version of FFMPEG for all clients, this just wouldn't work.  Plex would need to come up with a whole new transcoding system.

ahh... that i don't know, but i guess maybe plex should have a new transcoding system? i mean it just completely ignores the existence of GPU, i have a 295x but still, plex uses only cpu.   i understand they every device has gpu, but maybe an adaptive system?

i don't know if there is a license issue here, but as you can pretty much install PMS on any device, i assume plex can also include ffmpeg in clients as well?

10

Problem is the clients would need to be local to the server (Same network for fast access to the file to transcode it) and most traditional 'clients' would be useless at this, the only clients this could work with would be standard PC clients, which you could just run the transcoding slave plex on that PC too, for the same result.

Clients outside the local network can't help with the transcoding as the network would be a huge bottleneck. And the only useful clients for transcoding besides PCs which is what the linked thread talks about, would be phones. But your phone is

A ) not on the local network usually

B ) in-use or otherwise locked

C ) battery powered, so not wanting to be transcoding stuff a lot

It's a really neat idea, but 90% of plex clients are not fit for transcoding files, and the other 10% are PCs which is what the previous thread suggests.

The slave server idea, isn't as you suggest building whole other slave PCs for plex, but rather just installing something on your usual home pcs that are sitting around idling with spare CPU cycles. Even NAS boxes as you suggest could be running as a slave server. You might have 3 NAS boxes which each could only transcode 1 file max, but then thats 3 files transcoded at the same time plus your main servers amount. It makes these otherwise 'useless' plex boxes add up to be useful.

as about local network, like i said we can just register (their transcoding power) periodically and ping each one before start, and then rule out those ping>100.(or whatever). 

and I'm not saying to use clients on mobile devices. those devices can be ruled out when they register to the server (according to their power status, device id, cpu speed or whatever), and we only distribute transcoding to those 'stationary' devices, like tv boxes, or (if possible) ps/xbox.

i understand that slave server may not be 'doing nothing but transcoding', but really, most people just have one server at home the most right? there is no point having like three separate servers at home (for most people). but certainly more people have those set-the-top tv boxes at home for plex. not necessarily a nuc, it can be other stuff, like arm device, or those new x86 tv sticks. 

i mean I'm not like against 'slave server', just saying we can build it in some certain clients (certainly not iPhone client for example), this maximizes the efficiency right? and by building it in, it's really much easier for average users to enable. 

_______________________________________________

also, in a few years we will be watching 4k contents, by then what we will do? by a 4k tv and a 4k-ready tv box and start downloading 4k movies. but then, our servers are no longer capable for 4k-720p transcoding on the fly. wouldn't it be easier to distribute transcoding to the new 4k-ready tv boxes than adding a new server for transcoding?

11

You wouldn't want the clients doing the transcoding unless they also were beefy computers or maybe NUCs running PHT with free CPU cycles to spare for this.

well, not necessarily nuc, there are a lot of arm tv boxes out there. and really, most of the time, the tv boxes are just sitting there doing nothing. it doesn't have to be super powerful, but it can help

12

I was thinking more along the lines of the Intel NUCs that have i3,i5 or i7s in them. :slight_smile:
I might worry however about heat in a NUC since they don’t have the best cooling.

Carlo

13

I'm curious to know what device you envision being able to do this with an ARM processor in it?  I know Atom's aren't able to RT transcode.  (depending on the reso and bitrate...)

14

Who were you asking Mike?

I know I myself wasn’t thinking about ARM procs. However, with the right coding logic it “could” be done. Let’s suppose that an ARM NUC can only do 2/3 speed of realtime but the user had 3 of them. The 3 of them together could possibly handle 2 full users. This depends on a few factors that we won’t know until we can do some testing to find out how long each segment is that plex gives to the transcoder and how far ahead it trys to stay, etc. But I doubt this will work. I think each “transcoder” will need to be at least 1x speed.

Carlo

15

Sorry, was aiming the question at the OP...  You must have snuck in there while I was writing the question, and didn't see your comment...   :)

Yeah, I've been looking on line and haven't found any reliable stats for ARM devices that aren't phones or tablets...  The stats just don't seem to point to being capable of doing the job any better than my old NAS was...  :/

16

I'm curious to know what device you envision being able to do this with an ARM processor in it?  I know Atom's aren't able to RT transcode.  (depending on the reso and bitrate...)

well, first of all atoms are capable of real time transcoding.   the cpu in synology 15+ is an atom and it just capable. though one stream will cost ~80% cpu

as for arm, it doesn't have to be i5 or i7 like powerful, it just need to be powerful to 'help'.

17

Who were you asking Mike?

I know I myself wasn't thinking about ARM procs. However, with the right coding logic it "could" be done. Let's suppose that an ARM NUC can only do 2/3 speed of realtime but the user had 3 of them. The 3 of them together could possibly handle 2 full users. This depends on a few factors that we won't know until we can do some testing to find out how long each segment is that plex gives to the transcoder and how far ahead it trys to stay, etc. But I doubt this will work. I think each "transcoder" will need to be at least 1x speed.

Carlo

that's right,   the 'client' device doesn't have to be powerful enough for real time transcoding, but it can add to the total computational power

18

could this be accomplished using something along the lines of a distributed computing client on the local networked computers?

Like a Plex helper app that could be installed on other PCs in the local group that would be able to report availability to the server so the server could offload transcoding when needed?

This would solve the issue of which devices are available and the App could determine which PC is not doing anything CPU intensive at the moment.

This would be a good option for those who are running an NAS version where transcoding is not available.

19

Yep, that’s kind of the idea.

20

could this be accomplished using something along the lines of a distributed computing client on the local networked computers?

Like a Plex helper app that could be installed on other PCs in the local group that would be able to report availability to the server so the server could offload transcoding when needed?

This would solve the issue of which devices are available and the App could determine which PC is not doing anything CPU intensive at the moment.

This would be a good option for those who are running an NAS version where transcoding is not available.

well, helper, server, client or even assistant, it doesn't matter what you call it right?

my point is,

1. it doesn't have to be a ''server/NAS'', or full blown PC to offload transcoding, other devices, like openelec nuc, or even powerful arm tv boxes could also contribute

2. there has to be some sort of selection, like every 'helper/server/client' or whatever reports it's own computational power and (you made a good point) cpu idle status, as well as LAN connection status.