@jclifton said:
Just to follow up with everyone watching. I fail. My CIFS mount to the freenas host I use for storage was dying on the slaves for some reason. Got that sorted out and then @liviy’s solution worked flawlessly.-1 for me today.
You’re very welcome, in the words of Douglas Adams “Share and enjoy” 
hey @wnielson thanks so much for this script. I’ve been pondering a load balancing solution for months! one question i have completely setup your solution. I was wondering if there were any commands to see if the load balancing is actually working? i noticed on one of your prior post you posted a screen shot of transcode data i was wondering how this can be generated. I want to ensure that this is working. Currently I’m monitoring the cpu threads in VMware on my slave machine and I do see the CPU usage increase once a transcode request is started so I assume its working. If there’s a way to monitor this status it would be extremely helpful as I dont want to add more nodes to the cluster if the first one isn’t working haha.
Thanks
Joe
@JohnstonIT said:
I was wondering if there were any commands to see if the load balancing is actually working?
There currently aren’t any commands to verify that it is working (but that would be cool to have actually). However, there is a log file you can look at to see where the jobs are being sent. If you take a look at your prt.conf file, there is an options for "filename": "prt.log". You can either navigate to the directory where the transcoder is (/usr/lib/plexmediaserver/Resources/ on Linux) and you should see the prt.log file, or you can change this configuration option to point to a different location.
i noticed on one of your prior post you posted a screen shot of transcode data i was wondering how this can be generated.
That screen shot was simply taken of htop running on three machines. If you install htop you can view the Plex processes by running htop -u plex.
@segator said:
guys do you know if exist any way to control which user request the transcoding?
To follow up on this, I’ve started working on a new feature/PlexSessionData branch that will make transcode jobs session-aware. You can see here the sorts of data you will be able to access, which includes the Plex user that requested the stream. The plan is to send the session data to an (optional) user-defined script to make decisions before sending the transcode session out to the slave nodes.
@jclifton said:
@liviy said:
@alternativesurfer said:
@archangel00 said:
sounds interesting, would be keen to see how the Windows development come alongJust another monthly “Windows Please!” ping
A windows port is likely to be a considerable way off, I would suggest you look at migrating to linux either to a physical system or a virtual guest (using something like virtualbox). Linux is much more stable & less overhead than windows, even if you’ve never used linux before I would expect you will get it working that way in a much shorter time than waiting for a port.
Virtualbox + Ubuntu 14.04 on my Windows 10 physical works great as a third transcode server. Took about 15 minutes to install/configure and it was golden. Almost as easy as the vmware guests that are the other two transcoders.
I tried spinning up a Linux install but couldn’t get my media folder shared successfully from the Windows machine to the linux VM after messing around for days…
Gave up, will try again in a couple weeks when my frustration level dies back down.
Very interesting and useful project. Thanks for working on this.
Has anyone had any luck spinning up CentOS 7?
@stargazer7 Why wouldn’t this work in CentOS 7? basically this should work on any flavor of GNU/Linux/BSD/Solaris that has Python 2.6-2-7. I also do not see why this wouldn’t work on Windows with file shares, though the script should be updated for windows PATH structuring.
Some thoughts I have:
The tutorial should show people how to use SSHFS instead of NFS as NFS is a pain to setup and insecure if your hosting these trans-coders on the internet.
This tool gave me some ideas of how a client server implementation could be created without relying on any network file systems (NSF/CIFS etc…) where the Server(master) sends transcoding portions directly via sockets to the clients(slaves) which then sends the transcoded portions back to the server(master) which writes the file[s] to be served locally, hence you could send jobs to n boxes that would complete n portions of a transcode and combined by the master, good stuff.
@joshbaptiste said:
@stargazer7 Why wouldn’t this work in CentOS 7? basically this should work on any flavor of GNU/Linux/BSD/Solaris that has Python 2.6-2-7.
This is correct.
@joshbaptiste said:
I also do not see why this wouldn’t work on Windows with file shares, though the script should be updated for windows PATH structuring.
Communication between nodes is done via SSH, which is the major reason it doesn’t work out of the box on Windows. Future versions might remove this requirement, making Windows support easier. Having a Windows developer join in would also greatly help!
@joshbaptiste said:
Some thoughts I have:
The tutorial should show people how to use SSHFS instead of NFS as NFS is a pain to setup and insecure if your hosting these trans-coders on the internet.
There is no reason why SSHFS can’t be used. Anyone comfortable enough to follow the tutorial should be capable of using SSHFS instead. AFAIK SSHFS is implemented via Fuse, which is great, but also another thing to install (and another point of confusion). NFS is pretty common, so that’s why it was chosen for the tutorial. If you volunteer to write a guide on using SSHFS we’d gladly accept it!
@joshbaptiste said:
This tool gave me some ideas of how a client server implementation could be created without relying on any network file systems (NSF/CIFS etc…) where the Server(master) sends transcoding portions directly via sockets to the clients(slaves) which then sends the transcoded portions back to the server(master) which writes the file[s] to be served locally, hence you could send jobs to n boxes that would complete n portions of a transcode and combined by the master, good stuff.
Not having to rely on network shares would be great. Two points on this:
slave access to the original file can be accomplished by pulling the file from PMS via HTTP. We’ve already started working on this in the PlexSessionData branch. This will negate the need to setup network shares for the source media.slave back to the master without using network shares would likely solve many of the issues that we’re running into (such as Gracenote and photo transcoder issues). As you mention, we could go with some custom transfer method, but this isn’t exactly trivial so we’ll need to evaluate options and see what makes the most sense.@wnielson said:Communication between nodes is done via SSH, which is the major reason it doesn’t work out of the box on Windows. Future versions might remove this requirement, making Windows support easier. Having a Windows developer join in would also greatly help!
Not Windows developer per se but I write Python, if we’re talking about secure communications without SSH then I would have to dive a little deeper, PyCrypto etc…, I can investigate
@wnielson said:There is no reason why SSHFS can’t be used. Anyone comfortable enough to follow the tutorial should be capable of using SSHFS instead. AFAIK SSHFS is implemented via Fuse, which is great, but also another thing to install (and another point of confusion). NFS is pretty common, so that’s why it was chosen for the tutorial. If you volunteer to write a guide on using SSHFS we’d gladly accept it!
No issue for me, I can write this up…
@wnielson said:As you mention, we could go with some custom transfer method, but this isn’t exactly trivial so we’ll need to evaluate options and see what makes the most sense.
Yep was just mentioning some cool ideas for the future …
@joshbaptiste said:
@stargazer7 Why wouldn’t this work in CentOS 7? basically this should work on any flavor of GNU/Linux/BSD/Solaris that has Python 2.6-2-7.
The biggest gotcha with platforms like centos is that they are at the core commercially driven linux variants which means that they are quite often considerably far behind the likes of Fedora and Ubuntu due to stability being favored over newer versions of things plus the support base for certain packages being created for them by the community is smaller. The shorter version of that is that you are more likely to run into required dependencies not being available.
That being said, on my list of things to do is to get PRT functioning with a Fedora Core 23 system as a slave. After that I’m happy to look at centos 7.
Cheers,
Liviy
@jclifton said:
@liviy said:
@alternativesurfer said:
@archangel00 said:
sounds interesting, would be keen to see how the Windows development come alongJust another monthly “Windows Please!” ping
A windows port is likely to be a considerable way off, I would suggest you look at migrating to linux either to a physical system or a virtual guest (using something like virtualbox). Linux is much more stable & less overhead than windows, even if you’ve never used linux before I would expect you will get it working that way in a much shorter time than waiting for a port.
Virtualbox + Ubuntu 14.04 on my Windows 10 physical works great as a third transcode server. Took about 15 minutes to install/configure and it was golden. Almost as easy as the vmware guests that are the other two transcoders.
Can you write som kind of guide? my main system got 12k passmark and i would like to run it like that.
Got windows 10.
@Trevlig said:
@jclifton said:
@liviy said:
@alternativesurfer said:
@archangel00 said:
sounds interesting, would be keen to see how the Windows development come alongJust another monthly “Windows Please!” ping
A windows port is likely to be a considerable way off, I would suggest you look at migrating to linux either to a physical system or a virtual guest (using something like virtualbox). Linux is much more stable & less overhead than windows, even if you’ve never used linux before I would expect you will get it working that way in a much shorter time than waiting for a port.
Virtualbox + Ubuntu 14.04 on my Windows 10 physical works great as a third transcode server. Took about 15 minutes to install/configure and it was golden. Almost as easy as the vmware guests that are the other two transcoders.
Can you write som kind of guide? my main system got 12k passmark and i would like to run it like that.
Got windows 10.
I suppose I could do that. Basically I just followed his exact steps for deployment. I’ll work on the write up soon.
@liviy Now that the latest update is available for PMS, are there any special steps needed to perform the update and get PRT back to functional?
@jclifton
i would appreciate some type of step by step guide, from what you said, your PMS is on windows, and your using VM on a 2nd machine? I really think that would help…
@m1e1w1 said:
@jclifton
i would appreciate some type of step by step guide, from what you said, your PMS is on windows, and your using VM on a 2nd machine? I really think that would help…
My PMS is a Ubuntu VM running on top of ESXi 5.5 at the moment. I have another ESXi 6.0 host that has another Ubuntu 14.04 VM as a remote transcoder. In addition, I spun up another Ubuntu 14.04 VM on top of Oracle Virtual Box running on top of my Windows 10 system. That VM is an additional remote transcoder.
I guess I should spend some time this coming week writing this guide down, although it’s basically a vanilla install of Ubuntu on either hypervisor and then following the PRT install guide. – My install of PRT was a bit “special” since I had multiple VLANs to cross as well as NFS and CIFS mounting of various devices since my library is on my Freenas host. This may or may not be helpful to folks so I’ll write it up and ship it to @liviy
Has anybody testes to run plex master on a raspberry pi 2 and have OS X as slave?
Does the script also work for thumbnail generation?!
@Jonn3y said:
Does the script also work for thumbnail generation?!
It does, as long as you make a minor change to the NFS mount configurations to enable read/write (rw) instead of just read (r) for the other mounts.
Is there any word yet on the DigitalOcean auto deployment? I’d be super interested in how to set that up I’ve never worked with DO so I don’t know the first thing about doing anything like that. Or even an AWS auto deployment? Either one of those would be great I don’t have any spare computers laying around that I could do that with.
If I understand this correctly, with a local PMS, you’d have to move the file to AWS, transcode it, then back to PMS for delivery?
