Plex Remote Transcoder - A distributed transcoding backend for Plex

Dev Community Dev/API Corner
401

@wnielson said:

@jclifton said:
Which logs should I be pulling for ya?

Check in /tmp/prt.log first. Then check in the Plex Media Server.log. Finally, if none of that is useful, open up your prt.conf and make sure that you log level is set to DEBUG–this will allow PRT to capture output from ffmpeg.

Nothing useful in the Plex Media Server log file. Debug was already enabled on prt.log. Here’s a snippet:

2016-10-04 13:38:38,319 - prt - INFO - Host with minimum load is ‘192.168.0.21’

2016-10-04 13:38:38,320 - prt - INFO - Using transcode host ‘192.168.0.21’

2016-10-04 13:38:38,637 - prt - INFO - Debug mode - enabling verbose ffmpeg output

2016-10-04 13:38:38,637 - prt - INFO - Launching transcode_local: [‘/usr/lib/plexmediaserver/plex_transcoder’, ‘-codec:0’, ‘h264’, ‘-codec:1’, ‘ac3’, ‘-ss’, ‘343.597992’, ‘-noaccurate_seek’, ‘-i’, ‘/media/freenas/TV Shows/Blindspot/Season 02/Blindspot - S02E03 - Hero Fears Imminent Rot.mkv’, ‘-map’, ‘0:0’, ‘-metadata:s:0’, ‘language=eng’, ‘-codec:0’, ‘copy’, ‘-map’, ‘0:1’, ‘-codec:1’, ‘aac’, ‘-ar:1’, ‘48000’, ‘-channel_layout:1’, ‘5.1’, ‘-b:1’, ‘774k’, ‘-f’, ‘matroska’, ‘-avoid_negative_ts’, ‘disabled’, ‘-map_metadata’, ‘-1’, ‘-map_chapters’, ‘-1’, ‘-’, ‘-start_at_zero’, ‘-copyts’, ‘-y’, ‘-nostats’, ‘-loglevel’, ‘verbose’, ‘-loglevel_plex’, ‘verbose’, ‘-progressurl’, ‘http://192.168.0.6:32400/video/:/transcode/session/qyoqe09l5w62udazyrleah5mi/progress’]

2016-10-04 13:38:38,763 - prt - INFO - Transcode stopped on host ‘192.168.0.21’

402

@jclifton said:

@wnielson said:

@jclifton said:
Which logs should I be pulling for ya?

Check in /tmp/prt.log first. Then check in the Plex Media Server.log. Finally, if none of that is useful, open up your prt.conf and make sure that you log level is set to DEBUG–this will allow PRT to capture output from ffmpeg.

Nothing useful in the Plex Media Server log file. Debug was already enabled on prt.log. Here’s a snippet:

2016-10-04 13:38:38,319 - prt - INFO - Host with minimum load is ‘192.168.0.21’

2016-10-04 13:38:38,320 - prt - INFO - Using transcode host ‘192.168.0.21’

2016-10-04 13:38:38,637 - prt - INFO - Debug mode - enabling verbose ffmpeg output

2016-10-04 13:38:38,637 - prt - INFO - Launching transcode_local: [‘/usr/lib/plexmediaserver/plex_transcoder’, ‘-codec:0’, ‘h264’, ‘-codec:1’, ‘ac3’, ‘-ss’, ‘343.597992’, ‘-noaccurate_seek’, ‘-i’, ‘/media/freenas/TV Shows/Blindspot/Season 02/Blindspot - S02E03 - Hero Fears Imminent Rot.mkv’, ‘-map’, ‘0:0’, ‘-metadata:s:0’, ‘language=eng’, ‘-codec:0’, ‘copy’, ‘-map’, ‘0:1’, ‘-codec:1’, ‘aac’, ‘-ar:1’, ‘48000’, ‘-channel_layout:1’, ‘5.1’, ‘-b:1’, ‘774k’, ‘-f’, ‘matroska’, ‘-avoid_negative_ts’, ‘disabled’, ‘-map_metadata’, ‘-1’, ‘-map_chapters’, ‘-1’, ‘-’, ‘-start_at_zero’, ‘-copyts’, ‘-y’, ‘-nostats’, ‘-loglevel’, ‘verbose’, ‘-loglevel_plex’, ‘verbose’, ‘-progressurl’, ‘http://192.168.0.6:32400/video/:/transcode/session/qyoqe09l5w62udazyrleah5mi/progress’]

2016-10-04 13:38:38,763 - prt - INFO - Transcode stopped on host ‘192.168.0.21’

Didn’t change anything… tried playing it again for the 10th time or so and it started working. Am I missing something here?

Edit: It appears that it succeeded when trying the other of my two remote transcoder systems. Time to investigate what’s up with .21 vs the working .20 host.

Edit: Found the issue. Looks like the mount point for the .21 host wasn’t correct. The load algorithm kept kicking things over to the .21 host until that one random time it found .20 host. Did a quick check of the mount points and noticed the data wasn’t available for the media files. Updated fstab and things are happy.

Thanks for all the help! Donation location to show thanks?

403

I was wondering if I could take the discussion in a different direction

Power users that would want distributed transcode might be interested in this concept

Running plex as a VM or docker… What if it could spawn a transcode VM/docker for each transcode request. And even better, if it would spawn a session that was right sized processor wise for what needs to be done. IE if a 1080p needs to be transcoded, launch a 2cpu container, if a 4K needs a session launch a 4 core container.

The cloning or spawning of a new instance would be almost instant and plex could perhaps direct stream a pre roll while the it’s spinning up…

At least with docker when the transcode is finished the docker session ends and the container deletes … Stars new each time a session is needed and since they run as clones would use a lot of shared resources

Esxi is very good at scheduling resources but having all transcoder sessions running all the time even at idle would still que up and take some resources and if you have to make them for worst case… Let’s say 4cores… But 90 percent of the time you only need 2 cores, cpu wait times will increase as the more vCPU that a VM has the harder it is to schedule…

Anyone think that this project might be able to fork into something that could do this…?

Thoughts

404

@jclifton said:
Edit: Found the issue. Looks like the mount point for the .21 host wasn’t correct. The load algorithm kept kicking things over to the .21 host until that one random time it found .20 host. Did a quick check of the mount points and noticed the data wasn’t available for the media files. Updated fstab and things are happy.

Awesome, glad to hear you got it working!

Thanks for all the help! Donation location to show thanks?

Check the bottom of the README :slight_smile:

405

@dragonmel said:
Running plex as a VM or docker… What if it could spawn a transcode VM/docker for each transcode request. And even better, if it would spawn a session that was right sized processor wise for what needs to be done. IE if a 1080p needs to be transcoded, launch a 2cpu container, if a 4K needs a session launch a 4 core container.

Some people have been working on that. See here and here.

406

@wnielson thanks…

the mother of invention is to get there before the other guy… I am always late… hahaha

edit…

just looked at both of those links and doesnt look like either have anything really working

what is the nearest thing to this right now… just running VMs and alwasy have at least one PRT sitting idle?

407

Anyone try the latest prt with plex 1.2.2.2857-d34b464?

408

@boboki said:
Anyone try the latest prt with plex 1.2.2.2857-d34b464?

Yes and I’m about to make a post about it…news isn’t good…

409

Important notice about PMS V1.2.x and PRT

The current version of PRT isn’t compatible with PMS V1.2.x.

It seems with the 1.2 releases of PMS and their splitbrain function that they’ve introduced they’ve also apparently introduced a new incompatibility issue with PRT. This seems to be only myself that’s found this so far but my testing of the latest versions of PMS tend to be correct. Please do not upgrade unless you’re happy rolling back to a previous version of PMS if you also find problems. If you want to play safe then I’d advise sticking with 1.1.4.

We’ll look into this issue but as always when new features or major code rewrites happen with PMS which cause PRT to not function with it we don’t have a timeframe for a fix if at all. Unfortunately Plex don’t consult with us or tell us the specifics of their changes so we have to reverse engineer PMS and try to adapt PRT to continue working with it.

Cheers,
Liviy

410

@stargazer7 said:
Has anyone had any luck spinning up CentOS 7?

Yeah, I have several Centos 7 Plex installs running. I have installed all of mine using the minimal version and then just adding the few things I need to get the media shares working. Super easy once you have done it a few times. The hardest part is getting the Windows shares to work by hostname. : )

This is the entire command history from the first run of the VM after installation from inception.

    1  yum -y install samba-winbind samba-winbind-clients pam_krb5
    2  vim /etc/samba/smb.conf
    3  nmcli c modify eth0 ipv4.dns 10.200.1.2
    4  nmcli c down eth0; nmcli c up eth0
    5  yum install samba samba-client samba-common
    6  firewall-cmd --permanent --zone=public --add-service=samba
    7  firewall-cmd --reload
    8  ping mediaserver1
    9  systemctl start winbind
   10  ping mediaserver1
   11   yum install samba-client samba-common cifs-utils
   12  service network restart
   13  ping mediaserver1
   14  systemctl status winbind.service
   15  ping 10.200.1.1
   16  ping mediaserver1
   17  vim /etc/sysconfig/network-scripts/ifcfg-eth0
   18  service network restart
   19  ping mediaserver1
   20  vim /etc/resolv.conf
   21  vim /etc/nsswitch.conf
   22  service network restart
   23  ping mediaserver1
   24  vim /etc/fstab
   25  mkdir /media/adult
   26  mount -a
   27  ls /media/adult
   28  yum -y install wget
   29  wget https://downloads.plex.tv/plex-media-server/1.1.4.2757-24ffd60/plexmediaserver-1.1.4.2757-24ffd60.x86_64.rpm
   30  yum localinstall plexmediaserver-1.1.4.2757-24ffd60.x86_64.rpm
   31  service plexmediaserver start
   32  chkconfig plexmediaserver on
   33  vim /etc/firewalld/services/plexmediaserver.xml
   34  firewall-cmd --permanent --add-service=plexmediaserver
   35  firewall-cmd --permanent --zone=public --add-service=plexmediaserver
   36  systemctl restart firewalld.service
   37  systemctl start plexmediaserver.service
   38  top
   39  cd /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/
   40  cd /var/lib/plexmediaserver/Library/Application\ Support/Plex Media Server/
   41  locate
   42  yum install mlocate
   43  updatedb
   44  locate plexmediaserver
   45  cd /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/
   46  ls
   47  cd Plug-ins
   48  ls
   49  wget https://github.com/dagalufh/WebTools.bundle/releases/download/2.2/WebTools.bundle.zip
   50  unzip
   51  yum install unzip
   52  unzip WebTools.bundle.zip
   53  ls
   54  rm WebTools.bundle.zip
   55  ls
   56  vim /etc/firewalld/services/plexmediaserver.xml
   57  systemctl restart firewalld.service
   58  top
   59  sudo chown plex:plex ./WebTools.bundle -R
   60  top
   61  ls -la
   62  top
   63  ls -la
   64  pwd
   65  top
   66  pwd
   67  top
   68  history
411

@confusingboat said:
I run a highly virtualized Hyper-V/Windows/Active Directory environment and I don’t want any non-Windows production machines unless it’s an appliance as it adds to the management overhead.

Yeah, I used to think the same thing when I was a Linux noob and everything we ran was Windblows. Everything I do in Windows now is so clunky feeling, even PS. The only thing Windows is good for at this point is the Desktop/visual experience/gaming and Hyper-V/remote desktop. If I could find a good alternative for Outlook/Exchange/Hyper-V that was as visually appealing and easy to setup, I would stop using Windows Server altogether.

Now that Mono/.Net core is officially a M$ thing, there is almost no reason to run anything on Windows Server anymore except the above. It’s such a bloated mess for anything else that isn’t Windows specific. They are getting better with Nano Server, but they are late to the party, Linux has been doing this for years. If there is a Windows and a Linux version for something, you should always choose Linux. That’s the difference between Windows and Linux, Windows you have to constantly manage, you install something on Linux and it just works.

I also want to point out there is very little difference between an appliance (this is just a small server) running Linux versus a server running Linux. If you can do it on one or have people that can, then you can easily do it on the other.

412

@dragonmel said:
@wnielson thanks…

the mother of invention is to get there before the other guy… I am always late… hahaha

edit…

just looked at both of those links and doesnt look like either have anything really working

what is the nearest thing to this right now… just running VMs and alwasy have at least one PRT sitting idle?

I’m not sure about the Docker situation since I don’t use that. However, I do know that a few people (myself included) have gotten this working on VPS services like DigitalOcean. You can see this wiki post for details on using PRT on DigitalOcean.

413

I took a look at that article and was having trouble trying to figure out how to adapt it for an esxi box

Only reason I was thinking docker was that esxi is bringing on board with native support and a docker can spin up much faster, almost instant vs a VM

414

Any news about PRT and latest plex version guys? :wink:

Keep up the good work, much appreciated… Thanks!

415

Just tried to get this setup. My Linux skills are just not high enough right now. lol

416

Still waiting for news on latest. Love me some prt!!!

417

Before I go down the rabbit hole, can you put more than one slave node on this? I am trying to do this with one master and 3 slave mac mini’s.

418

Yeah, you can have as many slaves as you want. I currently run one master and 2 slaves.
You would just a prt add_host on the master, and then basically duplicate each of the settings you did for the first slave (exports and all that) for each additional slave you make.

Fun stuff after you have multiple slaves, you can su plex on the master then do prt get_cluster_load to see the load on each box, and prt sessions to see which boxes are transcoding what. very helpful but you need to run those from the plex account (which has no password) so you have to su plex to get there.

419

Thanks for the reply!!! Down the rabbit hole I go!

I am new to Linux so this is a bit challenging. How do I find what port my slaves are on?

420

It should be all in the install guide/walk through.