Plex Remote Transcoder - A distributed transcoding backend for Plex

Hello everyone, I used 2 physical machines with ubuntu server and prt. I can see the transcoding process on the slave for each streaming but it is not working on the server. Even if I added 127.0.0.1 via prt add_host all transcoding session keep going to the slave. Any idea ?

@archangel00 said:
sounds interesting, would be keen to see how the Windows development come along :slight_smile:

Just another monthly ā€œWindows Please!ā€ ping

@matdesmarais said:
@an3k I tried to decompile and ended up with lots of unreadable code.

Idealy Plex should take on this development. Anyone know the best way to approach them about this?

You are correct, they should, especially since the transcoder is already a separate process and could easily be made to run from slave nodes by them without much effort on their part at all. Basically, they are already creating slaves nodes, just locally on the same machine. The fact that they still have not made the Windows version run as a service instead of the dinky tray application probably precludes that they won’t be getting to it anytime soon.

The problem stems from the fact that the code base came from XBMC which runs from a single computer or instance, so that is likely their target market and most of the casual users with small libraries make up the majority so these more advanced pro like features are way less of a priority, if at all. If it could be added as an extra paid pro addon or something they might consider it.

They have done a great job so far with it, I would just like to see it taken to the next level. With x265, the need to spawn new trancoding nodes is even greater.

@segator said:
guys do you know if exist any way to control which user request the transcoding? i would like to deny transcoding to some users or atless no video transcoding only audio and remuxing.
and If you finally relĆ©ase your script to digitalocean autodeployment machine if you want i can make docker’s to be very easy installation for the mass

Hey,

Unfortunately as far as I know the decision process for how/what gets transcoded is taken care of prior to prt gets its hands on the stream so any decisions made by prt is likely to be considered an error by plex as it expects things to be transcoded if the request has been made and thus likely to throw up an error if you start forcing (if at all possible at that point as) direct streaming. I could be wrong but even if I’m wrong it would likely be quite a chunk of r&d to achieve it. It’s not an option I have really thought about personally at this point. I’m happy to put it in a list of things to consider but until either I look into it more or someone points out something which will help achieve it id say it would be unlikely considering what I’ve said and how plex is currently built.

@igowas said:
Hello everyone, I used 2 physical machines with ubuntu server and prt. I can see the transcoding process on the slave for each streaming but it is not working on the server. Even if I added 127.0.0.1 via prt add_host all transcoding session keep going to the slave. Any idea ?

Hey,
That’s quite unusual, from when I first started pulling plex & prt apart trying to get things to work I found the opposite. As in when I played with the code and broke things the stream would always default to local rather than to the slave. My initial guess would be that because what prt does is check to see which system had the lowest amount of load on it then allocate the transcoding stream to that system it’s likely your slave is less loaded that the master. Prt takes into account how many cores each system had when it does its decision. If you search of prt.log you should see it state that it’s deciding to use a system on a load basis. If you have multiple streams going with the master in the config as a slave (127.0.0.1) it should use it as well. On each system if you run ā€˜prt get_load’ even with no transcodes running you should get 3 numbers which are a percentage out of 100. Is the remote slave less than the local slave aka the master?

@alternativesurfer said:

@archangel00 said:
sounds interesting, would be keen to see how the Windows development come along :slight_smile:

Just another monthly ā€œWindows Please!ā€ ping

A windows port is likely to be a considerable way off, I would suggest you look at migrating to linux either to a physical system or a virtual guest (using something like virtualbox). Linux is much more stable & less overhead than windows, even if you’ve never used linux before I would expect you will get it working that way in a much shorter time than waiting for a port.

I actually ran both side by side utilizing the same media storage and for whatever reason the Linux version was always behind or wouldn’t find certain items compared to the Windows version. Of course, that was early on when they first released so hopefully it does a better job now as it seems their core development is Windows. Maybe I will spin up another instance and see how it works out now to compare.

@matdesmarais said:
@an3k @saitoh183 I have a exe drop in replacement for plexnewtranscoder.exe that currently captures all flags passed to it and dumps them to a log text file for analysis.

Combine this with FFServer/FFMPEG and you probably have 99.99% of the code.

I am sure all they did was wrap the code to convert commands from PMS to FFServer CLI options. Hell, it might even just be a straight compile and they are just spawning it with CLI options for that matter.

@segator said:
guys do you know if exist any way to control which user request the transcoding? i would like to deny transcoding to some users or atless no video transcoding only audio and remuxing.

Currently, there is no builtin way to do this. However, there might be a way you can hack it together. The first issue is that PRT does not receive details about who is requesting the file, only what file is requested. Plex has a sessions endpoint that let’s you see what file is being played and by whom. In PRT you can define the path_script config variable which is a user-defined script that is passed the full path to the requested file. So, you could try writing a simple script that reads in the path, checks the Plex sessions and figures out which user is requesting the file. If you deem that user unworthy, then the script can return a fake file path, which will cause the transcoder to fail and no video will play back.

Like I said, this is pretty hacky, but it might just work. Good luck!

Just as suspected, all they are doing is calling the Plex New Transcoder via command-line and almost all options are a direct correlation to FFMPEG with a few of their own sprinkled in there. : )

/usr/lib/plexmediaserver/Resources/Plex New Transcoder -noaccurate_seek -i /media/movies/Adventure/10,000 BC (2008).mkv -map 0:0 -metadata:s:0 language=eng -codec:0 copy -bsf:0 h264_mp4toannexb,h264_plex -map 0:1 -metadata:s:1 language=eng -codec:1 aac -strict:1 experimental -cutoff:1 15000 -channel_layout:1 stereo -b:1 256k -f matroska -avoid_negative_ts disabled -map_metadata -1 - -start_at_zero -copyts -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/1mkofa7z2ofajor/progress

There is your file and session info as well.

@Jusedawg said:
Just as suspected, all they are doing is calling the Plex New Transcoder via command-line and almost all options are a direct correlation to FFMPEG with a few of their own sprinkled in there. : )

/usr/lib/plexmediaserver/Resources/Plex New Transcoder -noaccurate_seek -i /media/movies/Adventure/10,000 BC (2008).mkv -map 0:0 -metadata:s:0 language=eng -codec:0 copy -bsf:0 h264_mp4toannexb,h264_plex -map 0:1 -metadata:s:1 language=eng -codec:1 aac -strict:1 experimental -cutoff:1 15000 -channel_layout:1 stereo -b:1 256k -f matroska -avoid_negative_ts disabled -map_metadata -1 - -start_at_zero -copyts -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/1mkofa7z2ofajor/progress

There is your file and session info as well.

That is completely correct, that is how it passes it’s information around. I am a little confused though…was this just a general ā€œhey, FYI I found thisā€? or are you responding to a particular comment/question? If it’s the latter it’s kind of hard to follow if you’re not quoting or mentioning the original item in your comment.

@wnielson said:

@segator said:
guys do you know if exist any way to control which user request the transcoding? i would like to deny transcoding to some users or atless no video transcoding only audio and remuxing.

Currently, there is no builtin way to do this. However, there might be a way you can hack it together. The first issue is that PRT does not receive details about who is requesting the file, only what file is requested. Plex has a sessions endpoint that let’s you see what file is being played and by whom. In PRT you can define the path_script config variable which is a user-defined script that is passed the full path to the requested file. So, you could try writing a simple script that reads in the path, checks the Plex sessions and figures out which user is requesting the file. If you deem that user unworthy, then the script can return a fake file path, which will cause the transcoder to fail and no video will play back.

Like I said, this is pretty hacky, but it might just work. Good luck!

It’ll be interesting to see how PMS reacts to that…because if it’s expecting transcoding across both video/audio you might just get a fail if it doesn’t get what it expects on the client end.

I’m actually interested what the logic is behind having the users able to access your library yet not allowed to transcode video but only audio. Apart from sheer curiosity, if we were to understand the reasoning behind it, perhaps there’s another path to what you’re trying to achieve rather than going down this path.

@liviy said:

That is completely correct, that is how it passes it’s information around. I am a little confused though…was this just a general ā€œhey, FYI I found thisā€? or are you responding to a particular comment/question? If it’s the latter it’s kind of hard to follow if you’re not quoting or mentioning the original item in your comment.

Yeah, I was mostly answering myself. : ) I don’t like how you have to quote everything in here instead of just being able to reply to a particular post. I was also referring to the direct previous post above it. Sorry about that.

I’ve been able to get PRT working perfectly with three remote Ubuntu VM’s for local network and remote network transcoding. Thanks for this awesome feature add! An issue I’m running into is that when I attempt to sync content, no transcoders seem to get chosen to handle the task and the sync job fails.

Any thoughts? Server Version 0.9.15.1

@liviy said:

@alternativesurfer said:

@archangel00 said:
sounds interesting, would be keen to see how the Windows development come along :slight_smile:

Just another monthly ā€œWindows Please!ā€ ping

A windows port is likely to be a considerable way off, I would suggest you look at migrating to linux either to a physical system or a virtual guest (using something like virtualbox). Linux is much more stable & less overhead than windows, even if you’ve never used linux before I would expect you will get it working that way in a much shorter time than waiting for a port.

Virtualbox + Ubuntu 14.04 on my Windows 10 physical works great as a third transcode server. Took about 15 minutes to install/configure and it was golden. Almost as easy as the vmware guests that are the other two transcoders.

@jclifton said:
I’ve been able to get PRT working perfectly with three remote Ubuntu VM’s for local network and remote network transcoding. Thanks for this awesome feature add! An issue I’m running into is that when I attempt to sync content, no transcoders seem to get chosen to handle the task and the sync job fails.

Any thoughts? Server Version 0.9.15.1

Hey there @jclifton ,

I have a couple of ideas because I suspect that the sync transcoder is hitting one of the bugs/features from plex that I’ve seen and is on my list of things to either add into the code or change the setup to ensure there’s a work around in place…
Before setting up my crash & burn environment to replicate what you’ve described I checked something to see if my hypothesis is correct and it would appear so…
I’m guessing you don’t get your systems to generate the image index files for your videos do you? Otherwise you would have noticed a similar issue during that process.
I think the issue you’re seeing is that while you have the option in the plex server setting of where you can set your ā€˜Transcoder temporary directory’, that only by default seems to apply for live transcoding (e.g. streams to players) and there’s another couple of transcoding types which don’t use it by default (thanks plex for not keeping everything the same!).

So…how to manually work around it (and I’m still mulling over if there’s a better way of doing this as I don’t like leaving that whole tree RW)

In the guide it says for the NFS sharing you only need /opt/plex/tmp shared as rw but then I’ve noticed that the bif generation fails and as you’ve described so would sync as they seem to use the same separate area.

Instead of in /etc/exports having this :
/var/lib/plexmediaserver 192.168.0.3(ro,sync,no_subtree_check)
Have this :
/var/lib/plexmediaserver 192.168.0.3(rw,sync,no_subtree_check)
Which is changing the ro to rw

then re-export your shares and on your slaves you can just do a

mount -o rw,remount /var/lib/plexmediaserver

Here’s an example of a sync I just ran and where PRT was directing the transcode (by plex’s request) :

/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/Transcode/Sync+/4033693/da/70356.mp4.temp

Hopefully that gets your back running again and I’ll chew on this a bit more and see if I can’t come up with a more elegant solution that changing that whole share to RW moving forward.

Cheers,
Liviy
(P.S. excuse all the edits as it’s still early and I think I need another coffee! :wink: )

@liviy said:

@jclifton said:
I’ve been able to get PRT working perfectly with three remote Ubuntu VM’s for local network and remote network transcoding. Thanks for this awesome feature add! An issue I’m running into is that when I attempt to sync content, no transcoders seem to get chosen to handle the task and the sync job fails.

Any thoughts? Server Version 0.9.15.1

Hey there @jclifton ,

I have a couple of ideas because I suspect that the sync transcoder is hitting one of the bugs/features from plex that I’ve seen and is on my list of things to either add into the code or change the setup to ensure there’s a work around in place…
Before setting up my crash & burn environment to replicate what you’ve described I checked something to see if my hypothesis is correct and it would appear so…
I’m guessing you don’t get your systems to generate the image index files for your videos do you? Otherwise you would have noticed a similar issue during that process.
I think the issue you’re seeing is that while you have the option in the plex server setting of where you can set your ā€˜Transcoder temporary directory’, that only by default seems to apply for live transcoding (e.g. streams to players) and there’s another couple of transcoding types which don’t use it by default (thanks plex for not keeping everything the same!).

So…how to manually work around it (and I’m still mulling over if there’s a better way of doing this as I don’t like leaving that whole tree RW)

In the guide it says for the NFS sharing you only need /opt/plex/tmp shared as rw but then I’ve noticed that the bif generation fails and as you’ve described so would sync as they seem to use the same separate area.

Instead of in /etc/exports having this :
/var/lib/plexmediaserver 192.168.0.3(ro,sync,no_subtree_check)
Have this :
/var/lib/plexmediaserver 192.168.0.3(rw,sync,no_subtree_check)
Which is changing the ro to rw

then re-export your shares and on your slaves you can just do a

mount -o rw,remount /var/lib/plexmediaserver

Here’s an example of a sync I just ran and where PRT was directing the transcode (by plex’s request) :

/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/Transcode/Sync+/4033693/da/70356.mp4.temp

Hopefully that gets your back running again and I’ll chew on this a bit more and see if I can’t come up with a more elegant solution that changing that whole share to RW moving forward.

Cheers,
Liviy
(P.S. excuse all the edits as it’s still early and I think I need another coffee! :wink: )

Wow thank you for such a quick turn around! Interestingly enough, that resolution didn’t work for me. I made the rw change, bounced nfs, remounted slaves (even rebooted as a test) and still the sync function doesn’t work.

@jclifton said:

Wow thank you for such a quick turn around! Interestingly enough, that resolution didn’t work for me. I made the rw change, bounced nfs, remounted slaves (even rebooted as a test) and still the sync function doesn’t work.

Hey @jclifton

No worries!

Interesting…I’m quite shocked it didn’t work as I have the same setup as yourself, including the PMS version. Only difference is I’ve come across some issues with PMS like this and have applied fixes. To try and help you can you gather some information for me please?

On the master, (for the first command as root or use sudo but the rest can be as plex user) run :
exportfs -v
ls -la /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Cache/Transcode/Sync\+/
tail -100 /opt/plex/tmp/prt.log
tail -100 /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Logs/Plex\ Media\Server.log

on one of the slaves, (as plex user) run :
cd /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Cache/Transcode/Sync\+/
df -h .
touch writetest && rm -f writetest

I don’t mean to sound patronising but if I was to make a guess it is that something didn’t work correctly with the rw export or you got the /usr/lib/plexmediaserver and the /var/lib/plexmediaserver mounts mixed up (I really despise plex’s decisions on where they keep their application files).

Hopefully the output from the above commands should shed some light on the issue. Feel free to PM me rather than spamming this thread if you wish. Or alternatively log an issue at :
github.com/wnielson/Plex-Remote-Transcoder/issues

Cheers,
Liviy

@liviy said:

@jclifton said:

Wow thank you for such a quick turn around! Interestingly enough, that resolution didn’t work for me. I made the rw change, bounced nfs, remounted slaves (even rebooted as a test) and still the sync function doesn’t work.

Hey @jclifton

No worries!

Interesting…I’m quite shocked it didn’t work as I have the same setup as yourself, including the PMS version. Only difference is I’ve come across some issues with PMS like this and have applied fixes. To try and help you can you gather some information for me please?

On the master, (for the first command as root or use sudo but the rest can be as plex user) run :
exportfs -v
ls -la /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Cache/Transcode/Sync\+/
tail -100 /opt/plex/tmp/prt.log
tail -100 /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Logs/Plex\ Media\Server.log

on one of the slaves, (as plex user) run :
cd /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Cache/Transcode/Sync\+/
df -h .
touch writetest && rm -f writetest

I don’t mean to sound patronising but if I was to make a guess it is that something didn’t work correctly with the rw export or you got the /usr/lib/plexmediaserver and the /var/lib/plexmediaserver mounts mixed up (I really despise plex’s decisions on where they keep their application files).

Hopefully the output from the above commands should shed some light on the issue. Feel free to PM me rather than spamming this thread if you wish. Or alternatively log an issue at :
github.com/wnielson/Plex-Remote-Transcoder/issues

Cheers,
Liviy

Much appreciated. DM incoming!

Just to follow up with everyone watching. I fail. My CIFS mount to the freenas host I use for storage was dying on the slaves for some reason. Got that sorted out and then @liviy’s solution worked flawlessly.

-1 for me today.