[Release] UnicornTranscoder, create a Plex Transcoding Cluster

Thank you for the additional information. I think your documentation is good but I personally was a tad confused on when and when not to use the ending “/” since your examples sometimes use IP and some are URLs. For me, since I was testing with IPs, the UnicornTranscoder document gives an example of a full url and the example of: “HTTP/HTTPS address of the UnicornLoadBalancer” yet it didn’t indicate you would need the actual ports for these, not just the IP, which is why I gave IP based examples, and at the time of testing I wasnt sure if the ending “/” would be required since some URLs forward to specific ports and others do not.

I greatly appreciate your teams effort in creating such a great solution for Plex expansion.

1 Like

Have you all looked at the Nvidia Jetson series as standalone encoders?

(https://www.newegg.com/Product/Product.aspx?Item=N82E16813190009)

I could see picking up a few $100 nano’s and having some extra streams worth of encoders…

The interesting part is what they list as encoder/decoder support…

Video Encoder
4K @ 30 | 4 x 1080p @ 30 | 9 x 720p @ 30 (H.264/H.265)
Video Decoder
4K @ 60 | 2 x 4K @ 30 | 8 x 1080p @ 30 | 18 x 720p @ 30 (H.264/H.265)

I’m not smart enough to know if since this runs a version of ubuntu on pretty standard ARM chips if drivers would work sort of out of the box to use this like a regular Nvidia GPU for encode/decode…

Is it possible to use this to pass custom ffmpeg parameters?

Running into a wall here. I had it working with VM’s only locally. Decided to build out physicals.

Old setup VM for Load Balancer, VM for plex media server, VM for transcoder.
New setup Physical Server runs Plex Media Server and Load Balancer (together). Physical server running UnicornTranscoder.

I’m trying to avoid 127.0.0.1 addresses and forcing each items to run on it’s assigned IP. This doesn’t seem to work for the load balancer. It still comes up in DEBUG as 127.0.01.

External public host’s with nginix wasn’t working.

I decided to start over and do everything locally to get to square one.
Load Balancer is 10.31.13.10:3001
Plex Media Server is 10.31.13.10:32400 (FFMPEG has been replaced with the npm built one.)
UnicornTranscode server is 10.31.13.11:3002

There is no mention in the documentation what user this stuff should be installed as. I used my regular user. I could only get debug info if I run it with “sudo npm start”. All of the plex media server is either plex:plex or root:root.

I have PMS Network set to internal IP of http://10.31.13.10:3001 (LoadBalancer)

Transcoder config.js:
module.exports = {
port: env.int (‘SERVER_PORT’, 3002),
host: env.string(‘SERVER_LISTEN’, ‘10.31.13.11’),
transcoder_decay_time: env.int (‘TRANSCODER_DECAY_TIME’, 120),
loadbalancer_address: env.string(‘LOADBALANCER_ADDRESS’, ‘http://10.31.13.10:3001’),
ping_frequency: env.int (‘PING_FREQUENCY’, 10),
instance_address: env.string(‘INSTANCE_ADDRESS’, ‘http://10.31.13.11:3002’),

Load Balancer environment variables:
SERVER_HOST=“10.31.13.10”
SERVER_PORT=“3001”
SERVER_PUBLIC=“http://10.31.13.10:3001/
PLEX_HOST=“10.31.13.10”
PLEX_PORT=“32400”
PLEX_PATH_USR="/usr/lib/plexmediaserver/"
PLEX_PATH_SESSIONS="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions"
DATABASE_MODE =“sqlite”
DATABASE_SQLITE_PATH ="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db"
CUSTOM_SCORES_TIMEOUT=“10”
CUSTOM_IMAGE_RESIZER=“false”
CUSTOM_DOWNLOAD_FORWARD=“true”
DEBUG=*
LB_URL=“http://10.31.13.10:3001/” <- just added that trying to force LB to not use localhost

When I start load balancer “sudo npm start” with debug at the end it says:
UnicornLoadBalancer Launched on http://127.0.0.1:3001/ +45ms (not 10.31.13.10:3001)

Internally I load a web browser and goto http://10.31.13.10:3001 and it launches the PMS page.

I try to transcode watch a show to test and the DEBUG on the Transcoder has this error:

UnicornTranscoder:Transcoder FFMPEG(stderr): [dash @ 0x1b86600] Opening ‘http://127.0.0.1:32400/video/:/transcode/session/3fsvnbu59pe4w3gm2arfzxg3/bcdc44e9-8326-41fe-bbfd-16d83e863798/manifest’ for writing
UnicornTranscoder:Transcoder +1ms
UnicornTranscoder:Transcoder FFMPEG(stderr): [tcp @ 0x1c373c0] Connection to tcp://127.0.0.1:32400 failed: Connection refused
UnicornTranscoder:Transcoder +0ms
UnicornTranscoder:Transcoder FFMPEG(stderr): [tcp @ 0x1c37440] Connection to tcp://127.0.0.1:32400 failed: Connection refused
UnicornTranscoder:Transcoder +0ms
UnicornTranscoder:Transcoder FFMPEG(stderr): [dash @ 0x1b86600] Unable to open http://127.0.0.1:32400/video/:/transcode/session/3fsvnbu59pe4w3gm2arfzxg3/bcdc44e9-8326-41fe-bbfd-16d83e863798/manifest for writing

Why is it trying to open 127.0.0.1:32400 on the Transcode Server? PMS isn’t running on this server. Am I supposed to launch something else for it to have a 32400 running?

Before this I was trying to get this to work from the outside the network.
PMS Network was set to https://host.domain.com:443
Load Balancer Environment was SERVER_PUBLIC=“https://host.domain.com:443/
Transcoder config.js had
loadbalancer_address: env.string(‘LOADBALANCER_ADDRESS’, ‘https://host.domain.com:443’),
instance_address: env.string(‘INSTANCE_ADDRESS’, ‘https://host-n1.domain.com:443’),
nginx had 2 entries for each hostname and redirected their 443 to they’re internal ports 3001 (LB) and 3002 (Transcode)
I could get manually to either site from outside. PMS would come up but if I tried playing something it would just spin and never see any debug on the Transcoder debug.
Also, somewhere in the combo of changing things around I was getting this error when trying to play something ““Error code: s1001 (Network)””. Not getting that now.

Any insight would be appreciated. Recommendations for permissions to install these component. Recommended server layout would be helpful (run each on a separate server, LB/PMS combo… etc). Sample of ngnix configuration would be helpful (or is the assumption you have multiple external IP’s available?). How do you tell if the Transcoder has registered with the Load Balancer? The documentation is good but has too many assumptions end users might not know about.

Love the project. Keep up the fine work.
Thanks
duhasst

Update… I have it mostly working now with remote access.

There is still the issue of “tcp://127.0.0.1:32400 failed: Connection refused” if I’m trying to play from the internal web interface of the load balancer (http://10.31.13.10:3001/web/index.html#) and kick off a transcode play.

Local app looks like it’s doing a direct play. I have to try forcing transcode with the local app to see if it causes the error.

External app (testing Plex iOS) is transcoding the video correctly.

I’ll try to go over my settings for others to try.
My internal network only has 1 public IP address for outside.
Physical server that is running Plex Media Server and UnicornLoadBalancer (VM would work too)
Physical server that is running UnicornTranscoder (VM would work too)
Docker running LetsEncryptIt + NGNIX (you could do a server or a VM)
Router is a EdgeRouter Lite
DNS I have control over. I use duckdns.org for keeping my external IP available

Router has port forwarding setup to the NGNIX server.
External Port 80 to Internal Port 80 Ngnix Server (I turn this on and off to renew certificates… otherwise it’s not really needed)
External Port 443 to Internal Port 443 Ngnix Server

DNS:
I created a CNAME loadbalancerhostname that points to myhostname.duckdns.org
I created a CNAME unicorntranscoderhostname that points to myhostname.duckdns.org

I have my own domain. You could maybe create 2 duckdns.org entries that point to the same IP.
EX: loadbalancerhostname.duckdns.org and unicorntranscoderhostname.duckdns.org. and use those instead of the CNAME’s I created.

NGNIX:
I used their plex as a subdomain template. I created 2 conf files one for the LoadBalancer and one for the Transcoder.
LoadBalancer ngnix site conf update the following:
server_name loadbalancerhostname.*; (CNAME you setup)
proxy_pass http://10.31.13.10:3001; (Internal IP Address of the LoadBalancer)

UnicornTranscoder ngnix site conf update the following:
server_name unicorntranscoderhostname.*; (CNAME you setup)
proxy_pass http://10.31.13.11:3002; (Internal IP Address of the UnicornTranscoder)

Plex Media Server is ubuntu server 18.04. Installed ubuntu server updated everything and installed Plex Media Server from apt install plexmediaserver
This installs the public PMS. I have a plex pass and after loading the PMS and logging in via the web interface set it to beta and did a check update. You have to manually download this and install it with dpkg.
You can update Settings -> Network -> Custom server access URLs with https://loadbalancerhostname.domain.com:443
Wait to enable remote access until the LoadBalancer is setup.

Follow the UnicornTranscoder information to load the UnicornFFMPEG and UnicornLoadBalancer.
I used my regular user and created a UnicornTranscoder directory. In there I did the git clones for both the UnicornFFMPEG and UnicornLoadBalancer.
I did the UnicornFFMPEG first and renamed the original files and replaced them with the built UnicornFFMPEG binaries.
Second I did the UnicornLoadBalancer. I backed up /etc/environment to /etc/environment.orig
“sudo vim /etc/environment” add below path:
SERVER_HOST=“10.31.13.10”
SERVER_PORT=“3001”
SERVER_PUBLIC=“https://loadbalancerhostname.domain.com:443/
PLEX_HOST=“10.31.13.10”
PLEX_PORT=“32400”
PLEX_PATH_USR="/usr/lib/plexmediaserver/"
PLEX_PATH_SESSIONS="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions"
DATABASE_MODE =“sqlite”
DATABASE_SQLITE_PATH ="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db"
CUSTOM_SCORES_TIMEOUT=“10”
CUSTOM_IMAGE_RESIZER=“false”
CUSTOM_DOWNLOAD_FORWARD=“true”
DEBUG=*
TRANSCODER_DEBUG=true

I changed CUSTOM_DOWNLOAD_FORWARD to true. I do lots of direct playing of files.

I didn’t edit the config.js file in the src directory. I used the environmental variables.

launch LoadBalancer “sudo npm start” (need sudo to get debug info on the screen)

UniconTranscoder server as my regular user I created a UnicornTranscoder directory and in that directory did the git clone.
I modified the following items in the config.js
port: env.int (‘SERVER_PORT’, 3002), (I changed the transcoder to run on port 3002 instead of 3000)
host: env.string(‘SERVER_LISTEN’, ‘10.31.13.11’), (this is the IP Address of the transcode server)
loadbalancer_address: env.string(‘LOADBALANCER_ADDRESS’, ‘https://loadbalancerhostname.domain.com:443’),
instance_address: env.string(‘INSTANCE_ADDRESS’, ‘https://unicorntranscoderhostname.domain.com:443’),
plex_build: env.string(‘PLEX_BUILD’, ‘1.15.6.1079-78232c603’), (build of my beta PMS)
codecs_build: env.string(‘CODECS_BUILD’, ‘392df4f-2233’), (latest codecs on my other PMS server)
eae_version: env.string(‘EAE_VERSION’, ‘eae-69c1de6-34’), (latest eae version on my other PMS server)
debug: env.boolish(‘TRANSCODER_DEBUG’, true),
maxSessions: env.int(‘MAX_SESSIONS’, 16), (my server has 16 threads)
maxDownloads: env.int(‘MAX_DOWNLOADS’, 16), (my server has 16 threads)
maxTranscodes: env.int(‘MAX_TRANSCODE’, 16), (my server has 16 threads)

then ran npm install
then ran npm run install. (this downloads the codecs)
then ran sudo npm start (need sudo to get debug on the screen)

I then setup ufw on the PMS/LB server.
sudo ufw default allow incoming
sudo ufw default allow outgoing
sudo ufw deny 32400/tcp
sudo ufw allow ssh
sudo ufw enable
sudo ufw status verbose

How internal network load a web browser and goto the IP Address of the LoadBalancer
EX: my internal LB is http://10.31.13.10:3001

Plex page should open. Now you should be able to goto settings -> remote access and enable remote access.

Then you need to use a device outside your network and load the external address of the plex server.
EX: https://loadbalancerhostname.domain.com:443
this will make you login to the server externally and usually resolves remote access not working. It’s a bit off compared to the normal port redirections. It will complain about double nat here and there but has been working so far. It will take a good 5-10 minutes to register with plex.tv.

Double test you can login to plex.tv and you should see your server available to select.

I think that is everything. Good luck. When I figure out the local transcoding issue I’ll update.

duhasst

Is hardware transcoding on your roadmap? Since this completely replaces the plex transcoder this throws a wrench in my plans.

I’d like to create a “transcoding cluster” between my NAS (ds1019+) and my gaming computer. The computer has to run software transcoding but the NAS has to run hardware… Would be cool to support both and have it be an option on the individual transcoder install.

1 Like

Does the UnicornTranscoder support the generation of video preview thumbnails?

You can make a Pull request to support this feature, it can be interesting to add it in the UnicornTranscoder configuration

Yes it’s planned, but I can’t give you any date, we don’t use it!
Feel free to help us to support this feature :wink:

Thumbnails are generated by Plex Media Scanner not by the Transcoder

Today we released UnicornTranscoder/UnicornLoadBalancer 2.3.0

Changelog

  • This release supports Plex 1.16.1.1291-158e5b199
  • Fix download redirect issue
  • Fix burned subtitles issue
  • Chromecast: Fix jump issue
  • Chromecast: Fix quality issue
  • Replace Plex download link by direct file path

Update notes

  • Shutdown your transcoder/load-balancer
  • Update Plex to 1.16.1.1291-158e5b199
  • Remove Plex Relay
  • Pull the 2.3.0 version on each
  • npm i
  • npm run install
  • restart


1 Like

Unicorn Transcoder/Load Balancer Release 2.3.1

Changelog

  • Hotfix: Support external Plex docker
  • Support Plex 1.16.3.1402-22929c8a2

Update notes

  • Shutdown your transcoder/load-balancer
  • Update 1.16.3.1402-22929c8a2
  • Remove Plex Relay
  • Pull the 2.3.1 version on each
  • npm i
  • npm run install
  • restart
1 Like

Unicorn Transcoder Release 2.3.2

Changelog

  • Support Plex 1.16.4.1469-6d5612c2f

Update notes

  • Shutdown your transcoder/load-balancer
  • Update Plex to 1.16.4.1469-6d5612c2f
  • Remove Plex Relay
  • Pull the 2.3.2 version
  • npm i
  • npm run install
  • restart

We didn’t tried the support of the new download feature from the new apps: “Plex for Windows” and “Plex for Mac”, can someone try and give us a feedback?

1 Like

I tested it on my Windows 10 pro v1803 build: 17134.950, working fine for me.

I’ve successfully setup unicorn-transcoder and unicorn-loadbalancer docker containers behind a nginx revproxy however I’m having difficulties with enabling remote access.

I’ve tried port forwarding directly to the load balancer and to the revproxy ports but neither seems to work.

I’m curious if anyone has successfully got it to work with the load balancer?

The new download feature don’t work yetwith UnicornLoadBalancer and UnicornTranscoder

@wazzgood Ask on github for advanced support, a lot of setup were explained in solved issues

1 Like

Unicorn Transcoder Release 2.3.3

Changelog

  • Support Plex 1.16.5.1488-deeb86e7f

Update notes

  • Shutdown your transcoder/load-balancer
  • Update Plex to 1.16.5.1488-deeb86e7f
  • Remove Plex Relay
  • Pull the 2.3.3 version
  • npm i
  • npm run install
  • restart

After tests on our side, we can confirm UnicornTranscoder project is compliant with the new “Plex for Windows” and “Plex for Mac” download feature, some issues can happend with Nginx configuration, see the following issue: https://github.com/UnicornTranscoder/UnicornTranscoder/issues/66

@Maxou44 hey! This project looks awesome! Would this NUC8i7BEH be good hardware for running Unicorn? Any other recommended hardware? What version of Linux would best support this?

Also, have you seen the new beta of the updated Plex transcoder? =) Hopefully Unicorn can support the upcoming HDR to SDR tone mapping…

Edit: I currently have a Synology DS718+ with the 5 bay expansion unit. It would be super cool to give Plex additional resources to speed up sync conversion (especially) and enable more streams.

1 Like

Hello,

I think it’s a good CPU even if it’s not the best performance/price candidate today, maybe @drouarb have some information about the best cpu now.

In general ubuntu/debian is the most used platform to setup transcoders, all linux OS are supported (if node and plex support it, of course)

I didn’t saw the new Plex transcoder, can you share me a link ? Latest supported version is the 1.16.5.1488-deeb86e7f build. The Latest Plex Pass don’t break anything, just update codecs if needed.

The HDR → SDR convert will be functionnal if it’s a CPU convert. GPU support isn’t supported yet :wink:

Sync (always) and download (stored in the cloud) are supported and can be served/transcoded by UnicornTranscoder

1 Like