I recently upgraded my Plex server to a dual socket Xeon E5520 which can handle 16 simultaneous threads. Unfortunately, I've never seen Plex utilize more than about 1/2 the total cpu capacity; my old server would routinely max out the cpu while transcoding. What gives?
Anyone else have a similar experience
Des,
You obviously made a good step up then. Transcoding is a single-threaded task, one for each active stream. Once the Plex transcoder has a buffer full (which won't take long with what you have now), it will only fill what's needed as it needs to. When the buffer level gets low enough, it simple wakes up the trancoder which fills it back up..
If you have some very really high bit rate media in an odd container format, you can observe as it sends it to something which needs it transcoded 'down' to lower bitrate (say 1.5 Mbps). You should see higher CPU utilization, but it still wont be staggering.
I should clarify that I saw it max out my previous system while transcoding for sync, which is most definitely is multi-threaded, but doesn't seem to use more than 6-8 cores on my new machine.
I should clarify that I saw it max out my previous system while transcoding for sync, which is most definitely is multi-threaded, but doesn't seem to use more than 6-8 cores on my new machine.
different codecs in ffmpeg use different numbers of threads in decode/encode/transcode - from memory nothing will use 16 threads, you'd need to be running multiple simultaneous transcodes to use all those cores. 1/2 the cores in that server are hyperthreading cores, which will never count as 100% of a real core when playing with video.
I know Plex New Transcoder can fully utilize 8 cores (that's the most I have to test with). However, there are some things to know about this. I found that the following scenarios cause the single-threaded transcoding problem:
* When source video is VC-1, with or without PGS subtitles enabled.
* When source video is h.264 or MPEG2, with PGS subtitles enabled.
The only workaround to this is to use a high clocked CPU that can do the whole decoding using a single thread.
This is an interesting thread. I'm looking to build a new server for Plex and other things but mostly for transcoding. I was thinking Dual Xeon 1230v3 or 1240v3. I want to keep some headroom as to not build again for a while. It sounds like even though it's beefy it won't help out with simultaneous 1080p plex streams.
This is an interesting thread. I'm looking to build a new server for Plex and other things but mostly for transcoding. I was thinking Dual Xeon 1230v3 or 1240v3. I want to keep some headroom as to not build again for a while. It sounds like even though it's beefy it won't help out with simultaneous 1080p plex streams.
Why do you say that? The only thing this thread so far has been discussing is one single transcode. If you start several simultaneous transcodes, the Xeon CPU is really helping. I have the 1230 model and it is quite amazing.
I did some testing when I got the CPU, and I killed the CPU when I started the sixth 1080P high bitrate movie file in transcode mode. Everything before that worked like a charm.
This is an interesting thread. I'm looking to build a new server for Plex and other things but mostly for transcoding. I was thinking Dual Xeon 1230v3 or 1240v3. I want to keep some headroom as to not build again for a while. It sounds like even though it's beefy it won't help out with simultaneous 1080p plex streams.
you can't run dual E3 xeons, they are uniprocessor chips. E5-2xxx's are the minimum you'll need for a dual config.
unless you need many (think 5+) simultaneous HD transcodes or have a particular need for ECC memory, anything more than an i7 is a waste of money.
Simultaneous transcodes will use lots of cores, the core limits are for a single transcode session, the 2nd and subsequent transcode sessions can use entirely different cores on the cpus (e.g. transcode 1 can use cores 1 through 8, transcode 2 cores 9 through 16 etc etc)
Why do you say that? The only thing this thread so far has been discussing is one single transcode. If you start several simultaneous transcodes, the Xeon CPU is really helping. I have the 1230 model and it is quite amazing.
I did some testing when I got the CPU, and I killed the CPU when I started the sixth 1080P high bitrate movie file in transcode mode. Everything before that worked like a charm.
Very helpful, thank you. I misread it.
you can't run dual E3 xeons, they are uniprocessor chips. E5-2xxx's are the minimum you'll need for a dual config.
unless you need many (think 5+) simultaneous HD transcodes or have a particular need for ECC memory, anything more than an i7 is a waste of money.
Thanks for this. I didn't know that. I'm still in the planning phase for a new rig and was doing some preliminary gathering of info. I'm planning on 8+ HD Transcodes. Whether I use it or not that is the case. My kids are getting bigger and using more Plex so between the TV's in the house, the multiple mobile devices, etc I'd just like the headroom.
Simultaneous transcodes will use lots of cores, the core limits are for a single transcode session, the 2nd and subsequent transcode sessions can use entirely different cores on the cpus (e.g. transcode 1 can use cores 1 through 8, transcode 2 cores 9 through 16 etc etc)
Can you explain this a little more? I'm obviously failing to grasp things today.
Thanks
Josh
Can you explain this a little more? I'm obviously failing to grasp things today.
Lets say a H.264 transcode can use up to 12 cores, and lets say you have a server with 48 cores in it.
Someone starts to watch something, the transcode starts and uses 12 cores to do it.
Someone else starts to watch the same thing, the transcode starts and uses 12 cores to do it.
The 12 cores used by the 2nd transcode does not need to be the same 12 cores used by the first transcode. So you now have a server with 24 of 48 cores in use for the 2 transcodes even though a single transcode is limited to 12 cores.
3rd person comes along, 36 cores now in use
4 person, 48 cores
5 person, the cores start to be shared amongst all the transcode sessions.
Ahhh.. .Makes sense... Thank you
I know Plex New Transcoder can fully utilize 8 cores (that's the most I have to test with). However, there are some things to know about this. I found that the following scenarios cause the single-threaded transcoding problem:
* When source video is VC-1, with or without PGS subtitles enabled.
* When source video is h.264 or MPEG2, with PGS subtitles enabled.
The only workaround to this is to use a high clocked CPU that can do the whole decoding using a single thread.
So this may answer my question, but I want to confirm.
I've been having trouble transcoding a 1080p bluray rip. The source is VC-1, with two audio tracks (DTS HD and regular DTS 5.1) and three subtitle tracks. I'm trying to stream to a Chromecast on the same network at maximum quality from the lossless audio track and with subtitles, but it's not going faster than a speed of around .5 in the transcode logs.
My rig is an Ubuntu box with two Xeon quads giving me 8 cores at 1.8GHz. Top shows only 200% being used during transcode (instead of utilizing more to increase speed). Is this because the VC-1 track can only be run by one thread and is held back by the relatively slow speed of the processor?
Any thoughts from the board on what else could be causing the speed issue/lack of core utilization?
A little more information: I'm fairly certain this setup streamed without a problem for different-yet-comparable videos within the last month without an issue utilizing 600-700% and transcoding at a speed of 1.9-2, so I'm not sure what could have changed.
My rig is an Ubuntu box with two Xeon quads giving me 8 cores at 1.8GHz. Top shows only 200% being used during transcode (instead of utilizing more to increase speed). Is this because the VC-1 track can only be run by one thread and is held back by the relatively slow speed of the processor?
I am almost 100% sure that I remember an old troubleshooting session ending with that conclusion yes.
One key point to understanding transcoding is that some inter-frame calculations are single threaded. Things like ‘change detection’ operate this way. If the single thread speed is low, the number of cores to perform the matrix ops will be insignificant
Sent with Tapatalk
Maybe this is a foolish question, but since Plex is transcoding in pieces anyway, why can't it have multiple jobs for different pieces?
Maybe this is a foolish question, but since Plex is transcoding in pieces anyway, why can't it have multiple jobs for different pieces?
Might be possible, don't know. Doesn't sound like an ideal solution to the problem though.
I have a server running two X5365. That is 8 total real cores, no hyper-threading. I never see CPU usage above 50% even with the “make my CPU hurt setting on transcoder”
+1 for the option to parallel transcode, particularly for sync. For example, right now we are getting ready to get on a plane with our 4 children. Server is working to sync several movies to 4 different mobile devices, but is bouncing between 40% and 55% CPU use. We would have nearly twice the selection of movies if we could encode two at a time.
A throttle that will increase parallel encoding when CPU, Memory and IO are all bellow a certain threshold would be awesome. Such as, a setting of “increase number of parallel transcoders if CPU < 50%, Memory < 50%, and IO < 10% used.”
@ahildoer said:
I have a server running two X5365. That is 8 total real cores, no hyper-threading. I never see CPU usage above 50% even with the “make my CPU hurt setting on transcoder”+1 for the option to parallel transcode, particularly for sync. For example, right now we are getting ready to get on a plane with our 4 children. Server is working to sync several movies to 4 different mobile devices, but is bouncing between 40% and 55% CPU use. We would have nearly twice the selection of movies if we could encode two at a time.
A throttle that will increase parallel encoding when CPU, Memory and IO are all bellow a certain threshold would be awesome. Such as, a setting of “increase number of parallel transcoders if CPU < 50%, Memory < 50%, and IO < 10% used.”
+1
My server is a second hand Dell R720 with 2x E5 2650, 32G mem,
When I syncing a season of TV shows, plex only use one transcoder.exe and about 20% CPU usage(totally I have 16 cores/32 threads @ 2.0GHz).
If parallel x264-self is difficult, it also can parallel the task: encoding 3-5 videos and transfer them to my ipad(network bandwidth is another problem, but in my case, it is 300Mbps Wi-Fi)
Simply … VC-1 is single threaded… and for guys like us running multiple cpu - multithreaded but power effeicient cpus… it really boils down to singe core/thread passmark… which is usually directly related to cpu freq…
If your rig… mine is an esxi VM on a dual l5640… cant keep up with VC-1 than the best option is to ‘pre-encode’ to a format/container combination that retains the quality you want but will multithread using more / all cores…
I never stream remotely from my full blu ray rips… i always stream from the h264 pre encode… I can usually direct play that format and if it does require or I force a lower bitrate due to cellular quality for example … the encode rate goes from 1.2x to 5x speed… thus WAN becomes the constraint not cpu encode…
The worst part of running a plex server is that 99% of the time (unless you are sharing with lots of people and even then the watch times are pretty narrow) the server sits idle…
So 99% of the time you need pi compute power… but then between 7 -12 pm… you either have to have all your media transcoded so that it direct plays/streams… or have the compute to back up requests…
