How multiple streams affect direct play and transcoding

Plex Media Server
1

In a recent thread troubleshooting 4K transcoding performance, @ChuckPa mentioned that it’s best practice to remove unneeded audio and subtitle streams from a file. I didn’t want to hijack that thread, so asking here.

Can anyone detail what data is read on both the decode side (Plex server reading the playback streams - video / audio / subtitles to either transcode or stream to clients) and then on the encode side when transcoding, or on the stream side when direct playing to the client?

I’d thought that, at least when transcoding, the Plex server is only serving up the selected audio and subtitle stream to the client, which is why if you change the audio or subtitle stream it often has to re-buffer.

For direct play, though, is Plex streaming all the streams from the file through to the end client, or is there some ability there to only select the streams necessary?

And then same question goes for the decode side where Plex is reading the file from storage? And I guess an even better question is, when direct playing, is Plex doing any decoding at all, or is it just acting as a proxy to send the media from storage to the client?

I used to remove other streams when doing rips of my bluray and 4k UHD rips, but after doing some comparison it wasn’t a huge difference in rip time or filesize, and I have a lot of storage and never need to upload the files to the internet, I stopped caring. I figured I could always go back and clean them up later with MKVToolNix, but didn’t want to rerip if I needed an additional language in the future for a family member or something.

My Plex server can handle the bitrates no problem, media is on an SSD array and I get 3-5 Gbps read rates from the storage array itself to the Plex server. It also has no problem transcoding, but I’m wondering if the source files can cause problems when direct playing to some clients if it is passing through a ton of extra stream data to them. The network link speed might be fine, but the client might not be able to process data as fast as needed due to hardware limits.

2

In any case, the server has to read the file first as it is, to process it further. If there are numerous audio, subtitle, or even several video streams in there, the storage subsystem has to read the file in all its high bitrate and at least transport it into RAM. Which takes up at least PCI bandwidth, and probably also SATA, or LAN bandwidth (if stored on a file share).
If the server is then playing in Direct Stream mode or is transcoding, the usused streams are discarded by the transcoder.
But the fact remains that the full, fat file needs to arrive at least in the server’s RAM as it is, thereby occupying resources which could be needed otherwise if there are several files to process concurrently.

Direct Play means exactly that. Play the file as-is. Which means that the whole file is served to the client as it is. It is then up to the client to deal with it.

That’s what Direct Stream is doing.
On some client types you can selectively disable Direct Play. Which then means Direct Stream is used (unless one of the streams requires transcoding).

https://support.plex.tv/articles/categories/plex-media-server/direct-play-direct-stream-transcoding/

The overall bandwidth of a computer to handle concurrent data transfers is not unlimted. And there are several bottlenecks, which are different, depending on things like RAM type, no. of RAM channels, CPU type, mainboard, no. of PCI lanes, clock speeds etc.pp.

You can bet that on particularly low-power architectures, there are more restrictions than on regular “desktop” or even “server” type platforms. The savings in energy consumption have to come from somewhere. (And no, it’s not only reductions in silicon structure widths which cause savings in power consumption.)

3

Thanks Otto for taking the time to break all of that down.

Understand about bottlenecks, server has 8 Comet lake cores, 96 GB RAM disk for transcoding, and 10 Gbps to storage, which is 4 stripes of 3 SSD RAIDZ1’s, so the bottleneck is about 5 Gbps read speed off the array, about 8-10 times faster than any 4k content I am playing back.

I am just seeing noticeable delays when switching between audio tracks and subtitles, so was trying to figure out what was happening there. Also, sometimes starting a video can take 10 seconds or so as well, but scrolling on the timeline to the middle of the movie, well in advance of where the buffer has made it to, resumes playing instantly. Only seeing this with 4k rips, all direct playing, but those are the ones I’ve been lazy about not cutting out streams when ripping.

It looks like the main problem is with direct play, I have a second server offsite with similar specs but slower storage, and transcoding over the internet at 40 Mbps the client is handling the exact same file to the exact same client without the delays when switching or when initiating playback. When direct play at home, its 63 Mbps and I see those delays.

When you say ‘has to read all the file’ it sounds like it is reading the whole file, but you mean that it has to play back all of the streams in the file and present them to the player - right? Because there’s no way Plex could be reading the whole file before starting to play it, it would take 2 minutes to start a video when playing a 4K remux (takes about that time to download a 4k rip from my storage to the Plex server), and the transcode directory grows in size gradually over the course of playback, nowhere near the size of the file.

So its presenting the file, with all streams, to the client and it is choosing which to use.

In the case of direct play, if the file has 1 video, 7 audio, and 10 subtitle tracks, the Plex server and the client are both receiving a stream of all of that data and the client is picking which to play. If the buffer is 60 seconds or so, is the client buffering all the streams? It seems like not with the delay when switching between audio/subtitles?

Maybe its the clients that have trouble with direct playing the high bitrates, and multiple streams. I guess I will do some experimenting before and after cleaning up the extra streams to see if that is the issue.

4

This is most likely due to transcoding. The server is creating a buffer of prepared chunks of data in advance.
If you change one or several of the stream selections, these chunks need to be discarded and the preparation has to start anew. Playback will only resume after a minimum of data chunks has been prepared and transmitted to the client.
That is also true if the playback mode is just Direct Stream.

Another possible cause of delays: If the source file container is difficult to seek in (e.g. due to poor or missing timecode markers), Plex will have to start from the beginning of the source file again and read it until it arrives at the intended playback position.
Remuxing the file into a known good container format may help.
If it is a mp4 container, “optimize for streaming” is a highly recommended action.

Yet another reason is a poor mux, where the audio and video streams are interspersed with each other (“interleaving” = good) but with a large delay (bad).
There are also still files out there, which don’t use interleaving at all (very bad).

5

The delays are happening with direct play, client is an Nvidia Shield and supports the subtitle formats, so nothing is being transcoded.

When forcing transcoding the same file, no delays occur.

Is MKV considered a bad container?

6

As I said above, it can also happen with Direct Stream.

Then it must be something in the original file which the Plex transcoder is silently fixing, but which causes some trouble for the Shield player.

Certainly not. But there can be different parameters used, which determine how a file is being muxed – even if it’s into the same container format.
It can also be important when the file was muxed (i.e. which software version of the MKV muxer library was used), because some issues were fixed with later revisions of the muxer.

Just try it yourself, it is very easy to do and relatively quick. MKVtoolnixGUI

7

Usually just use MKVToolnix to break up TV shows that get lumped together as a single title on the disc.

I tested it out with a file that had the following audio streams:

English 7.1
English 5.1
English Stereo
Spanish 5.1
French 5.1
English Stereo Commentary

And the following subtitles (all PGS):

English
Spanish
French
Spanish
French

So I left all 3 English tracks (not the commentary) and the English subtitles. The filesize only dropped from 54 to 53 GB.

Ironically, the playback bitrate when direct streaming on LAN to my Nvidia Shield went up from 69 Mbps in previous tests to 70 Mbps now for some reason, but that’s close enough that its probably just no change.

Despite that, the delays when initially playing the file or seeking far ahead in the video are gone.

So I guess now I have a project on my hands to edit all my 4K MKV’s.

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.