Which video and audio formats are supported for DirectPlay on Xbox One Plex client?

Plex Players Gaming Consoles
1

Dear Community,

I’ve struggling for a long time to identify why am i not able to utilize DirectPlay on Xbox One Plex Client.
I have Plex Server installed on my Synology DS216j, therefore server side processing/transcoding is the thing what i cannot do since this NAS CPU can’t handle these tasks.
However the Xbox One have the horsepower to deal all this processing, so I want to use the Xbox One but it’s not working. Always telling me that “Your server is not enough to play the movie”.
I’ve tried with a lot of different formats, but in most cases i would use it with mkv/h.265/DTS/AC3 formats.

Plex Support is not answering my question properly, so here are my questions:

  • What kind of video/audio formats are supported on Plex Client for Xbox One?
  • How can I enable the local processing on the client? (I’ve tried to check DirectPlay or DirectStream…)
  • Is there a difference in the supported video/audio formats for Plex Client for different platforms?

Thank you in advance,
Istvan

2

1.https://support.plex.tv/hc/en-us/sections/200642446-FAQ-Plex-for-Xbox-One
2. there is no such thing. not sure what you mean
3. all clients have different supported formats.

3

@istvan.gyori@icloud.com said:
I’ve tried with a lot of different formats, but in most cases i would use it with mkv/h.265/DTS/AC3 formats.

The bold above is likely the biggest problem:

  1. DTS is not a supported codec for the Xbox One (outside of the Bluray app at least). This will always trigger an audio transcode.
  2. h265 is supported, but if anything needs to be transcoded (see #1) then the h265 will need to be transcoded to h264 (I can’t remember the exact reasons for this offhand). During this transcode to h264 it will also transcode down to 1080p.
4

@adamskoog said:
2. h265 is supported, but if anything needs to be transcoded (see #1) then the h265 will need to be transcoded to h264 (I can’t remember the exact reasons for this offhand). During this transcode to h264 it will also transcode down to 1080p.

It’s because H265 (HEVC) is not currently supported by the HLS (the streaming format used) standard. So it has to be converted from HEVC to H264. And then the last bit was due to a problem with the Microsoft H264 decoder causing issues with 4K material playback.

5

I read these “which format should I use?” question threads every time, and every time walk away disappointed.
The referred docs page https://support.plex.tv/hc/en-us/articles/203824396-What-media-formats-are-supported- itself is vague (“Content matching the following can usually be Direct Played”), I keep hoping for something concrete which never materialises.

Is it actually that hard for Plex to document something that will work and if there are conditional exceptions actually qualify those? If we know it usually plays then by extension don’t we know those instances where it does not? The way re-encoding works, couldn’t Plex publish an output spec that everyone could target to get something that will direct play?

6

@jungwirth said:
I read these “which format should I use?” question threads every time, and every time walk away disappointed.
The referred docs page https://support.plex.tv/hc/en-us/articles/203824396-What-media-formats-are-supported- itself is vague (“Content matching the following can usually be Direct Played”), I keep hoping for something concrete which never materialises.

Is it actually that hard for Plex to document something that will work and if there are conditional exceptions actually qualify those? If we know it usually plays then by extension don’t we know those instances where it does not? The way re-encoding works, couldn’t Plex publish an output spec that everyone could target to get something that will direct play?

The problem is some people have the most random formats/settings/inclusions into a file all of which can cause a problem. mp4/h264/aac will usually direct play. But if you enable subtitles it will not direct play, if you have a bit rate limit set it might not direct play, if you’re playing from a server with resolution/total upload settings it might not direct play, if you have too many audio channels it might not play, if you change the h264 settings to modify the defaults when encoding it might not play, if the container is not an mp4 but the file was simply renamed *.mp4 it might not play, and the list goes on of settings the end user/server owner/file creator could tweak or change that can break playback.

Because of that there is literally no way for Plex to publish something that is guaranteed to direct play because there are too many factors outside of their control that can and will break direct play. That is why they can only recommend formats that will normally direct play such as:

mp4/h264/aac
H264 preferred h264 level of 4.0 or lower for best compatibility
AAC 2 channel for best compatibility.

With those settings the file is almost always work to direct play on any client, but again depending on the source material, audio settings, subtitles, server settings, client settings, encoding options, etc there is no way to guarantee it will work. Typically, if you are using that format and it will not direct play the server will log exactly why that file is causing a transcode. It is also possible to take the transcode call from the log and use that to convert your files, but even those files are still subject to some rules that at the time of playback could still disable direct play.

7

With all those problems, I’m wondering if a NAS upgrade would fix all this stuff (forums.plex.tv/discussion/281504/synology-ds416play-to-transcode-x264-x265-1080p-with-subtitles/p1?new=1

Last week I was watching episode 2 of GoT with subtitles very smoothly and with episode 3 it’s not playing smoothly… very strange! If someone can explain me why, based on the attached NFOs, would be appreciated :smiley: .

The only difference for me is the slightly increased bitrate on episode 3… but do you think it can explain the jerky playback??

Thanks for your help!

1 Like
8

@bPlaTyPuS said:
The only difference for me is the slightly increased bitrate on episode 3… but do you think it can explain the jerky playback??

I’m not an expert, but anytime I watch a higher bitrate video I have to turn off Direct Play to get it to play smoothly. I’ve got a PC that can handle the transcoding, so it’s only a minor annoyance at the moment. In your case, it definitely seems like a hardware upgrade would help. I’m really hoping the Xbox One X with the extra power won’t have the same issues. Of course, the issues won’t be fixed by power alone, so hopefully MS supports more audio/video formats soon too.

9

@jmckee said:
The problem is some people have the most random formats/settings/inclusions into a file all of which can cause a problem. mp4/h264/aac will usually direct play. But if you enable subtitles it will not direct play, if you have a bit rate limit set it might not direct play, if you’re playing from a server with resolution/total upload settings it might not direct play, if you have too many audio channels it might not play, if you change the h264 settings to modify the defaults when encoding it might not play, if the container is not an mp4 but the file was simply renamed *.mp4 it might not play, and the list goes on of settings the end user/server owner/file creator could tweak or change that can break playback.

I do understand that there are variables involved, and I hope that all the other people who have ever asked the question understand that too (though I know not all do) - but those are [mostly] precisely the caveats that should be documented in the doc [with the exception of people having terribly weirdly encoded files - I agree, they’re on their own]. Take a baseline optimal setup (PMS and Xbox on LAN, no subtitles, no bit rate limiting [not required because on LAN], not sure what ‘resolution/total upload settings’ are) and then work backwards.

Again I would hope that, most users would accept that if they’ve artificially capped bitrate, or are streaming from WAN rather than LAN that there will be transcoding. Or if they’re just renaming files and not checking the format/container. Or poorly producing encodes.

@jmckee I appreciate you’re trying to help, I do, but the ambiguity in the documentation is the disappointment :frowning: And unfortunately gives little for the helpful Ninja’s to really fall back on (and in that regard I feel for you - I know how much easier it is when documentation actually backs up what you’re telling someone). Plex isn’t alone in this as far as software is concerned: we define a standard environment and the predictable results from plugging something (in this case media of specific formats) in. Still, I do appreciate the time taken to give some thoughts on it :slight_smile:

10

@mbarylski said:

@bPlaTyPuS said:
The only difference for me is the slightly increased bitrate on episode 3… but do you think it can explain the jerky playback??

I’m not an expert, but anytime I watch a higher bitrate video I have to turn off Direct Play to get it to play smoothly. I’ve got a PC that can handle the transcoding, so it’s only a minor annoyance at the moment. In your case, it definitely seems like a hardware upgrade would help. I’m really hoping the Xbox One X with the extra power won’t have the same issues. Of course, the issues won’t be fixed by power alone, so hopefully MS supports more audio/video formats soon too.

it’s unbelievable that Microsoft is not giving access to all the decoders of the XO to third party apps… so in the future I’ll not count on it anymore!

11

@bPlaTyPuS said:
With all those problems, I’m wondering if a NAS upgrade would fix all this stuff (forums.plex.tv/discussion/281504/synology-ds416play-to-transcode-x264-x265-1080p-with-subtitles/p1?new=1

Last week I was watching episode 2 of GoT with subtitles very smoothly and with episode 3 it’s not playing smoothly… very strange! If someone can explain me why, based on the attached NFOs, would be appreciated :smiley: .

The only difference for me is the slightly increased bitrate on episode 3… but do you think it can explain the jerky playback??

Thanks for your help!

That is strange indeed. Currently there isn’t any maximum bit rate for the Xbox One client. And from other clients I have seen neither of those two files would be in the range of hitting that max (with either average or max bit rate). Looking at the other thread it could be possible that just the little bit extra bitrate might be pushing the NAS a bit too much. A quick test to see if that is the case is to look at the server log and search for speed=. Normally I will search for that and just skip through a couple of them to see if the values have dropped down to below 1. If they have then an upgraded NAS could fix the issue (I don’t know enough about the NAS’s them self to advise on that route though), however if all the speed values are staying above 1 then an upgraded NAS shouldn’t be the issue. I only say shouldn’t because it could be possible that something else is hampering the playback (But typically you would be able to identify them before spending money to upgrade)

12

@jungwirth said:
@jmckee I appreciate you’re trying to help, I do, but the ambiguity in the documentation is the disappointment :frowning: And unfortunately gives little for the helpful Ninja’s to really fall back on (and in that regard I feel for you - I know how much easier it is when documentation actually backs up what you’re telling someone). Plex isn’t alone in this as far as software is concerned: we define a standard environment and the predictable results from plugging something (in this case media of specific formats) in. Still, I do appreciate the time taken to give some thoughts on it :slight_smile:

I have brought this up before to developers, as I myself believe a basic listing of supported formats/codecs would be an excellent resource, but have not heard anything towards progressing in having this type of information compiled and posted (or even made available for the Ninja’s to fall back on to advise people).

But for the Xbox One my standard formats for direct play are:

Stereo channel:
mp4/h264/aac

Surround:
mkv/h264/ac3

For direct stream (Where the video direct plays and audio gets converted):
mkv/h264/(Pretty much any audio format TrueHD, Atmos, DTS HD)

4K video (Must be direct play, no other option will work for 4K material without causing transcoding)
mp4 or mkv / HEVC / AAC or AC3

The biggest thing I have found is to make sure the video itself is h264. A number of blurays I have ripped are VC-1 which will cause issues transcoding with a large number of servers. It is also possible to look at the Xbox One.xml in the profiles folder on the server where you can see what formats will directplay. They are formatted by container, video, then audio with a basic take 1 from each section for a compatible file.

13

@jmckee said:

@jungwirth said:
@jmckee I appreciate you’re trying to help, I do, but the ambiguity in the documentation is the disappointment :frowning: And unfortunately gives little for the helpful Ninja’s to really fall back on (and in that regard I feel for you - I know how much easier it is when documentation actually backs up what you’re telling someone). Plex isn’t alone in this as far as software is concerned: we define a standard environment and the predictable results from plugging something (in this case media of specific formats) in. Still, I do appreciate the time taken to give some thoughts on it :slight_smile:

I have brought this up before to developers, as I myself believe a basic listing of supported formats/codecs would be an excellent resource, but have not heard anything towards progressing in having this type of information compiled and posted (or even made available for the Ninja’s to fall back on to advise people).

But for the Xbox One my standard formats for direct play are:

Stereo channel:
mp4/h264/aac

Surround:
mkv/h264/ac3

For direct stream (Where the video direct plays and audio gets converted):
mkv/h264/(Pretty much any audio format TrueHD, Atmos, DTS HD)

4K video (Must be direct play, no other option will work for 4K material without causing transcoding)
mp4 or mkv / HEVC / AAC or AC3

The biggest thing I have found is to make sure the video itself is h264. A number of blurays I have ripped are VC-1 which will cause issues transcoding with a large number of servers. It is also possible to look at the Xbox One.xml in the profiles folder on the server where you can see what formats will directplay. They are formatted by container, video, then audio with a basic take 1 from each section for a compatible file.

Thx for the answers @jmckee.

How is the Plex server detecting if the video stream and/or audio stream should be transcoded or not?

Let’s say I try to play a mkv/h264 with DTS-HD audio (with & without subtitles) on a configuration like this:

Plex server (on NAS) - gigabit ethernet
Plex client (on XO) - gigabit ethernet / HDMI 1.4 to amplifier
Home cinema amplifier (DTS-HD compatible)

I’m asking this because I don’t understand how the Plex server (or maybe client) is detecting that the amplifier has DTS-HD support and therefor the audio stream could just be pass-through.

For me mkv/h264 with DTS-HD (WITHOUT subtitles) should be direct played on the mentioned configuration.

Now if you add subtitles, as the XO does not support them, I think that Plex server is going to transcode… Am I right? Even if it’s not truly transcoding as the only thing that should be done by the Plex server is burning the subtitles on the video stream. The audio should still be pass-through as is (DTS-HD).

Thanks!

14

@bPlaTyPuS said:
Thx for the answers @jmckee.

How is the Plex server detecting if the video stream and/or audio stream should be transcoded or not?
The way it currently works is when you hit play the app requests the video information for the file from the server. The app then determines if it can play the video and audio codecs and if they can be played in that stream. If they can the app simply requests the file as is from the server. If they can’t the app requests the server to transcode the file.

The server then determines how to transcode the file based on the information in the Xbox One.xml profile. It will determine if it can direct stream the video or audio or if they need to be converted. The server then performs the transcoding of the file and sends that stream to the Xbox One.

Currently, there is no passthrough support. This is because as you asked a little further down, the server/app currently don’t have a way to read what setting the Xbox One is set to. Because of that the developers can’t white list the HD audio formats because then anyone without passthrough set on the Xbox One will get no audio.

Let’s say I try to play a mkv/h264 with DTS-HD audio (with & without subtitles) on a configuration like this:

Plex server (on NAS) - gigabit ethernet
Plex client (on XO) - gigabit ethernet / HDMI 1.4 to amplifier
Home cinema amplifier (DTS-HD compatible)

I’m asking this because I don’t understand how the Plex server (or maybe client) is detecting that the amplifier has DTS-HD support and therefor the audio stream could just be pass-through.
The audio will have to transcode. There is no DTS decoder at all for the Xbox One at all HD or Non. And for the reasons above pass-through selection currently is not enabled because it can’t be detected yet.

For me mkv/h264 with DTS-HD (WITHOUT subtitles) should be direct played on the mentioned configuration.
In that situation the video should direct stream, but the audio will need to be transcoded. This still uses a lot less CPU power, but can still be understandably inconvenient.

Now if you add subtitles, as the XO does not support them, I think that Plex server is going to transcode… Am I right? Even if it’s not truly transcoding as the only thing that should be done by the Plex server is burning the subtitles on the video stream. The audio should still be pass-through as is (DTS-HD).
This will actually cause the most CPU usage of all. Burning in subtitles actually means the entire video stream needs to be decoded, then re-encoded while the subtitles are overlayed into the newly created video stream. In addition that DTS audio still will need to be transcoded as no option exists in the app use pass-through so the only codecs that will be sent to the Xbox are codecs that it currently supports decoding of.

15

Thanks again @jmckee for this very complete explanation!

Plex developers should add some checkbox in client settings like “Enable audio pass-through”. On XO, there’s an audio settings which allows bitstream pass-through (https://support.xbox.com/en-US/xbox-one/console/configure-audio-settings#863685f837eb4e6a8f566d2d005ac261) but I think that this setting is not readable by third-party apps, hence the need of the checkbox in Plex client.

I’m also wondering why could Plex client not transcode the video rather than letting the server do it? Could also be an option in Plex client settings like “Perform transcoding clientside”. In many configurations the Plex server is installed on non-powerful machines like NAS or on a raspberry system. I agree that this would not be option for smartphones or tablets but on a console (XO, PS4, Nvidia shield and maybe even Apple TV) it could be a very smart solution as they have well enough power to transcode maybe even 4k materials.