HW transcodng on Windows 10

I’m currently setting up a brand new server running windows 10 with a Ryzen 3600, MSI X570 MB and an nvidia 1080ti.

I hvae put the modified Drivers to have more thant 2 streams in HW transcoding.

The task manager is saying Video Decoding, it hould not be Video Encoding ?

the other weid thing is when going into plex web, only 2 sessions have hw behind transcode, but all have hw behind the source video, does it mean that only 2 streams are GPU transcoded ?

even if I’m not at 100% cpu and not 100% GPU, I get a lot of buffering and not a real smooth playback

any ideas ?

To process your files, Plex first needs to read them (decode) before transcoding them (encode).
From your screenshots it appears that all decodes happen with support from a GPU while only 2 are encoding the video using a GPU (with the 3rd one being encoded by the CPU).

As for why that is I cannot tell.
Could be a limit to the HW transcoding slots… maybe your driver hack/modification didn’t succeed or the 2x 1080p HEVC + 1x 4K HEVC decodes + 2x 1080p H264 encodes are the limit of what the GPU can deal with.

Your logs might provide more information (check out Plex Media Server.log and search for MDE: ).

Edit:
See…

I made some tests with standard 1080p and even 720p files, Starting the third stream, there is no HW encode.

I tried to reapply the patch, no luck !

I search the log and did not find any “MDE:” could you be more specific about the log ?

thanks !

I tried this patch as well, a while ago with no luck. It maybe a update on plex side that mess with this hack or Nvidia update or both. I gave up on this and will probably move to a p2000. Kinda like a luck of the draw hack to me.

I found this in the log

Apr 08, 2020 02:15:01.653 [6076] WARN - Codecs: avcodec_open2 returned -12 for encoder ‘h264_nvenc’
Apr 08, 2020 02:15:01.653 [6076] DEBUG - Codecs: testing h264_mf (encoder)
Apr 08, 2020 02:15:01.706 [6076] INFO - [FFMPEG] - MFT name: ‘NVIDIA H.264 Encoder MFT’
Apr 08, 2020 02:15:01.748 [6076] ERROR - [FFMPEG] - could not set output type (MF_E_UNSUPPORTED_D3D_TYPE)
Apr 08, 2020 02:15:01.748 [6076] WARN - Codecs: avcodec_open2 returned -542398533 for encoder ‘h264_mf’
Apr 08, 2020 02:15:01.748 [6076] DEBUG - Codecs: testing h264 (decoder) with hwdevice nvdec
Apr 08, 2020 02:15:01.749 [6076] DEBUG - Codecs: hardware transcoding: testing API nvdec

Apr 08, 2020 02:15:02.331 [6076] WARN - Codecs: avcodec_open2 returned -12 for encoder ‘h264_nvenc’
Apr 08, 2020 02:15:02.331 [6076] DEBUG - Codecs: testing h264_mf (encoder)
Apr 08, 2020 02:15:02.386 [6076] INFO - [FFMPEG] - MFT name: ‘NVIDIA H.264 Encoder MFT’
Apr 08, 2020 02:15:02.427 [6076] ERROR - [FFMPEG] - could not set output type (MF_E_UNSUPPORTED_D3D_TYPE)
Apr 08, 2020 02:15:02.427 [6076] WARN - Codecs: avcodec_open2 returned -542398533 for encoder ‘h264_mf’
Apr 08, 2020 02:15:02.427 [6076] DEBUG - MDE: Cannot direct stream video stream due to profile or setting limitations
Apr 08, 2020 02:15:02.428 [6076] DEBUG - Codecs: testing h264 (decoder) with hwdevice nvdec
Apr 08, 2020 02:15:02.428 [6076] DEBUG - Codecs: hardware transcoding: testing API nvdec

So I found the solution myself …

the patch was not well applied to the 32b file … I did apply it again, and all is working fine …

I was able to transcode 2 4K streams to 1080p in parallel to 4 1080p High to 1080p low without any problem (and all in hw)
I still have a little of memory in the GPU and the processor load is also Ok !

This was for testing purpose only, I do not want to transcode 4k in real life, but at least the perf are there !

1 Like

what patch are you referring too? I have the same issue in my logs

Apr 16, 2020 18:46:50.272 [5344] ERROR - [FFMPEG] - OpenEncodeSessionEx failed: out of memory (10)
Apr 16, 2020 18:46:50.284 [5344] WARN - Codecs: avcodec_open2 returned -12 for encoder ‘h264_nvenc’
Apr 16, 2020 18:46:50.360 [5344] INFO - [FFMPEG] - MFT name: ‘NVIDIA H.264 Encoder MFT’
Apr 16, 2020 18:46:50.407 [5344] ERROR - [FFMPEG] - could not set output type (MF_E_UNSUPPORTED_D3D_TYPE)

did you figure out the solution? you mentioned patch applied not well?? not sure what patch or what that means? I have the same errors and certain movies are buffering with that same error

You need to patch the driver for your nvidia (consumer) card in order for it to support more than two concurrent hw transcoded streams

okay for a quadro nvidia card do you have links? is it just a modded driver update?

You dont need a patch for quadro. He has a 1080.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.