Why can't I do hardware-accelerated _decoding_ (cuvid) with nVidia hardware?

Server Version#: 1.13.9.5456
Player Version#: Web Client ( 3.69.1) or AppleTV (latest)

Hi folks,

I run both Emby and Plex under Docker on Linux, using nvidia-docker to provide GPU capabilities to the container.

With Emby (which uses ffmpeg as a transcoder), I can use hardware acceleration for both encoding and decoding (h264_cuvid), and it works a treat. I use 1-2 CPU threads to stream a 4K movie.

Under Plex, I read that hardware decoding is not supported with nVidia GPU, but only with Intel Quicksync (which, sadly, my CPU doesn’t support).

Since it’s obviously technically possible to use cuvid decoding, and (AFAIK) the Plex transcoder uses ffmpeg, is there a reason why we can’t use nVidia GPUs for decoding?

Thanks!
David

Engineering working on it but there’s only so many hours in a day and never enough hands to go around.
I don’t know when they will have it done.

I wish the news could be better.

Appreciate the reply, understand re the finite number of hands :slight_smile:

I’ve been asking for it for over a year. They just never added the enable parameter for cuvid to their ffmpeg builder. It literally takes 2 seconds for whoever manages their jenkins to do that.

They did say they were going to add nvdec, the new implementation that is supported in ffmpeg v4, but they haven’t updated their ffmpeg build to v4 yet. Pretty ridiculous in my opinion

@aptalca

The polite response is:

The Plex Transcoder, while having FFMPEG roots, is a lot more than just FFMPEG.
It is not as easy as “Compile FFMPEG; rename the file; include in the PMS distribution package”

Don’t patronize me.

It is not as easy as “Compile FFMPEG; rename the file; include in the PMS distribution package”
That is exactly what the plex transcoder is. It is literally stock ffmpeg (confirmed by another plex team member), compiled with some local linked libraries including the intel driver (common practice when building ffmpeg; one can even build a completely static build that needs no dependencies from the host like zeranoe’s builds), then renamed to Plex Transcoder and included in the PMS build.

Here’s the compile options used:

root@Tower2:/# LD_LIBRARY_PATH=/usr/lib/plexmediaserver /usr/lib/plexmediaserver/Plex\ Transcoder
ffmpeg version git-2018-09-24-531e313 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8.2 (GCC)
  configuration: --libdir=/data/jenkins/pms-deps-universal-v2/BUILD_TAG/linux-ubuntu-x86_64/output/pms-depends-linux-ubuntu-x86_64-release-79dcc0f/lib --prefix=/data/jenkins/pms-deps-universal-v2/BUILD_TAG/linux-ubuntu-x86_64/output/pms-depends-linux-ubuntu-x86_64-release-79dcc0f --disable-bzlib --disable-ffplay --disable-ffprobe --disable-avdevice --disable-schannel --disable-linux-perf --disable-mediacodec --disable-debug --disable-doc --disable-shared --pkg-config-flags=--static --enable-muxers --enable-gpl --enable-version3 --enable-gnutls --enable-eae --disable-encoders --disable-decoders --disable-hwaccels --enable-libass --enable-librtmp --disable-devices --disable-lzma --disable-iconv --disable-protocol=concat --enable-libvorbis --enable-libopus --external-decoder=h264 --disable-bsfs --enable-bsf='aac_adtstoasc,extract_extradata,dca_core,h264_mp4toannexb,hevc_mp4toannexb,vp9_superframe,vp9_superframe_split,framedrop' --enable-decoder=png --enable-decoder=apng --enable-decoder=bmp --enable-decoder=mjpeg --enable-decoder=thp --enable-decoder=gif --enable-decoder=dirac --enable-decoder=ffv1 --enable-decoder=ffvhuff --enable-decoder=huffyuv --enable-decoder=rawvideo --enable-decoder=zero12v --enable-decoder=ayuv --enable-decoder=r210 --enable-decoder=v210 --enable-decoder=v210x --enable-decoder=v308 --enable-decoder=v408 --enable-decoder=v410 --enable-decoder=y41p --enable-decoder=yuv4 --enable-decoder=ansi --enable-decoder=alac --enable-decoder=flac --enable-decoder=vorbis --enable-decoder=opus --enable-decoder=pcm_f32be --enable-decoder=pcm_f32le --enable-decoder=pcm_f64be --enable-decoder=pcm_f64le --enable-decoder=pcm_lxf --enable-decoder=pcm_s16be --enable-decoder=pcm_s16be_planar --enable-decoder=pcm_s16le --enable-decoder=pcm_s16le_planar --enable-decoder=pcm_s24be --enable-decoder=pcm_s24le --enable-decoder=pcm_s24le_planar --enable-decoder=pcm_s32be --enable-decoder=pcm_s32le --enable-decoder=pcm_s32le_planar --enable-decoder=pcm_s8 --enable-decoder=pcm_s8_planar --enable-decoder=pcm_u16be --enable-decoder=pcm_u16le --enable-decoder=pcm_u24be --enable-decoder=pcm_u24le --enable-decoder=pcm_u32be --enable-decoder=pcm_u32le --enable-decoder=pcm_u8 --enable-decoder=pcm_alaw --enable-decoder=pcm_mulaw --enable-decoder=ass --enable-decoder=dvbsub --enable-decoder=dvdsub --enable-decoder=ccaption --enable-decoder=pgssub --enable-decoder=jacosub --enable-decoder=microdvd --enable-decoder=movtext --enable-decoder=mpl2 --enable-decoder=pjs --enable-decoder=realtext --enable-decoder=sami --enable-decoder=ssa --enable-decoder=stl --enable-decoder=subrip --enable-decoder=subviewer --enable-decoder=text --enable-decoder=vplayer --enable-decoder=webvtt --enable-decoder=xsub --enable-decoder=libzvbi_teletext --enable-decoder=eac3_eae --enable-decoder=truehd_eae --enable-decoder=mlp_eae --enable-encoder=flac --enable-encoder=alac --enable-encoder=libvorbis --enable-encoder=libopus --enable-encoder=mjpeg --enable-encoder=wrapped_avframe --enable-encoder=ass --enable-encoder=dvbsub --enable-encoder=dvdsub --enable-encoder=movtext --enable-encoder=ssa --enable-encoder=subrip --enable-encoder=text --enable-encoder=webvtt --enable-encoder=xsub --enable-encoder=pcm_f32be --enable-encoder=pcm_f32le --enable-encoder=pcm_f64be --enable-encoder=pcm_f64le --enable-encoder=pcm_s8 --enable-encoder=pcm_s8_planar --enable-encoder=pcm_s16be --enable-encoder=pcm_s16be_planar --enable-encoder=pcm_s16le --enable-encoder=pcm_s16le_planar --enable-encoder=pcm_s24be --enable-encoder=pcm_s24le --enable-encoder=pcm_s24le_planar --enable-encoder=pcm_s32be --enable-encoder=pcm_s32le --enable-encoder=pcm_s32le_planar --enable-encoder=pcm_u8 --enable-encoder=pcm_u16be --enable-encoder=pcm_u16le --enable-encoder=pcm_u24be --enable-encoder=pcm_u24le --enable-encoder=pcm_u32be --enable-encoder=pcm_u32le --enable-encoder=h264_vaapi --enable-encoder=hevc_vaapi --enable-encoder=h264_nvenc --enable-encoder=eac3_eae --arch=x86_64 --enable-libzvbi --cc='ccache gcc' --extra-ldflags=-L/data/jenkins/pms-deps-universal-v2/BUILD_TAG/linux-ubuntu-x86_64/output/pms-depends-linux-ubuntu-x86_64-release-79dcc0f/lib --extra-libs='-lgcrypt -lgpg-error -ldl'
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 12.102 / 58. 12.102
  libavformat    58.  9.100 / 58.  9.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

Yup, that’s just ffmpeg. Judging by the build option locations, it is likely built on jenkins, probably multi-arch and multi-platform. If so, it would take adding a single parameter to the config options to enable cuvid, and rebuilding. And no, it wouldn’t require any changes to the build environment either, since nvenc is already included.

If you think I’m patronizing, allow me to elaborate?

FFMPEG, being free public software, does not have to comply with the legal contracts Plex does as a for-profit company.

If you dispute this in any way, please feel free to contact Plex directly and inquire.

?? Honestly, what are you talking about?
What does any of what we discussed have anything to do with legal contracts?

I feel like we are not on the same page. Could you please ping someone else on the team who has more experience with the ffmpeg builds?

Perhaps @Ridley can provide more insight into the matter.
I remember he said a while back that there were plans to add nvdec when they switch to ffmpeg v4. Until then, can cuvid be added since it is supported in the ffmpeg version used by plex? It shouldn’t require any other changes than adding the enable flag. Cuvid had been the official cuda decode method in ffmpeg for a long time: https://trac.ffmpeg.org/wiki/HWAccelIntro#CUDACUVIDNVDEC

Thanks

basically, it sounds like the reason for the delay is a legal/contract issue, not a technical issue.

how or why emby can use a newer version is irrelevant if plex has not worked out there legal/contractual problem.

I highly doubt that (although I could be wrong). The last time ChuckPA started talking about legal contracts and Company Confidential info, he ended up locking a thread and called for a cease and desist, only for another employee to come in and do damage control:

I am sharing with you all I can. I am sorry if this isn’t satisfactory.

If someone gives me permission to disclose all the details, I will.

Until such time, your only recourse is to ask HQ via the contact page. They might give you the answer you seek.

Otherwise, Nvidia will be ready when it’s ready. Period.

1 Like

It is literally stock ffmpeg (confirmed by another plex team member)

Sorry if there’s been a miscommunication here, but this is inaccurate. Plex Transcoder contains a number of changes from upstream ffmpeg to handle our particular use-cases. We send a lot of patches upstream, but some features are either specific to Plex, or wouldn’t work generally enough with the current infrastructure (so they have to be implemented in more specific ways that won’t be accepted upstream until the infrastructure is expanded). You can download the source of the version used in a particular PMS version, with all our changes at a URL listed in the LICENSE file in the Resources directory.

We’ll be updating our ffmpeg to a newer version that includes NVDEC soon. Can’t promise any exact release timeframe, but we’ve already started on the process and I’m optimistic that we should be able to get it through testing soon.

It’s true that we could enable CUVID for decoding with the current version, but it’s more than just adding the build-time configuration flag. It’s actually fairly complex to automatically configure the ffmpeg filter graphs for a new hardware acceleration API, especially given all the different hardware configurations and media types that need to work reliably (e.g. zero-copy conversion can be done in some cases, but isn’t compatible with all filter configurations yet). We’ll be doing that when ffmpeg’s been updated, along with testing in a variety of cases. Testing to make sure we don’t break things for existing users is the biggest reason why changes that might seem simple don’t always get pushed through quickly.

Hope that shines some light on the reasoning here!

5 Likes

Thanks so much for the detailed answer. I hadn’t considered the filtergraphs, which makes sense.

I totally understand the testing needs for v4.

I’ll shut up about cuvid and will look forward to nvdec.

Thanks

1 Like

Can confirm, cuvid is a nightmare to work with. Some things can be transcoded using zero-copy, other things need to be copied to system ram temporarily. Also, Plex has to support many different generations of geforce cards which have different capabilities – ex:

  1. 1070/1080s support VP8 decoding but the 1080 ti and titan x/titan xp do not.
  2. 960 supports h265 decoding but the 980 does not.
    (there are many other examples, including 8/10/12 bit color, along with numerous problems involving pixel formats)

There are also certain nvidia drivers (can’t remember which, this was a year or 2 ago) where mpeg4 decoding is completely broken, and some drivers where the decoder outputs empty frames without warnings/errors… Plex would need to make sure to turn off decoding for those drivers.

Add on top of that that cuvid doesn’t fall back to cpu decoding if a command fails (DXVA2 decoding supports cpu fallback), and it’s kind of a pain to include in production software that end users expect to just work.

2 Likes

Decoding is already supported in Windows. Nvidia supports the same codecs and configs on Linux.

1 Like

I’m pretty sure plex is using dxva2 decoder for windows, which isn’t supported on Linux. Linux would have to use the cuvid/nvdec decoder, which is a pain in the ass to use compared to dxva2.

1 Like

DXVA2 is a Microsoft API, not Linux. Microsoft wouldn’t allow it to happen on Linux unless they get full ownership of Linux in the process.

Right, not very surprising.
But don’t they support the same codecs? Except that NVDEC produces higher image quality.

1 Like

Any more news? Really want this feature to lower temps, lower power requirements, and increase capacity of my aging server.

1 Like

While I respect that it is difficult, Emby has been doing this for a while now. And its not like they have hundreds of developers.they have maybe 3-4 developers at most.

1 Like