Intel QSV(ffmepg codec) support in Linux

Plex Media Server
25

VAAPI (libva) - is the official software name and applicable to Intel or AMD (if drivers present)

QSV – Intel Quick Sync Video – ONLY

They wouldn’t allow a feature suggestion for what already exits. True?

From your logs, here is how you confirm:

Feb 06, 2023 22:04:21.253 [0x7faeda7d1b38] VERBOSE - [Req#7fd/Transcode] [FFMPEG] - VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 22.4.4 (2bbbd9b).
Feb 06, 2023 22:04:21.253 [0x7faeda7d1b38] VERBOSE - [Req#7fd/Transcode] [FFMPEG] - Driver not found in known nonstandard list, using standard behaviour.
Feb 06, 2023 22:04:21.253 [0x7faeda7d1b38] DEBUG - [Req#7fd/Transcode] [FFMPEG] - Created surface 0.
Feb 06, 2023 22:04:21.253 [0x7faeda7d1b38] DEBUG - [Req#7fd/Transcode] [FFMPEG] - Direct mapping possible.
Feb 06, 2023 22:04:21.253 [0x7faeda7d1b38] DEBUG - [Req#7fd/Transcode] TPU: hardware transcoding: final decoder: vaapi, final encoder: vaapi
Feb 06, 2023 22:04:21.254 [0x7faeda7d1b38] VERBOSE - [Req#7fd/Transcode] Sufficient disk space: 878.95kB source file, 395.71GB capacity, 372GB available on "/var/lib/plex/Plex Media Server/Cache/Transcode/Sessions"
Feb 06, 2023 22:04:21.254 [0x7faeda7d1b38] DEBUG - [Req#7fd/Transcode/JobRunner] Job running: FFMPEG_EXTERNAL_LIBS='/var/lib/plex/Plex\ Media\ Server/Codecs/acf6c67-4446-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4781-af8b-a8d8d386e878 "/usr/lib/plexmediaserver/Plex Transcoder" "-codec:#0x01" h264 "-hwaccel:#0x01" vaapi "-hwaccel_fallback_threshold:#0x01" 10 "-hwaccel_output_format:#0x01" vaapi "-hwaccel_device:#0x01" vaapi "-codec:#0x02" aac_lc -analyzeduration 20000000 -probesize 20000000 -i /home/jellyfin/Videos/teenfidelity.e367.melody.marks.maintenance.baby.4k.mp4 -filter_complex "[0:#0x01]hwupload[0];[0]scale_vaapi=w=3840:h=2160:format=nv12[1];[1]hwupload[2]" -map "[2]" -codec:0 h264_vaapi -b:0 20000k -r:0 29.969999999999999 -force_key_frames:0 "expr:gte(t,n_forced*3)" -filter_complex "[0:#0x02] aresample=async=1:ochl='stereo':rematrix_maxval=0.000000dB:osr=48000[3]" -map "[3]" -codec:1 aac -b:1 256k -f dash -seg_duration 3 -dash_segment_type mp4 -init_seg_name 'init-stream$RepresentationID$.m4s' -media_seg_name 'chunk-stream$RepresentationID$-$Number%05d$.m4s' -window_size 5 -delete_removed false -skip_to_segment 1 -time_delta 0.0625 -manifest_name "http://127.0.0.1:32400/video/:/transcode/session/hs1ze1xl1wlh6zspcm6jscwp/7d294e74-e1c3-40d4-ba3a-53dc62d47500/manifest?X-Plex-Http-Pipeline=infinite" -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -vsync cfr -init_hw_device vaapi=vaapi: -filter_hw_device vaapi -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/hs1ze1xl1wlh6zspcm6jscwp/7d294e74-e1c3-40d4-ba3a-53dc62d47500/progress
  1. Vaapi driver found the Intel Media Driver (which will ultimately use QSV)
  2. After “MDE:” decisions complete, (which is where player and server negotiate what the player can handle vs what PMS must do)
  3. Notice vaapi for the decoder and encoder

Side note:

If you have docker on Arch (presume you do)

You can run the Plex (or LSIO) docker image in the container. You get the Debian package distro inside.

In the debian installer scripts, I look for a container environment, if I do, I trust the container and silently allow the package to install/upgrade.

Lastly, you can always @chuckpa and I’ll see it.

Out in the ‘port’, eh? :wink:
(I’m busted up with a bad chest cold. needed time off)

26

QSV is not a codec. It is the name of Intel’s ASIC for hardware acceleration of video codecs. There are different APIs for invoking the HWA of the ASIC—VAAPI and QSV. VAAPI is the method that PMS employs and works fine across many Linux distributions for a myriad of video codecs. Why do you think QSV is required?

27

Thank @ChuckPa and @Achilles very much for your patient reply!

QSV is not a codec. It is the name of Intel’s ASIC for hardware acceleration of video codecs

  • Yes, this is indeed a commercially advertised name by Intel.
  • However, this kind of Intel-modified VAAPI based on libmfx and libvpl can be used in FFMpeg (can you understand it this way) to get better graphics card performance utilization as I tested on the fifth floor.

Let’s understand why there are these differences?

图片
图片1705×293 38 KB

This picture is from the FFMPEG document. When you specify this codec method, it will preferentially use this modified VAAPI based on libmfx and libvpl to codec video.

图片
图片1622×129 12.2 KB

And what we usually know, open source, standard, VAAPI, is based on libva. This is the reason for the difference in usage of the 3D unit above.

In addition, in this decoding workflow, even though you are not an 8th and later generation CPU, it can still effectively fall back to the standard VAAPI interface process.

And based on @nagatoro answer, you can ask the technological consultant to look at the code here for ffmpeg - their implementation is different from the standard VAAPI anyway.

@nagatoro FFmpeg/libavcodec/qsvenc_h264.c at master · FFmpeg/FFmpeg · GitHub
QSV and VA-API are two different interfaces in FFmpeg to access Intel’s QuickSync hardware.

That’s why I think it’s a feature request.
In the current Master branch of ffmpeg, enable the --enable-libmfx flag in the buildtime to support this acceleration scheme implemented by FFMPEG.
Then you can directly use it with these ffmpeg flags: -hwaccel qsv -hwaccel_output_format qsv -c:v h264_qsv.

Therefore, when you choose to accept this workflow, it will result in significant benefits for new generations of Intel CPUs and Intel discrete graphics cards, such as A380, A750, A770.

28

@a632079

Please don’t discuss the Intel A-series GPUs here yet.

Engineering has not yet told us whether or not they will be supporting it.

If you want to add that to Feature Suggestions – that’s fine. I believe there is already a request you can upvote

Regarding:

I know what you’re saying but please don’t overthink this. The framework already exists albeit named a little differently because the HW tone mapping stack changes things.

29

Please look very carefully here

  1. Hardware identified

Feb 06, 2023 15:46:27.290 [0x7f575c772b38] DEBUG - [Req#4df/Transcode/h1f9kmds07wa54dfmm3kktyx] [FFMPEG] - Direct mapping possible.

  1. API selected

Feb 06, 2023 15:46:27.291 [0x7f575c772b38] DEBUG - [Req#4df/Transcode/h1f9kmds07wa54dfmm3kktyx] TPU: hardware transcoding: final decoder: vaapi, final encoder: vaapi

  1. FFMPEG invocation.
    – Transcode (video codec, audio codec, and bit rates) to fit browser
    – Use Intel Open Compute Runtime to perform HW tone mapping.
    – Scale to fit the window size requested by Plex/web browser.

Feb 06, 2023 15:46:27.291 [0x7f575c772b38] DEBUG - [Req#4df/Transcode/h1f9kmds07wa54dfmm3kktyx/JobRunner] Job running: FFMPEG_EXTERNAL_LIBS=‘/sata/plex/Plex\ Media\ Server/Codecs/583b3e4-4434-linux-x86_64/’ LIBVA_DRIVERS_PATH=/usr/lib/plexmediaserver/lib/dri OCL_ICD_VENDORS=“/sata/plex/Plex Media Server/Cache/CL-ICDs” X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4296-8fe8-f164acd7018c cl_cache_dir=“/sata/plex/Plex Media Server/Cache/Shaders/icr-9-linux-x86_64/” “/usr/lib/plexmediaserver/Plex Transcoder” -codec:0 hevc -hwaccel:0 vaapi -hwaccel_fallback_threshold:0 10 -hwaccel_output_format:0 vaapi -hwaccel_device:0 vaapi -codec:1 dca -ss 24 -analyzeduration 20000000 -probesize 20000000 -i “/glock/media/uhd/47 Ronin (2013)/47 Ronin (2013).mkv” -filter_complex “[0:0]hwupload[0];[0]scale_vaapi=w=2276:h=1280:format=p010[1];[1]hwmap=derive_device=opencl[2];[2]tonemap_opencl=tonemap=mobius:format=nv12:m=bt709:p=bt709:r=tv[3];[3]hwmap=derive_device=vaapi:reverse=1[4];[4]hwupload[5]” -map “[5]” -codec:0 h264_vaapi -b:0 14124k -maxrate:0 18833k -bufsize:0 37666k -r:0 23.975999999999999 -force_key_frames:0 “expr:gte(t,n_forced*1)” -filter_complex “[0:1] aresample=async=1:ochl=‘stereo’:rematrix_maxval=0.000000dB:osr=48000[6]” -map “[6]” -metadata:s:1 language=eng -codec:1 aac -b:1 175k -f dash -seg_duration 1 -dash_segment_type mp4 -init_seg_name ‘init-stream$RepresentationID$.m4s’ -media_seg_name ‘chunk-stream$RepresentationID$-$Number%05d$.m4s’ -window_size 5 -delete_removed false -skip_to_segment 25 -time_delta 0.0625 -manifest_name “http://127.0.0.1:32400/video/:/transcode/session/h1f9kmds07wa54dfmm3kktyx/9e11a99b-8366-4c4e-a173-9f3559c51ed7/manifest?X-Plex-Http-Pipeline=infinite” -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -filter_hw_device vaapi -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/h1f9kmds07wa54dfmm3kktyx/9e11a99b-8366-4c4e-a173-9f3559c51ed7/progress

There’s a lot more going on here than you think.

30

While doing some testing the performance difference was insane… In ffmpeg benchmark mode (-f null -) using hwaccel qsv yielded way better performance by also using fully utilizing the decoder compared to hwaccel d3d11va (only used the decoder by < 1/4)
This also was proven by jellyfin on linux - qsv vs vaapi - qsv was way “faster”.

Maybe plex could start using qsv for decoding purposes and vaapi/dxva/d3d11 for encoding - so nothing changed on the encoder side?