Hardware acceleration for video preview thumbnails?

For huge libraries, the current method of generating thumbnails takes forever. As Plex Transcoder is already capable of using hardware decoding, shouldn’t it be possible to use QuickSync and the like to decode the video for the thumbnail generation? Or is this already possible and I just overlooked the option?

2 Likes

I would also like to know this!

Plex doesn’t currently use HW/Quicksync for anything but real-time encoding. It’s not used for sync, optimize or anything else.

Looking at your signature, cayars, this feature might have saved you a couple of kilowatt hours :wink: Let’s hope it will be added in the future. From what I can tell comparing the command line parameters passed to the Plex Transcoder, it might just be possible to switch on hardware decoding by just adding a couple of parameters to the calls. Of course, there is most likely some codec- and hardware-specific plex-internal complexity involved in determining the right parameters for each file, QA complications aside.

1 Like

Well the transcoding engineers know how to do this already obviously since they do it for real-time playback.

For many people it’s not a big deal. If you can off loud your real-time needs to HW then that allows the CPU to handle the “background” processes. Of course that’s only viable or computers with decent CPUs to begin with. If your running on NAS or low end CPU that has HW transcoding available and not being utilized it’s another story! For these people (low end CPUs) the optimize, sync type features are worthless.

Not to mention HW Transcoding could be real useful for DVR functionality. It would be nice to be able to record and have your Plex server (not the specific tuner) be able to deinterlace and convert to H.264 on the fly while recording. With HW transcoding at the speeds we record at (real time) we would be able to do this on several simultaneous recordings at the same time and fall back to CPU use once X number of HW transcodes is in use.

Technically there is no reason why Plex couldn’t make use or multiple GPU cards in the server as well as QuickSync but we are still early in the life cycle for this type of thing. In time perhaps. :slight_smile:

Seems silly to not use it for all the other functions… I hope it will be added!

@RandomNinjaAtk02 said:
Seems silly to not use it for all the other functions… I hope it will be added!

Plex making silly (I would think something stronger than silly could be used) decisions. What is at all surprising about that?

Thinking about it, Plex might already do something that is considerably faster than decoding the whole movie with HW acceleration: extracting key frames (and not decoding the rest). On my server, it takes Plex about 3-4 minutes to extract the thumbnails of a movie. Decoding the whole movie with HW acceleration would probably take 10-12 minutes. I’m not sure the HW acceleration APIs are flexible enough to just decode keyframes.

@gerkensm said:
Thinking about it, Plex might already do something that is considerably faster than decoding the whole movie with HW acceleration: extracting key frames (and not decoding the rest). On my server, it takes Plex about 3-4 minutes to extract the thumbnails of a movie. Decoding the whole movie with HW acceleration would probably take 10-12 minutes. I’m not sure the HW acceleration APIs are flexible enough to just decode keyframes.

Are you referring only to chapter thumbnail extraction or the Netflix style seeking thumbnails? Chapters only are a certain time points, so it makes sense that it doesn’t need HW acceleration, but for the Netflix style seeking (ff/rw) thumbnails, it needs to decode the entire file pretty much… And it would be useful if it could speed up that time intensive process…

1 Like

Thumbnails are created by taking screengrabs of the file at certain intervals then saving these images into 1 file. HW acceleration cannot be used to perform this.

1 Like

Best I can tell the thumbnails are generated by PlexTranscoder.exe with ffmpeg-like commands ‘-codec:v h264 -threads 0 -nostats -q 3 -loglevel quiet’ that then feeds a ‘scale=w=320:h=240’ filter. While hardware-assisted compression may not be available, surely adding ‘-hwaccel dxva2’ will at least accelerate decoding.

2 Likes

+1 to this request for HW accel on thumbnails.

From what I can tell, HW/QuickSync is used with Sync now as it doesn’t peg CPU usage any longer on my NAS when syncing content. PlexTranscoder hovers around 5% during this time while finishing the sync even faster.

1 Like

Guys, they don’t read these forums. I posted a problem back in January and had multiple examples of others across more than a few other posts and the issue has still not been fixed or even acknowledged. They have taken the money and done a lot make no mistake. But fixing problems and taking suggestions are apparently really low on their list. You are wasting your time here.

well it will be awesome if they do , even the idea of using multiple gpu’s and internal gpu together with discrete to handle decoding will be nice. and having slider with options to select all that.

for the thumbnail I am doing that right now and its taking very long for me using 8700k so i either have to buy super expensive cpu for my plex and that will have to be 2066 socket x299 or x399 from amd threadrippers to just use plex thumbnail.

This can be done entirely using the hardware decoder AND hardware jpeg encoder.

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mkv -r 1/2 -c:v mjpeg_vaapi -vf 'scale_vaapi=w=320:h=-1:format=nv12' -global_quality 90 -f image2 img-%06d.jpg

This works on my Xeon E3-1245V6 using Linux with ffmpeg 4.0.4 with very little CPU usage. Creates the jpegs at the same size and rate that Plex currently does.

Based on my testing, it’s limited by the speed of the hardware decoder, the JPEG encoding is practically free time wise.

Would be great to have Plex use the hardware for this task.

1 Like

lol… it’s a shame users have to tell dev’s that :joy:
last statement from plex-dev’s was “not possible” :joy:

question now is, can we feed plex with it ?
because let’s face it - plex won’t embed this (or anything newly requested) in the near future.

To be fair, it looks like only Intel GPUs have MJPEG hardware encoding, so it only works on those platforms. Something similar to the hack that was put together for NVidia hardware decoding before it was built in to Plex proper might be possible.

yeah, but again, does anyone know if plex can be feeded with that thumbnails?

emby once more share their thoughts on this, plus they already have another solution

Emby High-Speed Extraction
Quite recently we’ve introduced a new mechanism for image extraction.
This procedure requires only a minimal amount of frames to be decoded, many of them being keyframes.
Previously we had to decode every single frame of a video for thumbnail extraction. With an extraction interval of 10s and a framerate of 25 fps that means we had to decode 250 frames for a single thumbnail image.
Now, we’re seeking to the position, decode 0-20 frames for accuracy (from keyframe to desired position), might be about 10 frames average.
That means, with the new image extraction we need to decode only 4% of all video frames instead of 100%.

https://emby.media/community/index.php?/topic/72311-why-is-there-no-hw-acceleration-for-image-extraction/

As of 1.18.x, Plex does something similiar, using only keyframes at a rate of 0.5 fps (one frame extracted every 2 seconds).

Just started doing thumbnails for my entire library and noticing it’s only taking 1-2 minutes per movie. Seems like you’re right, they must have changed something under the hood. Glad I waited… or really more so that I’m glad I found the setting a year later. :sweat_smile: