Anyone have been able to HW transcode on an Intel NUC 11 (Iris Xe)?

[geeooff]

Here, for a NUC 11, i7-1165G7:

# Plex Media Server installation configuration info:  Fri Jul  2 06:00:33 PM EDT 2021
Init=0
Systemd=1
Custom=0
LinuxContainer=0
NewInstall=0
HaveOverride=1
OverrideFile="/etc/systemd/system/plexmediaserver.service.d/override.conf"
PlexUser="plex"
PlexGroup="plex"
VideoGroup="render"
AppSuppDir="/var/lib/plexmediaserver/Library/Application Support"
PlexTempDir="/tmp"
LangEncoding="en_US.UTF-8"
ExistingVersion=12303
HaveHardware=1
HaveTranscoderPref=0
NeedUser=0
NeedGroup=0
NeedVideo=0
Verbose=1
Running=1
Errors=0
Warnings=0
HaveIntel=1
HaveIntelCore=1
HaveIntelXeon=0
Havei915=1
HaveNvidia=0
Processor=11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
CPUSKU=11657
NeedOpenCL=1
NeedBeignet=0
NeedIntelCompute=1
HaveIntelGmmlib=1
HaveIntelIGCCore=1
HaveIntelIGCOpenCL=1
HaveIntelOpenCL=1
HaveIntelOCLoc=1
HaveIntelZeroGPU=1
HaveIntelCompute=1
HaveBeignet=0
HaveOpenCL=1
Platform="NUC11PAHi7"
Processor="11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz"
Distro="Ubuntu 21.04"
Kernel="Linux sargeras 5.11.0-22-generic #23-Ubuntu SMP Thu Jun 17 00:34:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux"    
Memory="31733 625 30436"

[ChuckPa]

@geeooff

Thank you.

a little more liberal than I hoped but it is still well within spec and won’t violate the rules.

[DarthBJW]

@ChuckPa I cant use the script on current setup (docker / fedora) but since I have a vested interest in a solution here quite happy to do a clean ubuntu install if it will help

I’m running on a i5-1135G7 so a different model to the other examples.

[ChuckPa]

@DarthBJW

I use Fedora as well. I developed here so there’s no dpkg. Expect that at the normal shell.

Docker is different. Beignet libraries are included. I don’t know if ICR libraries are yet included.

[turfrider]

@geeooff

ffmpeg -y -hwaccel qsv -c:v h264_qsv -i "$input" -map_metadata 0 -c:a copy -c:v hevc_qsv -load_plugin hevc_hw -preset slow -q:v $quality -look_ahead 1 -movflags +faststart "$output"

@ChuckPa Do you think it’s something to feedback to plex dev’s or not, that a simple ffmpeg command calling on QSV to do a transcode does not cause any kernel panics or crash the system. Whereas we can’t transcode anything with plex without causing system hangs.

Is there something more that they could suggest we test to be closer to how plex uses ffmpeg?

@geeooff I don’t think it matters you can’t manage the HDR file since we can’t even manage h264 to h264 without a crash good effort so far though.

[ChuckPa]

Oh , that definitely looks like something to feedback .

Do you have a comparative log from one which does cause crash?

@turfrider

@turfrider

we can’t even manage h264 to h264 without a crash

Is this on your new CPU or in general on other CPUs inclusive ?

[turfrider]

There has already been many crash reports submitted prior to this, the only new information is that @geeooff has been using ffmpeg by itself and has not caused a GPU hang with the process he is following.

@ChuckPa

Is this on your new CPU or in general on other CPUs inclusive

I mean on the new nuc’s, plex in general will crash with any source file just not always on the first try. This we showed many times already.

This was @geeooff first message showing he can repeatedly transcode files using ffmpeg without any crash.

@geeooff

FYI, I’ve managed to transcode a few H.264 files to H.265 (CBR, VBR and quality-based) for personal purposes with ffmpeg and QSV plugin. Package is v4.3.2 from Ubuntu Hirsute.

Strangely, I’ve transcoded dozen of files, and got ZERO crash… The transcoding performance is huge! I’m confused, I don’t understand what causes Plex crashes…

And his second comment regarding this.

@geeooff

To be completely fair with Plex standard behavior, I will retry to encode to H.264 (CBR mode).

FYI, here was my command to transcode H.264 to H.265 (CQP mode) using QuickSync:

ffmpeg -y -hwaccel qsv -c:v h264_qsv -i "$input" -map_metadata 0 -c:a copy -c:v hevc_qsv -load_plugin hevc_hw -preset slow -q:v $quality -look_ahead 1 -movflags +faststart "$output"

I’m struggling to transcode @ChuckPa’s H.265 HDR sample to H.264 (i’m missing the zscale filter to convert to SDR), so I’ve tested H.264/8-bit to H.264/8-bit transcoding with one of my files.

No problem at all, transcoding at >17x speed (1.5M constant bitrate).

ffmpeg -y -hwaccel qsv -c:v h264_qsv -i "$input" -map_metadata 0 -c:a copy -c:v h264_qsv -b:v $bitrate -load_plugin h264_hw -preset slow "$output"

[ChuckPa]

If I may? Plex specifies the vaapi driver and not qsv.

[turfrider]

@ChuckPa

If I may? Plex specifies the vaapi driver and not qsv .

Good point. I had forgotten you mentioned that before.

That gives @geeooff something to test if he’s able to on his current setup and gives me a reason to try something else again to see if the QSV implementation works.

[geeooff]

I was thinking about VAAPI as well.

Here’s the command I’m executing (yes, still running):

ffmpeg -y -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i "$input" -map_metadata 0 -c:a copy -c:v h264_vaapi -b:v $bitrate "$output"

No crash so far, but I’m not making assumptions it’s stable.
I need more aggressive tests.

[ChuckPa]

Want to be aggressive ?

1. Replace the transcoder executable with a Shell script that prints out all the command line arguments to a log file and exits cleanly.

2. Now run that manually, using the PMS-supplied pieces which run with regular FFMPEG (The transcoder communicates with PMS and the Codec licensing – which is proprietary)

It’s immediately obvious what you need to do.
Now you have a way to test.

[ChuckPa]

I’ve sent a message off to come look.

ANY info you can provide (the command line which causes the panic would be best but probably hard to get).

Maybe you set it to launch in gdb (so it’ll sit there and hang?)

NOTE: Ridley just messaged…

This must be reported to the i915 kernel driver maintainers.
VAAPI is user-space and i915 is kernel space.

This is another manifestation of the i915 driver deficiency for the CPU

[ChuckPa]

@turfrider

@ChuckPa Do you think it’s something to feedback to plex dev’s or not, that a simple ffmpeg command calling on QSV to do a transcode does not cause any kernel panics or crash the system. Whereas we can’t transcode anything with plex without causing system hangs.

I missed this question … sorry.

vaapi vs QSV –

If QSV were used , it would break other hosts which can’t use that.
There are several AMD platforms which rely on vaapi to function.

I don’t know how such a thing could be implemented .
I also, unfortunately, don’t know if RocketLake has achieved critical mass yet to warrant such specialized development manhour attention – but I could also easily be wrong.

In all that, I think the best solution, since we’ve known about this for months now, is to get the i915 fixed upstream.

make some noise

[jmt089]

Is anyone able to say where the noise should be directed?

How far upstream are we talking?

[ChuckPa]

YEP…

Linux kernel – i915 driver development / support team.

This is a problem (panic) as the i915 operates for the user-space VAAPI driver.

User-space drivers (applications) cannot cause kernel panics. The i915, because it directly interfaces to the hardware and user-space memory, can and, in this case, is the root cause of the panic.

[geeooff]

You are right to use “standard” way to use hardware accelerated transcoding.
FFMPEG’s QSV plugin is Intel only, it won’t help Plex to support more platforms (hello, AMD).

Though, I noticed increased performance with QSV compared to VAAPI (14x vs 17x).
Maybe you can implement QSV proprietary support as a preview feature for platforms which support it? VAAPI could remain the default stable implementation.

I’m curious how you support Nvidia Shield TV. Is it VAAPI also on it, or NVENC ?

[DarthBJW]

Well I did have high hopes for the 5.13 kernel release, but while it did solve my NVME drive issue, still getting same problems with transcoding in hardware.

From the kern log, not much has changed from before:

Jul 6 10:00:44 palpatine kernel: [ 721.078120] i915 0000:00:02.0: [drm] Resetting vcs1 for preemption time out
Jul 6 10:00:44 palpatine kernel: [ 721.080029] i915 0000:00:02.0: [drm] GPU HANG: ecode 12:4:4ab6fff5, in Plex Transcoder [5127]
Jul 6 10:00:46 palpatine kernel: [ 723.030420] i915 0000:00:02.0: [drm] GPU HANG: ecode 12:4:4ab6fff5
Jul 6 10:00:46 palpatine kernel: [ 723.031441] i915 0000:00:02.0: [drm] Resetting vcs1 for stopped heartbeat on vcs1
Jul 6 10:00:46 palpatine kernel: [ 723.032014] i915 0000:00:02.0: [drm] Resetting chip for stopped heartbeat on vcs1
Jul 6 10:00:46 palpatine kernel: [ 723.226506] i915 0000:00:02.0: [drm] ERROR Failed to reset chip
Jul 6 10:00:46 palpatine kernel: [ 723.226554] i915 0000:00:02.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_reset+0x209/0x230 [i915]
Jul 6 10:00:46 palpatine kernel: [ 723.433443] show_signal_msg: 21 callbacks suppressed
Jul 6 10:00:46 palpatine kernel: [ 723.433446] Plex Media Serv[4072]: segfault at 0 ip 0000000000000000 sp 00007f0781cb9778 error 14
Jul 6 10:00:46 palpatine kernel: [ 723.433450] Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
Jul 6 10:01:00 palpatine kernel: [ 737.186001] Fence expiration time out i915-0000:00:02.0:Plex Transcoder<5127>:21ee!
Jul 6 10:01:06 palpatine kernel: [ 743.332039] Fence expiration time out i915-0000:00:02.0:Plex Transcoder<5318>:4!

Interestingly when I have that fault, it then prevents a reboot cycle from executing. server shuts down and hangs. Reboot is needed to bring plex back to a stable state (until of course I try and transcode again).

Unsure where to take this to next to be honest. @ChuckPa you mentioned the i915 driver support team, does anyone know the mechanism for reporting a fault with them?

[ChuckPa]

https://bugzilla.kernel.org.

Give every possible bit of information possible,

uname -a , processor, the logs, kernel panic, the works.

[geeooff]

Got reboot issue too since Docker is installed.
It seems to be a pretty common issue since loooooong time.

I’m testing this in /etc/default/grub: GRUB_CMDLINE_LINUX_DEFAULT="reboot=pci" but i’m not sure the issue is fixed.

To report a bug you can go there:
Issues · drm / intel · GitLab

[Samwise41]

@DarthBJW @jmt089 @geeooff @ChuckPa I see there’s an open bug for GPU hangs on a Tiger Lake NUC, referencing VAAPI and Plex and more. Unfortunately looks like it hasn’t been touched since it was opened two months ago. I noticed it doesn’t have a platform tag (e.g. platform = TGL), maybe that’s why it’s escaping visibility.

GPU HANG: ecode 12:4:4ab6fff on Tiger Lake NUC11TNHv5 i5-1145G7 (#3398) ·...

Maybe all of us with NUC 11’s (or otherwise) just need to raise i915/VAAPI bugs and/or reference the existing ones and comment on them, generate some visible demand until there’s some progress… Or would that too pushy?..

Also if anyone wants to make some noise - How to file i915 bugs · Wiki · drm / intel · GitLab

Good to hear your NVME issue is sorted now btw @DarthBJW! After my last post I had another recurrence, so I disabled ASPM in the BIOS, and updated the kernel to 5.12.14 (from 5.12.9), and it’s been stable for 3 days without any issues so far, so that’s progress