Hardware Subtitle Burn-In HEVC

Plex Media Server
1

Server Version#: 1.41.0.8994 and 1.41.1.9065
Player Version#: 4.138.0

Hello,

While burning-in subtitles using hardware, HEVC encoded videos automatically switch from hardware encoding to software, and may produce garbled video. Disabling hardware encoding fixes the issue. Disabling subtitles fixes the issue. H264 files burn in subtitles with hardware with no issues. Installing 1.41.1.9065 from the “HEVC Encoding Forum Preview” thread does not fix the issue.

Media info:

    Codec HEVC
    Bitrate 1417 kbps
    Bit Depth 10
    Chroma Location left
    Chroma Subsampling 4:2:0
    Coded Height 800
    Coded Width 1920
    Color Primaries bt709
    Color Range tv
    Color Space bt709
    Color Trc bt709
    Frame Rate 23.976 fps
    Height 800
    Level 4.0
    Profile main 10
    Ref Frames 1
    Width 1920
    Display Title 1080p (HEVC Main 10)
    Extended Display Title 1080p (HEVC Main 10)

    Codec AAC
    Channels 2
    Bitrate 163 kbps
    Language English
    Language Tag en
    Audio Channel Layout stereo
    Profile lc
    Sampling Rate 48000 Hz
    Title Surround 7.1
    Display Title English (AAC Stereo)
    Extended Display Title Surround 7.1 (English AAC Stereo)

    Codec AC3
    Channels 2
    Bitrate 224 kbps
    Language English
    Language Tag en
    Audio Channel Layout stereo
    Sampling Rate 48000 Hz
    Title Surround 7.1
    Display Title English (AC3 Stereo)
    Extended Display Title Surround 7.1 (English AC3 Stereo)

    Codec PGS
    Bitrate 41 kbps
    Language English
    Language Tag en
    Display Title English (PGS)
    Extended Display Title English (PGS)

    Codec PGS
    Bitrate 76 kbps
    Language English
    Language Tag en
    Display Title English (PGS)
    Extended Display Title English (PGS)

    Codec PGS
    Language English
    Language Tag en
    Display Title English (PGS)
    Extended Display Title English (PGS)

GPU info:

root@plex3:~# lspci -v -s $(lspci | grep VGA | cut -d" " -f 1)
00:02.0 VGA compatible controller: Intel Corporation Iris Plus Graphics G1 (Ice Lake) (rev 07) (prog-if 00 [VGA controller])
        Subsystem: Dell Iris Plus Graphics G1 (Ice Lake)
        Flags: bus master, fast devsel, latency 0, IRQ 137
        Memory at 90000000 (64-bit, non-prefetchable) [size=16M]
        Memory at 80000000 (64-bit, prefetchable) [size=256M]
        I/O ports at 4000 [size=64]
        Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit-
        Capabilities: [d0] Power Management version 2
        Capabilities: [100] Process Address Space ID (PASID)
        Capabilities: [200] Address Translation Service (ATS)
        Capabilities: [300] Page Request Interface (PRI)
        Kernel driver in use: i915
        Kernel modules: i915

Kernel:

root@plex3:~# uname -ra
Linux plex3 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Console Log:

Oct 16, 2024 17:06:36.401 [139694791891768] Info — [Req#15a5/Transcode] Preparing driver imd for GPU Iris Plus Graphics G1 (Ice Lake)
Oct 16, 2024 17:06:44.716 [139694791891768] Error — [Req#1628/Transcode/bjj9p5boi49203qv0az5dw84/823ca9b5-0e8c-48b9-9bb9-729322ac9d28] [hwupload @ 0x7fb75947f380] Failed to upload frame: -22.
Oct 16, 2024 17:06:44.717 [139694785448760] Error — [Req#162c/Transcode/bjj9p5boi49203qv0az5dw84/823ca9b5-0e8c-48b9-9bb9-729322ac9d28] Error while filtering: Invalid argument
Oct 16, 2024 17:06:44.717 [139694795995960] Error — [Req#162f/Transcode/bjj9p5boi49203qv0az5dw84/823ca9b5-0e8c-48b9-9bb9-729322ac9d28] Failed to inject frame into filter network: Invalid argument
Oct 16, 2024 17:06:44.718 [139694789782328] Error — [Req#1632/Transcode/bjj9p5boi49203qv0az5dw84/823ca9b5-0e8c-48b9-9bb9-729322ac9d28] Error while processing the decoded data for stream #0:0
Oct 16, 2024 17:06:49.485 [139694791891768] Warning — [Req#168f] Invalid library metadata ID 5d776b0c9ab544002150686d passed.
2

What’s the CPU and runtime environment ? Iris XE graphics are normally found in the newer CPUs which require a 6.0 or higher kernel to function correctly.

What’s the distro?

Might it be possible to obtain the DEBUG logs ZIP file (full zip) which captures this happening? Diagnosing from snippets is not possible unless a glaring fault.

3

Environment:
Plex server is running directly on a laptop.

Distro:
Ubuntu Jammy 22.04.5 LTS

Debug Logs:
Attached. HW encoding stopped @ 13:41:13 CT. Garbled video started @ 13:42:30 CT.

CPU:

root@plex3:/# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 126
model name      : Intel(R) Core(TM) i5-1035G1 CPU @ 1.00GHz
stepping        : 5
microcode       : 0xc6
cpu MHz         : 1200.000
cache size      : 6144 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 27
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm md_clear flush_l1d arch_capabilities
vmx flags       : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple pml ept_mode_based_exec tsc_scaling
bugs            : spectre_v1 spectre_v2 spec_store_bypass swapgs itlb_multihit srbds mmio_stale_data retbleed eibrs_pbrsb gds bhi
bogomips        : 2380.80
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

Plex Media Server Logs_2024-10-16_18-43-07.zip (910.2 KB)

4

Interesting.

I have an AlderLake which has no difficulty.
I wonder if the Intel PCI IDs for your machine are correct?

Do you have difficulties burning subtitles with the Public (9057) release ?

Screenshot from 2024-10-16 15-16-14
Screenshot from 2024-10-16 15-16-141087×460 83.7 KB

Here, you’ll see I’m encoding to H264 (deliberately) to help diagnose your issue.

I can provide you with a sample of this file if you wish (it’s full BR-Rip)

5

Your source video is HEVC? 10-bit, 4:2:0?

I just tried with 1.41.1.9057, same issue.

Your question about Intel PCI IDs is interesting. According to the Dell spec PDF for my laptop, the Intel Iris Plus Graphics should only be available on i7 models. So, my laptop should actually have Intel UHD Graphics with the i5. I picked this laptop because the i5 Ice Lake QuickSync hardware should support encode/decode for HEVC 10-bit, 4:2:0.

https://www.intel.com/content/www/us/en/docs/onevpl/developer-reference-media-intel-hardware/1-1/overview.html#ENCODE-OVERVIEW-10TH

6

My question about the PCI IDs is here:

  1. PMS does see the GPU (it scanned the PCI bus)

Oct 16, 2024 18:39:02.336 [139692567690040] DEBUG - [GPU] Got device: Iris Plus Graphics G1 (Ice Lake), intel@builtin, default true, best true, ID 8086:8a56:1028:097c@0000:00:02.0, DevID [8086:8a56:1028:097c], flags 0x3ae7

  1. I presume subtitle burning of H.264 works?

  2. I also presume encoding to HEVC also works? (This ‘9065’ build.)

The next question is the table (inside the transcoder which says the GPU can HEVC + Subtitle burn.

I don’t have access to the source code branch my teammate is developing this on and Engineering is off-site this week for conference.

I’m sending you, (via PM) a copy of my test file with embedded PGS subs.
It’s a good ‘stress test’.

You should be able to replicate my results.

Please let me know how it turns out.

7
  1. Does this mean there is no issue that the GPU is probably misidentified?
  2. Actually no, I spent yesterday doing a bunch more testing and hardware subtitle burn-in can fail in the same way for some H264 rips.
  3. Most of my testing has been using FireFox, but I installed the Windows app and HEVC works. Same issues with hardware burn-in subs though.

Your test file works fine.

I created a 5 minute test rip that reliably reproduces the error. I’ll PM it to you in a minute.

8

I did some checking in the source code and the different “branches”:

  1. Is your GPU misidentified ? NO.
  2. Is your GPU lacking definition info (which we get from Intel) YES.
    – My Nvidia RTX2000 is also lacking.
  3. Should HW subtitles and HEVC encoding work? NO. The HEVC work started before the subtitle work was released (two different development branches in the CM tree)
  4. Should HW subtitles work for H.264? Not guaranteed. Again, two different branches in the CM tree).
    – Some work was completed and made it into production release.
    – Chris took a snapshot of production release (created his development branch) and started adding HEVC.
    – At some point, Chris will merge his work back into the main source tree and we’ll have working HEVC + Subtitles.

This is a tech preview of which HEVC encoding is the primary target

Managing our own expectations and wanting everything at this point is just a touch premature. It’s close… and is what Chris will be finishing when he’s back from Plex’s annual conference.

All I can ask is to please be patient. We’re almost at the point where HEVC and subtitle burning will be in the same build

9

Understood. Hardware subtitle burn-in is still a work in progress, especially for HEVC.

Can you confirm that you were able to replicate the issue with the file I sent?

Also, is there an option to turn off hardware subtitles? If not, can you tell me what is the most recent version of plex server before hardware subtitles was included? I may need to disable or revert since so many of my files cause issues.

10

@arcimus

I can confirm, which is how it currently should be:

  1. HEVC output – no hardware subtitle burn-in (HEVC preview thread)
  2. H.264 output – correct HW subtitle burn-in (Public PMS - build 8994)

Turning off subtitles is at the player side. Each player can disable subtitles.

11
  1. Can you confirm what you mean by “no hardware subtitle burn-in (HEVC preview thread)”. I’m running server 1.41.1.9065 playing the HEVC file you sent with subtitles and it looks like hardware is doing the subtitles. Video encode says (hw) and CPU usage is the same with and without.
  2. I noted in a previous message that H.264 can also fail when doing hardware subtitles. I created a test file and will PM you in a moment.
  3. Turning off subtitles on the player is not an option for me. I want subtitles. Is there an option to turn off hardware subtitles? If not, can you tell me what is the most recent version of plex server before hardware subtitles was included?

Thanks for confirming that you also had the issue with the HEVC file I sent.

12

The team lead for HW transcoding is back.

We have confirmed:

  1. HW encode to Plex/web on Windows and MacOS :white_check_mark:
  2. HW encode to Plex/Web on Linux :x: (Manifest 4032 error)

Be careful about stating “H.264 w/ hardware subtitles”.
– In the context of public release 8994, this should be working
– In the context of HEVC build 9065, this is not guaranteed and likely to miss for most when encoding to HEVC output.

I’ve asked my team lead but software subtitle burn should be the default.

The design currently actually does the subtitle burn in software WITHOUT disturbing the HW transcoding pipeline.

We actually HW decode → Software burn the subtitle → HW encode the resultant frame → Send to player.

In the past, it was either ALL HW or ALL SW

13

HEVC hardware subtitles not guaranteed. You confirmed that you reproduced the issue with the HEVC files I sent. Understood.

Did you have a chance to test the H.264 file that reliably gives me issues with hardware subtitles?

To answer my own question, it looks like 1.40.5.8897 is the most recent plex server version that doesn’t have hardware subs.

14

Any updates?

Did you have a chance to test the H.264 file that reliably gives me issues with hardware subtitles?

15

Apologies for not responding sooner.

I was able to test the file.

The H.264 file (TEST3) also gives me software transcoding fallback.
There is definitely something neither my Nvidia or AlderLake like in build 9057 (Public) or any of the HEVC builds.

It starts ok but will then fallback.

arcimus-Screenshot from 2024-10-29 11-41-21
arcimus-Screenshot from 2024-10-29 11-41-21544×611 54.9 KB

When I go grab a full HEVC file from my library, I don’t have any issues with hardware whatsoever.

Screenshot from 2024-10-29 11-47-46
Screenshot from 2024-10-29 11-47-46709×604 78 KB

I hate to say it as I’ve nothing hard & concrete yet to point at (I’m not a video engineer) but experience is telling me it’s something with how it was reduced by handbrake.

I am leary of the “webm” file container. It is not as accommodating as MKV.

Is there any way you can retest this using MKV output?

16

Thanks for confirming. Out of curiosity, do you also get garbled video several seconds later? This consistently happens with both the HEVC and H.264 sample files I sent.

Both files I sent were MKV. If you would like me to resend, or retest, let me know.

17

Yes, I did get a opening shot of garbled – until the next reference frame put it back in order.

Hmm. You sent MKV but I got WEBM. not cool.

Would it be possible to put them in a ZIP then share with me?

18

Mine typically open fine, but then I get garbled video 5-30 seconds after subs switch to software.

Yup, sent PM!

20

Technically, webm is mkv. It’s just that the webm spec does restrict the types of codecs and the number of tracks you can use in a file.
Only if the video stream uses the VP8, VP9, or AV1 codecs it can be considered a valid webm file.

So if yours have H.264 or H.265 video, then some code in-between upload and download is messing up.

21

Played for 15 minutes. No issues with hardware subs.

Played for 15 minutes. No issues with hardware subs.

This does match with my experience where some of my own files are just fine, but others aren’t- all made using the same handbrake settings. If you have anything else specific you would like me to test, let me know.