Hardware HDR tonemapping still broken on 1.31.0.6654

Amazing @ChuckPa! Thanks so much for taking the time to troubleshoot with us.

Issues like this are annoying to have while being super frustrating to pin down and troubleshoot. My little home server is also my testing device for many things, I assumed this oddity it was my own doing (an assumption that is often correct).

If there are any other things I can help with or test just let me know.

Just adding that I’m having the same issues as above with a 13900K in Unraid with Intel GPU TOP plugin installed to allow access to iGPU. Using Linuxserver’s plex docker. HW transcoding works fine on 1080p content, or 4K with HDR tone mapping disabled, but won’t work with HDR tone mapping enabled. So doesn’t appear to be isolated to the processors noted above.

Tacking my logs on here.:

Mar 03, 2023 21:24:37.872	ERROR	[Req#bbe/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] Error reinitializing filters!
Mar 03, 2023 21:24:37.872	ERROR	[Req#bbf/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] Failed to inject frame into filter network: No such device
Mar 03, 2023 21:24:37.872	ERROR	[Req#bc0/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] Error while processing the decoded data for stream #0:0
Mar 03, 2023 21:24:37.865	ERROR	[Req#bbb/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] [AVHWDeviceContext @ 0x14f47b122dc0] Failed to get number of OpenCL platforms: -1001.
Mar 03, 2023 21:24:37.865	ERROR	[Req#bbc/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] [Parsed_hwmap_2 @ 0x14f486dfcfc0] Failed to created derived device context: -19.
Mar 03, 2023 21:24:37.865	ERROR	[Req#bbd/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] [Parsed_hwmap_2 @ 0x14f486dfcfc0] Failed to configure output pad on Parsed_hwmap_2
Mar 03, 2023 21:24:37.864	ERROR	[Req#bb2/Transcode/F6D4B821-CB29-4103-8042-1D766810739E/cb1c0366-c29c-4c5f-913b-f463ffda6d4a] [AVHWDeviceContext @ 0x14f47b122dc0] Failed to get number of OpenCL platforms: -1001.
Mar 03, 2023 21:24:37.328	INFO	[Req#b8d/Transcode/F6D4B821-CB29-4103-8042-1D766810739E] Preparing driver icr for GPU CoffeeLake-S GT2 [UHD Graphics P630]

Running an UnRaid system, docker image from linuxserver/plex:latest, which is Plex 1.31.2.6739.

HW transcoding works on everything except 4K HDR tone mapping. Reverting to version 1.29.0.6244 corrects it for me. Currently running the latest anyway, and telling my friends to get better hardware if they can’t direct play/stream…

Edit: Forgot to mention, this is a Xeon E-2146G (8th Gen).

As an update to my post above, switching to the docker container to hotio from linuxserver on my unraid machine with a 13900k allowed for HW transcoding to work in all instances (including hdr tone mapping from 4K to 1080p). This is on server version 1.31.1.6733.

For anyone on unraid, just note that the library folder structure is a little different between the two containers, so you’d need to manually move your metadata folders, etc. if switching.

Not sure the difference between linuxserver/official container and hotio, but happy it worked on my end. Hope that helps someone.

@knaack84 interesting, thanks for updating us. I will give 1.31.1.6733 a shot when I can today and report back. *update at bottom

I did have some odd HW detection issues with linuxserver in the past, and as such have mostly used hotio images (I use a few others too). To keep troubleshooting simple here and rule out oddities I used the official Plex images here.

*Tried the official and hotio images, both still exhibit the same issue for me. I’m glad your issue seems to be resolved @knaack84 :slight_smile:

@ChuckPa Any updates on this matter? Thank you

I tried updating to the latest beta version ( 1.31.3.6792) and unfortunately hw tone mapping has not been fixed (Unraid 6.11.5, intel j4105, kernel 5.19.17).

See the warning logs

Mar 14, 2023 20:07:40.671 [0x148c20967b38] VarovĆ”nĆ­ — [Req#96e/Transcode] [FFMPEG] - No quality level set; using default (20).

Mar 14, 2023 20:07:41.597 [0x148c20bfcb38] Chyba — [Req#f44/Transcode/2cuwfjvi78m59bhd0q9kb6fy/0d3b9b3d-b2db-4785-a83b-47c0cfad1d21] [AVHWDeviceContext @ 0x1504369c2300] No matching devices found.
Mar 14, 2023 20:07:49.123 [0x148c24db1b38] Chyba — [Req#f7e/Transcode/2cuwfjvi78m59bhd0q9kb6fy/0d3b9b3d-b2db-4785-a83b-47c0cfad1d21] [Parsed_tonemap_opencl_3 @ 0x1504369c16c0] Failed to finish command queue: -5.
Mar 14, 2023 20:07:49.628 [0x148c2663db38] Chyba — [Req#fa1/Transcode/2cuwfjvi78m59bhd0q9kb6fy/0d3b9b3d-b2db-4785-a83b-47c0cfad1d21] Error while filtering: I/O error
Mar 14, 2023 20:07:49.629 [0x148c24db1b38] Chyba — [Req#fa3/Transcode/2cuwfjvi78m59bhd0q9kb6fy/0d3b9b3d-b2db-4785-a83b-47c0cfad1d21] Failed to inject frame into filter network: I/O error
Mar 14, 2023 20:07:49.629 [0x148c21923b38] Chyba — [Req#fa7/Transcode/2cuwfjvi78m59bhd0q9kb6fy/0d3b9b3d-b2db-4785-a83b-47c0cfad1d21] Error while processing the decoded data for stream #0:0
Mar 14, 2023 20:08:01.585 [0x148c21278b38] Chyba — [Req#e16/Transcode] Unable to create output stream for transcode session, no segments found

@ChuckPa any updates?

Please attach the full ZIP file so I can give to Engineering.

This whole 4xxx series CPU should long be supported.
J4125’s in NAS’s are working just fine.

Since my last update, I have Unraid now (thanks to them graciously providing a Development/Support license to me) on a NUC 7

@ChuckPa He’s running a J4105 which is essentially the same as the J5005 mentioned in this thread. You had mentioned before that an engineer had found where it was having problems and he suggested possible fixes. Any update on that?

That’s why I’m confused. Everything identified is in PMS now

While this is a big ask on your (collectively) part, about all we can do is set a breakpoint in the debugger and trap PMS/FFMPEG when it gets to the point of deciding the hardware isn’t there.

I’m trying to figure out the debugger now because once in a container, it can’t discern the architecture (I need learn how to use it better).

If anyone is up to the challenge, I’ve got the instructions of what we need.
I’ve just given up on it for tonight (Was working on this earlier for a pair of RTX/8000’s in a container)

@ChuckPa Get some rest! When you’re up to it I’m sure there’s a few of us that will put the work in to get you what you need. Thanks for the help.

thanks. (so much for working ā€˜part-time’ :man_facepalming: lol )

1 Like

@ChuckPa see the logs
Plex Media Server Logs_2023-03-17_08-34-33.zip (2.2 MB)

@premikkoci

Do you have something without subtitles in it ?

(This has ASS subtitles and PMS is triggering on it / disabling hardware transcoding)

Mar 17, 2023 08:29:46.495 [0x14e61f9f1b38] DEBUG - We're going to try to auto-select a subtitle.
Mar 17, 2023 08:29:46.495 [0x14e61f9f1b38] DEBUG - Subtitles: Found a candidate subtitle language [en] for a foreign film
Mar 17, 2023 08:29:46.495 [0x14e61f9f1b38] DEBUG - Audio Stream: 40065, Subtitle Stream: 40066
Mar 17, 2023 08:29:46.504 [0x14e626475b38] DEBUG - [HttpClient/HCl#8a] HTTP/2.0 (0.2s) 200 response from GET https://plex.tv/api/codecs/easyaudioencoder?build=linux-x86_64-standard&deviceId=2b97a39f-f880-4240-9817-b132e874665c&oldestPreviousVersion=1%2E25%2E8%2E5663-e071c3d62&version=1847 (reused)
Mar 17, 2023 08:29:46.505 [0x14e61fdf7b38] DEBUG - [Req#18b/Transcode/GetFile/HCl#8b] HTTP requesting GET https://downloads.plex.tv/codecs/1847/linux-x86_64-standard/EasyAudioEncoder-linux-x86_64-standard.zip
Mar 17, 2023 08:29:46.507 [0x14e6275f3b38] DEBUG - Completed: [192.168.88.207:64418] 200 GET /hubs/promoted?contentDirectoryID=2&pinnedContentDirectoryID=1%2C2&includeMeta=1&excludeFields=summary&count=12&includeStations=1&includeLibraryPlaylists=1&includeRecentChannels=1&excludeContinueWatching=1 (10 live) #156 TLS GZIP 300ms 18145 bytes (pipelined: 1)
Mar 17, 2023 08:29:46.642 [0x14e626475b38] DEBUG - [HttpClient/HCl#8b] HTTP/2.0 (0.1s) 200 response from GET https://downloads.plex.tv/codecs/1847/linux-x86_64-standard/EasyAudioEncoder-linux-x86_64-standard.zip
Mar 17, 2023 08:29:46.742 [0x14e624ebeb38] INFO - [Req#18b/Transcode] CodecManager: starting EAE at "/tmp/pms-11cefde6-1f86-407b-a97b-c8a79757c182/EasyAudioEncoder"
Mar 17, 2023 08:29:46.742 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode/JobRunner] Job running: "/config/Library/Application Support/Plex Media Server/Codecs/EasyAudioEncoder-1847-linux-x86_64/EasyAudioEncoder/EasyAudioEncoder"
Mar 17, 2023 08:29:46.742 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode/JobRunner] In directory: "/tmp/pms-11cefde6-1f86-407b-a97b-c8a79757c182/EasyAudioEncoder"
Mar 17, 2023 08:29:46.742 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode/JobRunner] Jobs: Starting child process with pid 1014
Mar 17, 2023 08:29:46.743 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode] [Universal] Using local file path instead of URL: /tv/The Mandalorian/Season 3/The.Mandalorian.S03E02.2160p.DSNP.WEB-DL.x265.10bit.HDR.DDP5.1.Atmos-NTb.mkv
Mar 17, 2023 08:29:46.744 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode] TPU: hardware transcoding: final decoder: , final encoder: 
Mar 17, 2023 08:29:46.744 [0x14e624ebeb38] DEBUG - [Req#18b/Transcode/JobRunner] Job running: EAE_ROOT=/tmp/pms-11cefde6-1f86-407b-a97b-c8a79757c182/EasyAudioEncoder FFMPEG_EXTERNAL_LIBS='/config/Library/Application\ Support/Plex\ Media\ Server/Codecs/2584b51-4483-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxx4d71-8a6a-ea16bea73ecb "/usr/lib/plexmediaserver/Plex Transcoder" -codec:0 hevc -codec:1 eac3_eae -eae_prefix:1 azxwiwziegr9gxqbk1m47kn5_ -ss 0 -noaccurate_seek -analyzeduration 20000000 -probesize 20000000 -i "/tv/The Mandalorian/Season 3/The.Mandalorian.S03E02.2160p.DSNP.WEB-DL.x265.10bit.HDR.DDP5.1.Atmos-NTb.mkv" -map 0:0 -codec:0 copy -filter_complex "[0:1] aresample=async=1:ochl='stereo':rematrix_maxval=0.000000dB:osr=48000[0]" -map "[0]" -metadata:s:1 language=eng -codec:1 aac -b:1 256k -f dash -seg_duration 5 -dash_segment_type mp4 -init_seg_name 'init-stream$RepresentationID$.m4s' -media_seg_name 'chunk-stream$RepresentationID$-$Number%05d$.m4s' -window_size 5 -delete_removed false -skip_to_segment 1 -time_delta 0.0625 -manifest_name "http://127.0.0.1:32400/video/:/transcode/session/azxwiwziegr9gxqbk1m47kn5/1306f0bb-c93c-4b7a-8a44-e5d8a692f756/manifest?X-Plex-Http-Pipeline=infinite" -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -map 0:2 -metadata:s:0 language=eng -codec:0 ass -strict_ts:0 0 -f segment -segment_format ass -segment_time 1 -segment_header_filename sub-header -segment_start_number 0 -segment_list "http://127.0.0.1:32400/video/:/transcode/session/azxwiwziegr9gxqbk1m47kn5/1306f0bb-c93c-4b7a-8a44-e5d8a692f756/manifest?stream=subtitles&X-Plex-Http-Pipeline=infinite" -segment_list_type csv -segment_list_size 5 -segment_list_separate_stream_times 1 -segment_format_options ignore_readorder=1 -segment_list_unfinished 1 -fflags +flush_packets "sub-chunk-%05d" -start_at_zero -copyts -vsync cfr -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/azxwiwziegr9gxqbk1m47kn5/1306f0bb-c93c-4b7a-8a44-e5d8a692f756/progress

You guys and your containers are going to drive me to drink! :rofl:

I’ve been working on that for the last hour. No progress yet.

@ChuckPa Turned off subs
Plex Media Server Logs_2023-03-17_23-41-16.zip (2.9 MB)

Thank you!

@premikkoci

I took the log to the engineer.

She scolded me because I didn’t see what was right in front of me :see_no_evil:

Mar 17, 2023 08:30:22.690 [0x14e6244e0b38] ERROR - [Req#b41/Transcode/2ti4n2v2itmjqp7972x3vm0h/d26f4f4d-93e5-4568-96df-32f3ef0452c0] [Parsed_tonemap_opencl_3 @ 0x14fcaf3136c0] Failed to finish command queue: -5.
Mar 17, 2023 08:30:23.195 [0x14e6244e0b38] ERROR - [Req#b51/Transcode/2ti4n2v2itmjqp7972x3vm0h/d26f4f4d-93e5-4568-96df-32f3ef0452c0] Error while filtering: I/O error
Mar 17, 2023 08:30:23.196 [0x14e6244e0b38] ERROR - [Req#b54/Transcode/2ti4n2v2itmjqp7972x3vm0h/d26f4f4d-93e5-4568-96df-32f3ef0452c0] Failed to inject frame into filter network: I/O error
Mar 17, 2023 08:30:23.196 [0x14e6244e0b38] ERROR - [Req#b57/Transcode/2ti4n2v2itmjqp7972x3vm0h/d26f4f4d-93e5-4568-96df-32f3ef0452c0] Error while processing the decoded data for stream #0:0
Mar 17, 2023 08:30:24.260 [0x14e6279f9b38] DEBUG - Jobs: '/usr/lib/plexmediaserver/Plex Transcoder' exit code for process 1086 is 1 (failure)

See the I/O error ?

OpenCL (e.g. HDR tone mapping) failed. It ran out of resources.

How many other transcodes are using the video memory at that instant?
How much main memory is allocated to the container?

@ChuckPa, I get the same thing (below). But I don’t believe it’s actually running out of resources.

Mar 18, 2023 14:11:51.665 [0x7f4f13363b38] ERROR - [Req#bc/Transcode/57d07995-f6ba-47c8-adaf-c756adcb5539-252/54d638f8-b00d-4637-8ea5-d2bde21982ea] [Parsed_tonemap_opencl_3 @ 0x7f6c8446bc40] Failed to finish command queue: -5.
Mar 18, 2023 14:11:52.171 [0x7f4f10e85b38] ERROR - [Req#e2/Transcode/57d07995-f6ba-47c8-adaf-c756adcb5539-252/54d638f8-b00d-4637-8ea5-d2bde21982ea] Error while filtering: I/O error
Mar 18, 2023 14:11:52.171 [0x7f4f1291cb38] ERROR - [Req#e3/Transcode/57d07995-f6ba-47c8-adaf-c756adcb5539-252/54d638f8-b00d-4637-8ea5-d2bde21982ea] Failed to inject frame into filter network: I/O error
Mar 18, 2023 14:11:52.172 [0x7f4f13363b38] ERROR - [Req#e4/Transcode/57d07995-f6ba-47c8-adaf-c756adcb5539-252/54d638f8-b00d-4637-8ea5-d2bde21982ea] Error while processing the decoded data for stream #0:0

Only a single transcode happening (the one being attempted). I tried it with 4GB RAM reserved, and again with no reserve nor limit set (8GB in system), both with the same results. It does not run out doing HW tonemapping when using Plex server 1.29.0.6244 on the exact same hardware, using the same Docker settings, on the same video and playback device.

Here’s my full log files for 1.31.2.6810
1.31.2.6810 Plex Media Server Logs_2023-03-18_14-12-50.zip (292.8 KB)
(HW tonemapping not working)

and for 1.29.0.6244
1.29.0.6244 Plex Media Server Logs_2023-03-18_14-16-49.zip (352.7 KB)
(HW tonemapping is working)

Config and settings in both are exactly the same, the only difference being which offical Plex image is used.

Error -5 is Out of resources.

The documentation says ā€œinsufficient vmemā€.

IIRC – So here’s the challenge … video memory or OpenCL internal memory.
ToneMapping is a program which runs as an OpenCL program in the GPU of the CPU .

The engineer has to look into this more and find out why only these chips are failing.

It could also be something as simple (and stupid) as an Intel Media Driver bug (it’s not like Intel hasn’t done that to us before … lol)