NVidia transcoding stopped working on newest beta build. Worked fine previously. No changes to hardware or drivers. Nvdec / nvenc work fine in other applications (such as handbrake and ffmpeg).
Server Version#: 1.32.2.7002
OS: Debian
Kernel: 6.2.11
NVidia drivers: 530.41.03
GPU: RTX a2000 12GB
Need DEBUG logs ZIP please
Plex Media Server Logs_2023-05-10_12-38-41.zip (372.1 KB)
Thank you, sorry for delay.
Also seeing hw transcode not working, using a Nvidia GTX 1080. Works when I roll back to 1.32.1.6999.
May 10, 2023 13:55:32.079 [139903478717240] ERROR - [Req#ef/Transcode/fcamd47gscvoe4urpo48o5el/13cd4046-b534-4162-beac-409e9621cb16] [Parsed_tonemap_cuda_2 @ 0x7f14331f9d40] cu->cuLinkAddData(link_state, CU_JIT_INPUT_PTX, (void*)data, length, name, nb_options, options, option_values) failed → CUDA_ERROR_INVALID_SOURCE: device kernel image is invalid
May 10, 2023 13:55:32.080 [139903478717240] ERROR - [Req#fa/Transcode/fcamd47gscvoe4urpo48o5el/13cd4046-b534-4162-beac-409e9621cb16] [Parsed_tonemap_cuda_2 @ 0x7f14331f9d40] CUDA linker output: error : Memory space doesn’t match for ‘depth_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘fmt_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘range_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb_matrix’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘trc_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘luma_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb2rgb_passthrough’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb2rgb_matrix’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘tonemap_func’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’
May 10, 2023 13:55:32.080 [139903478717240] ERROR - [Req#fb/Transcode/fcamd47gscvoe4urpo48o5el/13cd4046-b534-4162-beac-409e9621cb16] Error while filtering: Generic error in an external library
May 10, 2023 13:55:32.080 [139903478717240] ERROR - [Req#fc/Transcode/fcamd47gscvoe4urpo48o5el/13cd4046-b534-4162-beac-409e9621cb16] Failed to inject frame into filter network: Generic error in an external library
May 10, 2023 13:55:32.080 [139903478717240] ERROR - [Req#fd/Transcode/fcamd47gscvoe4urpo48o5el/13cd4046-b534-4162-beac-409e9621cb16] Error while processing the decoded data for stream #0:0
May 10, 2023 13:55:32.127 [139903534947128] DEBUG - Jobs: ‘/usr/lib/plexmediaserver/Plex Transcoder’ exit code for process 646 is 1 (failure)
May 10, 2023 13:55:32.128 [139903482936120] DEBUG - Streaming Resource: Changing client to use software decoding
It appears the problem more specifically relates to hardware accelerated HDR tone mapping. I don’t have problems with SDR content, even if 4K.
Looks like you’re correct. If I disable tonemapping, hardware transcoding works again on 1.32.2.7002.
So it’s still blocking me from upgrading to the latest version as not all of my devices I watch from support HDR (after testing with disabling tonemapping I’ve once again rolled back to 1.32.1.6999) but good that the issue’s been narrowed down.
Definitely seeing the same thing. I’ll try to get debug logs in the morning if they are still needed.
Server Version#: 1.32.2.7002
OS: RHEL8
Kernel: 4.18.0-425.19.2
NVidia drivers: 530.30.02
GPU: Quadro P2000
Plex Media Server Logs_2023-05-11_07-51-12.zip (6.3 MB)
Your system isn’t detecting the Nvidia
May 11, 2023 07:11:21.278 [139642696284984] DEBUG - TPU: hardware transcoding: final decoder: , final encoder:
I’ll clear the logs and get you a more concise debug. It is using it for anything non-hdr. I should have looked at the logs before I exported this morning, there were two files being played when I saved that one was hardware transcoding like normal and confirmed with nvdia-smi along with showing hw in the dashboard, the other stomped the crud out of the CPU and drove the load up to 6 on the server when it typically rides in the .5 - 2 range
Shutdown plex, cleared the logs, applied the latest beta again. Had three people currently on the system two were getting hardware transcodes at the time. After it came back up both of those got new transcode jobs that showed up in nvidia-smi. I launched one of the files that was showing the hdr issue last night and forcing CPU transcoding instead of hardware assist and the CPU load shot through the roof again. check for bridgerton to hone in on any possible clues.
Looks like the fix is in testing for the next release of Plex:
Folks:
You guys type FAST! 
LOL
I’ve got an engineer’s build in hand (complete with wonky numbering so i can’t give it to you).
The production build is running now.
Soon as it’s done and I’ve kicked the tires, I will share it for all to beat on
LOL
Sadly still can’t hardware transcode an HDR video with tonemapping enabled on a GTX 1080 and 1.32.2.7088-319128df8, same error:
May 17, 2023 17:13:45.694 [140518800628536] ERROR - [Req#111/Transcode/9tuwahfd88i53d8pg901qqjc/b61f7442-fed3-4181-af73-3b8d0b42d385] [Parsed_tonemap_cuda_2 @ 0x7f589257db80] cu->cuLinkAddData(link_state, CU_JIT_INPUT_PTX, (void*)data, length, name, nb_options, options, option_values) failed → CUDA_ERROR_INVALID_SOURCE: device kernel image is invalid
May 17, 2023 17:13:45.695 [140518803262264] ERROR - [Req#117/Transcode/9tuwahfd88i53d8pg901qqjc/b61f7442-fed3-4181-af73-3b8d0b42d385] [Parsed_tonemap_cuda_2 @ 0x7f589257db80] CUDA linker output: error : Memory space doesn’t match for ‘depth_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘fmt_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘range_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb_matrix’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘trc_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘luma_src’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb2rgb_passthrough’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘rgb2rgb_matrix’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’t match for ‘tonemap_func’ in ‘tonemap.ptx’, first specified in ‘constants’
error : Memory space doesn’
May 17, 2023 17:13:45.695 [140518830410552] ERROR - [Req#118/Transcode/9tuwahfd88i53d8pg901qqjc/b61f7442-fed3-4181-af73-3b8d0b42d385] Error while filtering: Generic error in an external library
May 17, 2023 17:13:45.695 [140518795516728] ERROR - [Req#119/Transcode/9tuwahfd88i53d8pg901qqjc/b61f7442-fed3-4181-af73-3b8d0b42d385] Failed to inject frame into filter network: Generic error in an external library
May 17, 2023 17:13:45.695 [140518800628536] ERROR - [Req#11a/Transcode/9tuwahfd88i53d8pg901qqjc/b61f7442-fed3-4181-af73-3b8d0b42d385] Error while processing the decoded data for stream #0:0
May 17, 2023 17:13:45.740 [140518919326520] DEBUG - Jobs: ‘/usr/lib/plexmediaserver/Plex Transcoder’ exit code for process 480 is 1 (failure)
May 17, 2023 17:13:45.740 [140518864161592] DEBUG - Streaming Resource: Changing client to use software decoding
Let me know if you want me to test anything else, like try a different version or something
I tested on a P2200 . I know this works.
I will review your logs.
Same result. Not working on my end either. I will have to dump logs later.
I give up!
LOL
I’ve sent the logs to the engineer.
The morning build worked.
The afternoon (what you have) build fails.
I confirmed on our AV1 machine with Nvidia 3040 GPU
The engineer will NOT be happy.
Sorry folks, It foobar’d somewhere above my paygrade. 
It appears to be fixed in the newest release.
This morning, after the build was completed, I tested:
After testing verified, I sent message to the Engineering team and they finished their work to release it.
