Hardware Transcoding fails: "Cannot load libcuda.so.1" in Docker (RTX 2000 Ada, driver 590)

Plex Media Server
1

Description:
I am trying to enable NVIDIA hardware transcoding in Plex running inside Docker. The GPU is correctly detected and accessible inside the container, but Plex fails to initialize hardware acceleration and falls back to CPU transcoding.

Environment:

  • Plex Media Server: 1.43.0.10492

  • Docker image: plexinc/pms-docker:latest

  • Host OS: Arch Linux (kernel 6.12.68-1-lts)

  • GPU: NVIDIA RTX 2000 Ada Generation (AD107GL)

  • Driver: 590.48.01

  • CUDA: 13.1

  • NVIDIA Container Toolkit: installed and working

Docker run configuration:

  • --gpus all

  • --runtime=nvidia

  • NVIDIA_VISIBLE_DEVICES=all

  • NVIDIA_DRIVER_CAPABILITIES=all

What works:

  • nvidia-smi works inside the container

  • NVIDIA libraries are present:

    • libcuda.so.1

    • libnvcuvid.so.1

    • libnvidia-encode.so.1

  • ldd on these libraries shows no missing dependencies

  • Plex detects the GPU and attempts to use NVENC/NVDEC

What fails:
During playback requiring transcoding, Plex logs show:

[FFMPEG] Cannot load libcuda.so.1
[FFMPEG] Could not dynamically load CUDA
error: Operation not permitted

Additionally:

hardware transcoding: enabled, but no hardware decode accelerator found

As a result:

  • No hardware decoder/encoder is selected

  • Transcoding falls back to software (libx264)

  • GPU usage remains at 0%

Observations:

  • The issue occurs even with the official Plex Docker image

  • All required NVIDIA libraries are present and correctly linked

  • The failure occurs specifically when FFmpeg tries to dynamically load CUDA

  • The error suggests a runtime permission or compatibility issue rather than missing libraries

Steps already attempted:

  • Switched from linuxserver image to official Plex image

  • Verified NVIDIA container runtime and GPU access

  • Verified library presence and dependencies (ldd)

  • Tested multiple media files (HEVC, H264, SDR, no subtitles)

  • Confirmed no GPU usage during transcoding

Question:
Is this a known issue with:

  • newer NVIDIA drivers (590.x)

  • Ada generation GPUs

  • or Plex’s bundled FFmpeg build in Docker?

Any guidance on resolving CUDA loading failures inside Docker would be appreciated.

Attachments:

plex-debug-grep.txt (501.6 KB)

plex-fresh-hwtest.txt (528.7 KB)

Plex Media Server.log (1.6 MB)

2

Small update:

I got past the original Cannot load libcuda.so.1 error.

Inside the container, nvidia-smi works, and Plex now also sees the GPU device correctly. I also verified that libcuda.so.1 is now being loaded successfully.

However, hardware transcoding still fails at CUDA initialization.

Relevant log output:

DEBUG - [GPU] Got device: NVIDIA AD107GL [RTX 2000 / 2000E Ada Generation], nvidia@unknown, default true, best true, ID 10de:28b0:10de:1870@0000:15:00.0, DevID [10de:28b0:10de:1870], flags 0xe8
ERROR - [GPU] Cuda function failed with error unknown error

During an actual transcode attempt:

DEBUG - [Req#1ef/Transcode] Codecs: testing h264_nvenc (encoder)
DEBUG - [Req#1ef/Transcode] Codecs: hardware transcoding: testing API nvenc for device 'pci:0000:15:00.0' (NVIDIA AD107GL [RTX 2000 / 2000E Ada Generation])
ERROR - [Req#1ef/Transcode] [FFMPEG] - -> CUDA_ERROR_UNKNOWN: unknown error
DEBUG - [Req#1ef/Transcode] Could not create hardware context for h264_nvenc

DEBUG - [Req#1ef/Transcode] Codecs: testing hevc (decoder) with hwdevice nvdec
DEBUG - [Req#1ef/Transcode] Codecs: hardware transcoding: testing API nvdec for device 'pci:0000:15:00.0' (NVIDIA AD107GL [RTX 2000 / 2000E Ada Generation])
ERROR - [Req#1ef/Transcode] [FFMPEG] - -> CUDA_ERROR_UNKNOWN: unknown error

So the current situation is:

  • GPU is visible inside the container
  • Plex detects the NVIDIA GPU correctly
  • libcuda.so.1 is no longer missing
  • but both NVENC and NVDEC fail with CUDA_ERROR_UNKNOWN
  • Plex cannot create the hardware context

I also tested this not only with the LinuxServer image, but with the official Plex Docker image as well, and the result is the same.

At this point, the issue no longer seems to be just the missing library path. It now looks more like Plex/FFmpeg can see the GPU, but CUDA initialization itself fails.

For reference, this is the command I am currently using to start the LinuxServer container:

docker run -d \
  --name=Plex \
  --runtime=nvidia \
  --gpus all \
  --network macvlan_network \
  --ip 192.168.3.3 \
  -e PUID=1007 \
  -e PGID=1234 \
  -e UMASK=002 \
  -e TZ=Europe/Zurich \
  -e VERSION=docker \
  -e PLEX_CLAIM=EXAMPLE \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility \
  -e LD_LIBRARY_PATH=/usr/lib/plexmediaserver/lib:/usr/lib \
  -v /usr/lib:/host_usr_lib:ro \
  -v /srv/configs/plex/config:/config \
  -v /srv/configs/plex/data:/data \
  -v /srv/configs/plex/transcode:/transcode \
  -v /mnt/raid6/Media:/Media \
  --restart unless-stopped \
  lscr.io/linuxserver/plex:latest