Docker compose 1.29.0 - Hardware Transcoding

Plex Media Server
1

Server Version#: 1.22.2.4282

I updated docker-compose to the latest version and it seems like hardware transcoding is no longer working with the new schema for adding gpus. This is with both the runc and nvidia runtimes.

version: '3.8'
services:
  plex:
    container_name: plex
    image: plexinc/pms-docker:beta
    restart: unless-stopped
    network_mode: host
    environment:
      - TZ=$TZ
      - PLEX_UID=$PUID
      - PLEX_GID=$PGID
    volumes:
      - /data/.config/plex:/config
      - /mnt/media:/media
      - /dev/shm:/transcode
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

nvidia-smi works fine in the container

$ docker exec -it plex nvidia-smi
Sun Apr 11 21:48:36 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.67       Driver Version: 460.67       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000        Off  | 00000000:01:00.0 Off |                  N/A |
| 48%   37C    P8     7W /  75W |     55MiB /  5057MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

However, transcodes are now using software instead of hardware.
image

2

I see this is Plex Media Server.log

Apr 11, 2021 23:56:20.446 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: testing h264_vaapi (encoder)
Apr 11, 2021 23:56:20.446 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Apr 11, 2021 23:56:20.446 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Generic error in an external library
Apr 11, 2021 23:56:20.446 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: testing h264_nvenc (encoder)
Apr 11, 2021 23:56:20.446 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvenc
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice vaapi
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Generic error in an external library
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice nvdec
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvdec
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Apr 11, 2021 23:56:20.447 [0x7f10f9ffb700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred
3

Further evidence that this is a plex specific problem is that emby has no problem making use of the GPU acceleration.

  emby:
    image: emby/embyserver:beta
    container_name: emby
    restart: unless-stopped
    ports:
      - 8096:8096
      - 8920:8920
    environment:
      - TZ=$TZ
      - UID=$PUID
      - GID=$PGID
    volumes:
      - /data/.config/emby:/config
      - /mnt/media:/media
      - /dev/shm/emby:/transcode
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

image

4

While I don’t have Nvidia (NUC8i7-HVK here),
I have no difficulties with hardware transcoding in Docker containers.

It is important to note the Beignet libraries are installed in the container for proper HDR tone mapping in hardware.

sudo docker stop plex
sudo docker rm plex

sudo docker run \
-d \
--name plex \
--network=host \
-e TZ="EST" \
-e LANG="en_US.UTF-8" \
-e ADVERTISE_IP="http://<hostIPAddress>:32400/" \
-e PLEX_UID=1001 \
-e PLEX_GID=1001 \
-e PUID=1001 \
-e PGID=1001 \
-h dockerplex \
-v /dockerplex:/config \
-v /dockerplex/tmp:/tmp \
-v /dockerplex/transcode:/transcode \
-v /nas:/data \
--device=/dev/dri:/dev/dri \
plexinc/pms-docker

docker start plex
docker update --restart=unless-stopped plex

Does this help?

Hint: Do you have the Nvidia drivers installed in the container?

5

No that doesn’t help. Since docker v19.03 you’ve been able to use the --gpus flag. Meaning gpu support was built natively into the runc runtime. It was no longer necessary to specify the nvidia runtime and use nvidia specific environment variables.
This option has finally made it to docker compose via the new schema used above. However, it doesn’t work with plex specifically. While it works with other images like emby. Nvidia drivers (v460.67) are installed on the host system as you can see from the nvidia-smi command run directly in the plex container. As well as the latest version of nvidia container toolkit. Also, hardware transcoding works fine when using the old method of editing the default runtime in /etc/docker/daemon.json and using nvidia specific environment variables.

6

I just looked back through the history of the packaging.
Nothing has changed in a long time.

That might be part of , or even the root of, the problem.

I just created a test case using both Emby and Plex.
One thing Emby does, which Plex doesn’t do, is GPU device selection.

In light of a recent experience with ESXi and GPU passthrough, I did have to specify the specific HardwareDevicePath="/dev/dri/renderD129" in my case (being the second device discovered.

I don’t know if this has any applicability to how the Nvidia drivers present themselves in your desktop Linux. I did see them present themselves in /dev/dri and /sys/module on QNAP (of all things).

If you can help me identify what needs to be done, I will gladly write it up and submit it.

7

Passing the GPU is a pretty different animal than exposing /dev/dri. Much of it is taken care of with the nvidia container toolkit.

It may be a problem with the plex image rather than plex itself. The linuxserver image appears to have no problem using hardware transcoding. /data/.config/plex-lsio is a direct copy of /data/.config/plex

  plex-lsio:
    container_name: plex-lsio
    image: linuxserver/plex
    restart: unless-stopped
    network_mode: host
    environment:
      - TZ=$TZ
      - PUID=$PUID
      - PGID=$PGID
    volumes:
      - /data/.config/plex-lsio:/config
      - /mnt/media:/media
      - /dev/shm:/transcode
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

image

edit: same problem with the plexinc/pms-docker image (no :beta tag)

9

I chatted with the engineer who developed and supports the docker images.

He said, in short, that the Docker / Nvidia is such a <Insert denegrating words here> because it broke everything else.

For that reason, it has been left as manual.

10

Meaning what exactly? I’m fine using the linuxserver image, and just an FYI, the docker-compose update isn’t yet available in the Ubuntu apt repository. I don’t know how many people will update their containers to the new GPU schema when it does become available. I maintain an unrelated docker image that also uses GPUs, so I was trying out other containers with the new schema and noticed the issue with plex, so I wanted to report it.

11

Means exactly:

You pass through the device manually.

I have two GPUs (renderD128 and renderD129) so I can write it this way)

--device=/dev/dri:/dev/dri \

In the containerized PMS, if I want to use the 2nd GPU card, I specify:

HardwareDevicePath="/dev/dri/renderD129"
12

AFAIK that’s not exactly possible when using nvidia GPUs as the drivers are not included in the image. That process is only applicable with integrated GPU chipsets.

13

They aren’t and never are/were.

  1. Install the appropriate Nvidia drivers for your system.
  2. Create the container which points to the device.

Are you trying to create your own image file?

14

No, I’m not interested in making another plex image. I’m stating that nvidia GPUs won’t work in the plex image without the appropriate drivers. Drivers installed on the host system will not be passed to the container without nvidia-container-toolkit.

15

Then install the toolkit

PMS in Docker, on a Ubuntu/Debian host, can be upgraded using dpkg / apt-get.

There’s nothing blocking what you’re trying to do AFAIK.

Time to start adding to the container as you need.

May I ask what you’re trying to do that the native package cannot do?

16

Installing the toolkit itself doesn’t inject drivers in every container automatically. You still need to A) use the --gpus flag with docker run v19.03+. Or B) use the schema in the OP with docker-compose v1.28.0+.

The “old” way in docker-compose required changing the default docker runtime from runc to nvidia (a wrapper for runc that accepts nvidia specific environment variables) in /etc/docker/daemon.json, or specifying --runtime: nvidia in the docker run command. As well as using NVIDIA_DEVICES and NVIDIA_DRIVER_CAPABILITIES environment variables. This method still works correctly with the official plex image.

Just “manually” passing the device path doesn’t work, because there are no drivers installed in the image itself.

  plex2:
    container_name: plex2
    image: plexinc/pms-docker
    restart: unless-stopped
    network_mode: host
    environment:
      - TZ=$TZ
      - PLEX_UID=$PUID
      - PLEX_GID=$PGID
    volumes:
      - /data/.config/plex2:/config
      - /mnt/media:/media
      - /dev/shm:/transcode
    devices:
      - /dev/dri/card0:dev/dri/card0

Apr 13, 2021 23:36:46.345 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: testing h264_vaapi (encoder)
Apr 13, 2021 23:36:46.345 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Apr 13, 2021 23:36:46.345 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Generic error in an external library
Apr 13, 2021 23:36:46.345 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: testing h264_nvenc (encoder)
Apr 13, 2021 23:36:46.345 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvenc
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice vaapi
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API vaapi
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Generic error in an external library
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: testing h264 (decoder) with hwdevice nvdec
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: testing API nvdec
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] ERROR - [Transcode] [FFMPEG] - Cannot load libcuda.so.1
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] ERROR - [Transcode] [FFMPEG] - Could not dynamically load CUDA
Apr 13, 2021 23:36:46.346 [0x7fe512ffd700] DEBUG - [Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: Unknown error occurred

image

Installing the drivers by entering the container docker exec -it plex /bin/bash is not “the docker way”. It’s a bad idea for a couple of reasons, the main one being that the drivers would have to be manually installed every single time the image is updated or replaced. If that’s your only option, you’re better off building your own image. Besides, the drivers and GPU card are accessible when using nvidia-container-toolkit.

As stated in the OP, I’m just raising the fact that the official plex image fails to use hardware transcoding with the new docker-compose 1.29.0 deploy schema, while other images (like emby, and even linuxserver plex) have no problem with it.

17

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.