Plex HW acceleration in LXC container - anyone with success?

I spent few hours to configure Intel GPU on my LXC container. (according to presented procedure)… but unfortunately with no success.

here is my output from container:

root@TVH:~# vainfo
error: can't connect to X server!
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.39 (libva 1.7.3)
vainfo: Driver version: Intel i965 driver for Intel(R) Sandybridge Desktop - 1.7.3
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc
root@TVH:~# ls -la /dev/dri
total 0
drwxr-xr-x 2 root root       80 Aug 12 13:20 .
drwxr-xr-x 9 root root      540 Aug 12 13:20 ..
crw-rw-rw- 1 root root 226,   0 Aug 12 13:20 card0
crw-rw-rw- 1 root root 226, 128 Aug 12 13:20 renderD128
root@TVH:~#


root@TVH:~# intel_gpu_top
(intel_gpu_top:568) intel-mmio-CRITICAL: Test assertion failure function intel_mmio_use_pci_bar, file ../../lib/intel_mmio.c:145:
(intel_gpu_top:568) intel-mmio-CRITICAL: Failed assertion: !(error != 0)
(intel_gpu_top:568) intel-mmio-CRITICAL: Last errno: 30, Read-only file system
(intel_gpu_top:568) intel-mmio-CRITICAL: Couldn't map MMIO region
Stack trace:
  #0 [_init+0x7959]
  #1 [_init+0x55c0]
  #2 [_init+0xf7e]
  #3 [__libc_start_main+0xf1]
  #4 [_init+0x1d42]
  #5 [<unknown>+0x1d42]
Test (null) failed.
**** DEBUG ****
(intel_gpu_top:568) intel-chipset-DEBUG: Test requirement passed: pci_dev
(intel_gpu_top:568) intel-mmio-CRITICAL: Test assertion failure function intel_mmio_use_pci_bar, file ../../lib/intel_mmio.c:145:
(intel_gpu_top:568) intel-mmio-CRITICAL: Failed assertion: !(error != 0)
(intel_gpu_top:568) intel-mmio-CRITICAL: Last errno: 30, Read-only file system
(intel_gpu_top:568) intel-mmio-CRITICAL: Couldn't map MMIO region
****  END  ****
FAIL (-1.000s)

I dont really know what is the issue :slight_smile:

here is the output from host:

root@pve:/# intel_gpu_top

                   render busy:   0%:                                        render space: 0/131072
                bitstream busy:   0%:                                     bitstream space: 0/131072
                  blitter busy:   0%:                                       blitter space: 0/131072

                          task  percent busy
                                                              vert fetch: 0 (0/sec)
                                                              prim fetch: 0 (0/sec)
                                                           VS invocations: 0 (0/sec)
                                                           GS invocations: 0 (0/sec)
                                                                GS prims: 0 (0/sec)
                                                           CL invocations: 0 (0/sec)
                                                                CL prims: 0 (0/sec)
                                                           PS invocations: 0 (0/sec)
                                                           PS depth pass: 0 (0/sec)

So I’m no expert on passing the intel gpu through but based on the “Read-only file system” my guess would be the permissions aren’t getting set for the container to be allowed to write through to the device.

You mean about “unprivileged container” ?
I’m using also PCIe passthrough and everything works fine .

Which link did you follow? The one in this thread or the original Linux Tips

ix-9xxx cpus are not working yet.

Hi, love your guide, but had a problem after finishing all the steps, without any failure.

So I can see my GTX960 in the LXC (Ubuntu) and “nvidia-smi” is working well in there. But when I run a movie/series etc. and starting transcoding using the HW acceleration, the GPU does nothing (Well, the memory start filling a little bit, but just about 0-1% GPU-Utility).

I know that the GPU can handle >20x transcoding 1080p → 240p, because I get it run before, but after upgrading my machine I don’t know my steps done before :confused:

Hope someone can help me with this problem.

Are you looking at nvidia-smi in the LXC container or the host?

It won’t show any activity in the LXC container while transcoding, you’ll have to look at it on the host to see if it’s transcoding.

1 Like

I was look on the host, as well as on LXC.
Here are some pictures, which show that the HW acceleration is (more or less) just working on two streams. While going through your ‘guide’ the patch also said “Patched!”.

Yesterday I also have redone all the steps, but it was giving the same result.


I wouldn’t say that’s unusual, I get 0% utilization on my 980Ti decoding and encoding from 1080p to SD h264.

image image

1 Like

Okay, so it seems a problem with the patch.
Do you know a way to fix this problem? I’m not sure what I have made wrong…

Here are the results of the “patch-steps”:


I’m not sure why you think something is wrong. Everything you’ve shown looks like it works to me.

1 Like

Well, it is still not working, so something need to be wrong :stuck_out_tongue:
I will try to find a solution and then port it here :wink:

@evomod It’s perfectly normal for the nvidia-smi utility to show 0% GPU activity while transcoding. It is only showing you a point in time snapshot of activity of the GPU. And since the transcoder does not keep a sustained load running on the GPU (because the GPU can transcode several times faster than real time, which is the point) the GPU/hardware transcoder will go idle after it has a filled a buffer that matches the setting you have in ‘Transcoder default throttle buffer’. Which typically will only take a few seconds. Then after some percentage of that buffer is depleted from playback, it will burst again to refill it. If you’d like to watch activity of the GPU as you start a transcoded stream to see this in action, use this command:

watch -n 2 -d -c ‘nvidia-smi -q -d UTILIZATION’

As far load goes, think about it, if your GPU can handle 20 simultaneous transcodes and you’re only running 2, it’s going to be idle AT LEAST 90% of the time.

1 Like

I found my problem and solved it.

The instruction @constiens has given were 100% right.
I can not say why, but I have made a mistake with the

ls -l /dev/nv*

The output was “195” for nvidia-modeset and “236” for nvidia-uvm.
But I typed 234 instead of 236 in for the LXC.
I thought I have checked everything twice, but forgot about this command.

My bad … Thank you guys :wink: .

@evomod Very cool. I actually have had that same exact issue occur on my proxmox host. Interesting that it was showing you it was kind of working. For me it was after a kernel update on the host and I had to reinstall the nvidia driver.

But for me, basically the nvidia device ID does not always stay the same. I’m not sure if it relates to driver version or the order of which devices are initialized at boot. I’ve seen my GPU bounce between 236 and 237. So my solution has been to put all the device IDs the GPU has shown up as in the LXC config file and leave them all in simultaneously.

example:

lxc.cgroup.devices.allow: c 195:* rwm
lxc.cgroup.devices.allow: c 236:* rwm
lxc.cgroup.devices.allow: c 237:* rwm

I’ve not seen other devices use those ID’s so this config doesn’t seem to interfere with anything else.

1 Like

That could have happened to me too.
I thought that the 234 was correct, so not looking after this anymore. Doing a restart and the hardware transcoding was not working, because of the change in numbers. I will also add some more numbers to the config file because in some weeks I do not think about this and have the same issue again :confused: … never had this problem with proxmox 5 and my old config, strange.

This is interesting because I just did the same thing on mine too include all the numbers I get. I’m in travel right now but I’ll update the guide with a note with this suggestion when I get back.

Mine typically ping pong between two different ids

1 Like

Guide updated to include the some general notes from the various questions above.

Great write up. Got me unstuck when I was trying to figure this out. Wish I had found it earlier.

Hi Guys,

i’m still unlucky to get this to work:

root@proxmox:~# nvidia-smi
Sun Oct 27 23:23:22 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40       Driver Version: 430.40       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 750     On   | 00000000:01:00.0 Off |                  N/A |
| 33%   34C    P8     1W /  38W |      1MiB /   981MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
root@plex:~# nvidia-smi
Sun Oct 27 22:23:53 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40       Driver Version: 430.40       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 750     Off  | 00000000:01:00.0 Off |                  N/A |
| 33%   34C    P8     1W /  38W |      1MiB /   981MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
root@proxmox:~# cat /etc/pve/lxc/109.conf
rch: amd64
cores: 4
hostname: plex
memory: 4096
mp0: /media/Y/m,mp=/media/Y/m
mp1: /media/X/m,mp=/media/X/m
nameserver: 10.10.0.1
net0: name=eth0,bridge=vmbr0,gw=10.10.0.1,hwaddr=66:6C:XX:XX:XX:XX,ip=10.10.0.109/16,type=veth
onboot: 1
ostype: debian
rootfs: local:109/vm-109-disk-1.raw,size=20G
searchdomain: local.domain.de
swap: 0
lxc.cgroup.devices.allow = c 226:* rwm
lxc.cgroup.devices.allow = c 195:* rwm
lxc.cgroup.devices.allow = c 29:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/dri/card1 dev/dri/card1 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.autodev: 1
lxc.hook.autodev: /var/lib/lxc/109/mount_hook.sh

root@proxmox:~# cat /var/lib/lxc/109/mount_hook.sh
mkdir -p ${LXC_ROOTFS_MOUNT}/dev/dri
mknod -m 666 ${LXC_ROOTFS_MOUNT}/dev/dri/card1 c 226 1
mknod -m 666 ${LXC_ROOTFS_MOUNT}/dev/dri/card0 c 226 0
mknod -m 666 ${LXC_ROOTFS_MOUNT}/dev/dri/renderD128 c 226 128
mknod -m 666 ${LXC_ROOTFS_MOUNT}/dev/fb0 c 29 0

did i forgot something?

Thanks!

BR Daniel

I don’t use the mount_hook method, so I’m not sure how to help you there. Have you tried doing it the way I posted here: Plex HW acceleration in LXC container - anyone with success?