PMS 1.31.3.6868 crash: libc++abi: terminating with uncaught exception

Thanks for the quick callout. It is right there in the release notes, but I just do not keep up with the [awesome and consistent] release cycles: Plex Media Server - #547 by gbooker02

→ Its still broken with 525 !

Interesting, as the Debian 10 nvidia (418.226.00) was even older, and [seemed to] work fine, so I was not expecting PMS 1.31.3 to fail with a newer version. Debian 12 upstream is at 525.89.02, which fits nicely into my simple evergreening image.

To avoid the upgrade-nvidia-into-unsupported version during the Debian distribution upgrade scenario, where PMS version has not changed, I might suggest to enhance the following check with some version dependency insight and a warning:

PlexMediaServer install: Nvidia GPU card: Found

Here is a quick set of steps for other Debian 11 users who are looking to maintain a package-based [evergreening] configuration for their PMS.

First you need to add the Debian 12 repositories, and the new non-free-firmware if you want to pull the latest firmware-nvidia-gsp. Adding the following to your apt sources.list

Debian 12 Pull for Nvidia

deb Index of /debian bookworm main non-free-firmware contrib non-free

To ensure your Debian 11 system remains based on Debian 11, you will need some apt preferences. Due to the number of nvidia packages (dkms, mps, driver, libraries, cuda, etc.) we use wildcards to help. Additionally, a few Debian 12 core bits will be required (dkms, libc6) to meet kernel module building dependencies. The result is apt upgradeable without requiring any third party binary blobs fron nvidia or otherwise.

media:/etc/apt/preferences.d# cat nvidia
Package: *
Pin: release n=bookworm
Pin-Priority: -1

Package: nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: libcuda*
Pin: release n=bookworm
Pin-Priority: 900

Package: libnv*
Pin: release n=bookworm
Pin-Priority: 900

Package: libxnv*
Pin: release n=bookworm
Pin-Priority: 900

Package: libegl-nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: libgl1-nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: libgles-nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: libglx-nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: xserver-xorg-video-nvidia* libwayland-client* libffi8 libjansson4
Pin: release n=bookworm
Pin-Priority: 900

Package: firmware-nvidia*
Pin: release n=bookworm
Pin-Priority: 900

Package: dkms libc6 libc-bin libc-dev-bin libc6-dev libc6-i386 locales locales-all libc-l10n
Pin: release n=bookworm
Pin-Priority: 900

And this is what I ended up with using the preferences above

media:/etc/apt/preferences.d# dpkg -l | grep nvidia
ii firmware-nvidia-gsp 525.89.02-1 amd64 NVIDIA GSP firmware
ii glx-alternative-nvidia 1.2.1~deb11u1 amd64 allows the selection of NVIDIA as GLX provider
ii libegl-nvidia0:amd64 525.89.02-1 amd64 NVIDIA binary EGL library
ii libgl1-nvidia-glvnd-glx:amd64 525.89.02-1 amd64 NVIDIA binary OpenGL/GLX library (GLVND variant)
ii libgles-nvidia1:amd64 525.89.02-1 amd64 NVIDIA binary OpenGL|ES 1.x library
ii libgles-nvidia2:amd64 525.89.02-1 amd64 NVIDIA binary OpenGL|ES 2.x library
ii libglx-nvidia0:amd64 525.89.02-1 amd64 NVIDIA binary GLX library
ii libnvidia-allocator1:amd64 525.89.02-1 amd64 NVIDIA allocator runtime library
ii libnvidia-cfg1:amd64 525.89.02-1 amd64 NVIDIA binary OpenGL/GLX configuration library
ii libnvidia-egl-gbm1:amd64 1.1.0-2 amd64 GBM EGL external platform library for NVIDIA
ii libnvidia-egl-wayland1:amd64 1:1.1.10-1 amd64 Wayland EGL External Platform library – shared library
ii libnvidia-eglcore:amd64 525.89.02-1 amd64 NVIDIA binary EGL core libraries
ii libnvidia-encode1:amd64 525.89.02-1 amd64 NVENC Video Encoding runtime library
ii libnvidia-glcore:amd64 525.89.02-1 amd64 NVIDIA binary OpenGL/GLX core libraries
ii libnvidia-glvkspirv:amd64 525.89.02-1 amd64 NVIDIA binary Vulkan Spir-V compiler library
ii libnvidia-ml1:amd64 525.89.02-1 amd64 NVIDIA Management Library (NVML) runtime library
ii libnvidia-ptxjitcompiler1:amd64 525.89.02-1 amd64 NVIDIA PTX JIT Compiler library
ii libnvidia-rtcore:amd64 525.89.02-1 amd64 NVIDIA binary Vulkan ray tracing (rtcore) library
ii nvidia-alternative 525.89.02-1 amd64 allows the selection of NVIDIA as GLX provider
ii nvidia-cuda-mps 470.161.03-2 amd64 NVIDIA CUDA Multi Process Service (MPS)
ii nvidia-driver 525.89.02-1 amd64 NVIDIA metapackage
ii nvidia-driver-bin 525.89.02-1 amd64 NVIDIA driver support binaries
ii nvidia-driver-libs:amd64 525.89.02-1 amd64 NVIDIA metapackage (OpenGL/GLX/EGL/GLES libraries)
ii nvidia-egl-common 525.89.02-1 amd64 NVIDIA binary EGL driver - common files
ii nvidia-egl-icd:amd64 525.89.02-1 amd64 NVIDIA EGL installable client driver (ICD)
ii nvidia-installer-cleanup 20220217+2 amd64 cleanup after driver installation with the nvidia-installer
ii nvidia-kernel-common 20220217+2 amd64 NVIDIA binary kernel module support files
ii nvidia-kernel-dkms 525.89.02-1 amd64 NVIDIA binary kernel module DKMS source
ii nvidia-kernel-support 525.89.02-1 amd64 NVIDIA binary kernel module support files
ii nvidia-legacy-check 525.89.02-1 amd64 check for NVIDIA GPUs requiring a legacy driver
ii nvidia-modprobe 525.78.01-1 amd64 utility to load NVIDIA kernel modules and create device nodes
ii nvidia-persistenced 525.85.05-1 amd64 daemon to maintain persistent software state in the NVIDIA driver
ii nvidia-settings 525.85.05-1 amd64 tool for configuring the NVIDIA graphics driver
ii nvidia-smi 525.89.02-1 amd64 NVIDIA System Management Interface
ii nvidia-support 20220217+2 amd64 NVIDIA binary graphics driver support files
ii nvidia-vdpau-driver:amd64 525.89.02-1 amd64 Video Decode and Presentation API for Unix - NVIDIA driver
ii nvidia-vulkan-common 525.89.02-1 amd64 NVIDIA Vulkan driver - common files
ii nvidia-vulkan-icd:amd64 525.89.02-1 amd64 NVIDIA Vulkan installable client driver (ICD)
ii xserver-xorg-video-nvidia 525.89.02-1 amd64 NVIDIA binary Xorg driver
media:/etc/apt/preferences.d#

What I do not know, is what from nvidia I can leave behind if only running plex. It certainly does not use all the components (vdpau, persistenced, egl, cuda mps, glx, opengl, vulkan, ptx, gsp-firmware). Surely the bits for window managers and rendering that is not required for simple encode/decode can be removed, which is the advantage vs. the mega blob binary download from nvidia, in addition to being built directly against the OS libraries.

If you have other packaged apps that are doing other things with your cuda cores, it is not going to be easy to evergreen with apt preferences and stay on Debian v11. You will probably have to go with the big binary solution download direct from nvidia and maybe consider application containers. It isn’t going to evergreen or maintain dependencies between nvidia and the host OS, but here are a few links related to that process, and its going to work, for a while:

https://www.reddit.com/r/PleX/comments/g1ss2q/nvidia_drivers_installation_in_debian/

Here is the kicker, after migrating to nvidia 525 cleanly, I am still seeing the same error :slight_smile:

media:~# dmesg | grep nvidia
[ 4.843454] nvidia: loading out-of-tree module taints kernel.
[ 4.845504] nvidia: module license ‘NVIDIA’ taints kernel.
[ 4.878816] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[ 4.901524] nvidia-nvlink: Nvlink Core is being initialized, major device number 249
[ 4.906888] nvidia 0000:00:05.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[ 5.888304] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 525.89.02 Wed Feb 1 23:09:40 UTC 2023
[ 5.974129] audit: type=1400 audit(1680276952.348:3): apparmor=“STATUS” operation=“profile_load” profile=“unconfined” name=“nvidia_modprobe” pid=378 comm=“apparmor_parser”
[ 5.976534] audit: type=1400 audit(1680276952.348:4): apparmor=“STATUS” operation=“profile_load” profile=“unconfined” name=“nvidia_modprobe//kmod” pid=378 comm=“apparmor_parser”
[ 7.091075] [drm] [nvidia-drm] [GPU ID 0x00000005] Loading driver
[ 7.091118] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:00:05.0 on minor 1
media:~# systemctl status plexmediaserver
● plexmediaserver.service - Plex Media Server
Loaded: loaded (/lib/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
Active: deactivating (stop-sigterm) (Result: signal) since Fri 2023-03-31 11:39:23 EDT; 43s ago
Process: 1378 ExecStartPre=/bin/sh -c /usr/bin/test -d “${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}” || /bin/mkdir -p “${PLEX_MEDIA_SERVER_AP>
Process: 1380 ExecStart=/bin/sh -c export PLEX_MEDIA_SERVER_INFO_VENDOR=”$(grep ^NAME= /etc/os-release | awk -F= “{print $2}” | tr -d " )";>
Main PID: 1380 (code=killed, signal=ABRT)
Tasks: 41 (limit: 19154)
Memory: 113.6M
CPU: 6.500s
CGroup: /system.slice/plexmediaserver.service
├─1405 Plex Plug-in [com.plexapp.system] /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/Framework.bundle/Contents/Resources/Ve>
├─1474 Plex Plug-in [com.plexapp.agents.none] /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/Framework.bundle/Contents/Resourc>
├─1476 Plex Plug-in [com.plexapp.agents.localmedia] /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/Framework.bundle/Contents/R>
└─1478 Plex Plug-in [com.plexapp.agents.lyricfind] /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/Framework.bundle/Contents/Re>

Mar 31 11:39:18 media systemd[1]: Starting Plex Media Server…
Mar 31 11:39:18 media systemd[1]: Started Plex Media Server.
Mar 31 11:39:23 media Plex Media Server[1380]: libc++abi: terminating with uncaught exception of type std::bad_cast: std::bad_cast
Mar 31 11:39:23 media Plex Media Server[1380]: ****** PLEX MEDIA SERVER CRASHED, CRASH REPORT WRITTEN: /var/lib/plexmediaserver/Library/Applicatio>
Mar 31 11:39:23 media systemd[1]: plexmediaserver.service: Main process exited, code=killed, status=6/ABRT
Mar 31 11:39:23 media Plex Media Server[1474]: Error starting framework core for /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/PersonalMed>
Mar 31 11:39:23 media Plex Media Server[1478]: Error starting framework core for /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/LyricFind.b>
Mar 31 11:39:23 media Plex Media Server[1476]: Error starting framework core for /usr/lib/plexmediaserver/Resources/Plug-ins-28fc46b27/LocalMedia.>
lines 1-23/23 (END)

So back to 1.30 for now, with my shiny upgraded nvidia core on Debian 11.

media:~# dpkg -i plexmediaserver_1.30.0.6486-629d58034_amd64.deb
dpkg: warning: downgrading plexmediaserver from 1.31.3.6868-28fc46b27 to 1.30.0.6486-629d58034
(Reading database … 95947 files and directories currently installed.)
Preparing to unpack plexmediaserver_1.30.0.6486-629d58034_amd64.deb …
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.30.0.6486-629d58034) over (1.31.3.6868-28fc46b27) …
Setting up plexmediaserver (1.30.0.6486-629d58034) …
PlexMediaServer install: PlexMediaServer-1.30.0.6486-629d58034 - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install: Installation Type: Update
PlexMediaServer install: Process Control: systemd
PlexMediaServer install: Plex User: plex
PlexMediaServer install: Plex Group: plex
PlexMediaServer install: Video Group: video
PlexMediaServer install: Metadata Dir: /var/lib/plexmediaserver/Library/Application Support
PlexMediaServer install: Temp Directory: /tmp
PlexMediaServer install: Lang Encoding: en_US.UTF-8
PlexMediaServer install: Processor: Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz
PlexMediaServer install: Intel i915 Hardware: Not found
PlexMediaServer install: Nvidia GPU card: Found
PlexMediaServer install:
PlexMediaServer install: Completing final configuration.
Failed to write ‘change’ to ‘/sys/devices/virtual/input/input2/uevent’: Cannot allocate memory
PlexMediaServer install: PlexMediaServer-1.30.0.6486-629d58034 - Installation successful. Errors: 0, Warnings: 0
Processing triggers for mailcap (3.69) …
media:~# systemctl stop plexmediaserver
media:~# systemctl start plexmediaserver
media:~# systemctl status plexmediaserver
● plexmediaserver.service - Plex Media Server
Loaded: loaded (/lib/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2023-03-31 11:43:05 EDT; 1min 43s ago
Process: 2085 ExecStartPre=/bin/sh -c /usr/bin/test -d “${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}” || /bin/mkdir -p "${PLEX_MEDIA_SERVER_AP>
Main PID: 2087 (Plex Media Serv)
Tasks: 139 (limit: 19154)
Memory: 664.9M
CPU: 32.355s
CGroup: /system.slice/plexmediaserver.service
├─2087 /usr/lib/plexmediaserver/Plex Media Server
├─2120 Plex Plug-in [com.plexapp.system] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Contents/Resources/Ve>
├─2156 /usr/lib/plexmediaserver/Plex Tuner Service /usr/lib/plexmediaserver/Resources/Tuner/Private /usr/lib/plexmediaserver/Resource>
├─2188 Plex Plug-in [com.plexapp.agents.themoviedb] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Contents/R>
├─2189 Plex Plug-in [com.plexapp.agents.plexthememusic] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Conten>
├─2191 Plex Plug-in [tv.plex.agents.series] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Contents/Resources>
├─2272 Plex Plug-in [com.plexapp.agents.imdb] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Contents/Resourc>
├─2326 Plex Plug-in [com.plexapp.agents.localmedia] /usr/lib/plexmediaserver/Resources/Plug-ins-629d58034/Framework.bundle/Contents/R>
├─2378 Plex EAE Service
└─2379 /usr/lib/plexmediaserver/Plex Transcoder -codec:0 hevc -hwaccel:0 nvdec -hwaccel_fallback_threshold:0 10 -threads:0 1 -hwaccel>

Mar 31 11:43:05 media systemd[1]: Starting Plex Media Server…
Mar 31 11:43:05 media systemd[1]: Started Plex Media Server.
Mar 31 11:43:36 media [2156]: Critical: libusb_init failed
Mar 31 11:43:36 media Plex Media Server[2156]: Critical: libusb_init failed
Mar 31 11:44:47 media Plex Media Server[2378]: Dolby, Dolby Digital, Dolby Digital Plus, Dolby TrueHD and the double D symbol are trademarks of Do>
lines 1-25/25 (END)