Looks like it’s happened again today against a new replacement file fetched after a purge of all DV/HDR10 content in an attempt to prevent Plex entering this hang state every day.
The last log entry from was from 8:19am this morning (It is now 8:07PM, I killed it at 8:05PM), it has been hung for nearly 12 hours and I received some complaints earlier today before I could get to things.
The last log line is: Sep 07 08:19:58 hostnameHere plexmediaserver[2170498]: Generated new chapter thumbnails for Thor.
One of the two files which match the 2011 title has
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
I’m at a point where I’m gonna have to write some check script to keep an eye on things. The platform continuously kills itself and refuses to die as a process instead of letting my family watch things.
This is made worse by automated content procurement methods having no idea the inbound content is HDR10 before syncing or otherwise.
Before killing plexmediaserver tonight I tried to figure out if it could be saved without a SIGKILL.
/usr/lib/plexmediaserver/Plex Transcoder had an entirely irrelevant movie file open and was in a SLEEP state. The movie file which it had open as a file descriptor was only AVC / High@L5 and presumably unrelated to these crashes. Attaching strace to it showed this shortly before being detached.
[pid 2215987] futex(0x7fb8acc70a34, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 2215986] futex(0x7fb8acc93a34, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 2215985] futex(0x7fb8ad044a34, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 2215984] futex(0x7fb8ad074a34, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 2215983] futex(0x7fb8ad097a34, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 2214744] restart_syscall(<... resuming interrupted read ...>) = 0
[pid 2214744] poll([{fd=3, events=POLLIN}], 1, 100) = 0 (Timeout)
[pid 2214744] poll([{fd=3, events=POLLIN}], 1, 100) = 0 (Timeout)
[pid 2214744] poll([{fd=3, events=POLLIN}], 1, 100) = 0 (Timeout)
/usr/lib/plexmediaserver/Plex Tuner Service kept respawning as I killed it but the process was happy to die and respawn shortly after without a SIGKILL, so I doubt it was relevant to today’s hang.
Plex EAE Service was also in a sleep state but had 226minutes 54s of cpu time clocked and had a start date of Sep06 (Some timestamp the previous day, at least 8 hours of existing). Attaching strace to it only showed epoll_wait(5, and this process would not stop without a SIGKILL.
Next up was Plex Plug-in which had a date of the previous day with only 2m:31s of cpu time and was also sleeping. Attaching strace showed lines like the below and this one also required a SIGKILL.
[pid 2167763] futex(0x7f0048e7f480, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2167762] futex(0x7f0048e7ed00, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165989] futex(0x7f004968e2f0, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165988] futex(0x7f004968e260, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165987] futex(0x7f004968e080, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165986] futex(0x7f004968d2d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165985] futex(0x7f004968d330, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165984] futex(0x7f004968d1b0, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165983] futex(0x7f004968d060, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2165982] futex(0x7f004968be60, FUTEX_WAIT_PRIVATE, 4294967295, NULL <unfinished ...>
[pid 2164991] read(0, <unfinished ...>
[pid 2167772] madvise(0x7f00487f3000, 8192, MADV_FREE) = 0
[pid 2167772] getpid() = 2164991
[pid 2167772] epoll_pwait(7, [], 1023, 200, NULL, 8) = 0
[pid 2167772] madvise(0x7f00487e6000, 12288, MADV_FREE) = 0
[pid 2167772] epoll_pwait(7, [], 1023, 200, NULL, 8) = 0
[pid 2167772] madvise(0x7f00487f0000, 8192, MADV_FREE) = 0
[pid 2167772] epoll_pwait(7, [], 1023, 200, NULL, 8) = 0
[pid 2167772] madvise(0x7f00487f3000, 8192, MADV_FREE) = 0
The remaining processes at this point were Plex Tuner Service (Which kept reviving after being killed gracefully.) and the main process /usr/lib/plexmediaserver/Plex Media Server which required a SIGKILL as usual to restart and resume serving content.
I’ve uploaded the last few minutes of '/var/lib/plex/Plex Media Server/Logs/Plex Media Server.log' to this comment for more info. The log timestamps Sep 07, 2023 08:20:00.943 and Sep 07, 2023 08:20:01.219 were for Thor which lines up within a minute or so of the service hang.
plexmediaserver.log (36.8 KB)
