Server Version#: 1.23.4.4712
Player Version#: 1.33.0.2444-a220eae4
I’ve been having issues with my Plex server on Truenas for a while. Truenas seems to crash with the same error periodically - mostly over night. Has anyone got an idea what might be causing the crashes?
Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address = 0x3454f8f9
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff829ec2ce
stack pointer = 0x28:0xfffffe014391c758
frame pointer = 0x28:0xfffffe014391c790
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 6252 (Plex Media Server)
trap number = 12
panic: page fault
cpuid = 2
time = 1625363857
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe014391c410
vpanic() at vpanic+0x17b/frame 0xfffffe014391c460
panic() at panic+0x43/frame 0xfffffe014391c4c0
trap_fatal() at trap_fatal+0x391/frame 0xfffffe014391c520
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe014391c570
trap() at trap+0x286/frame 0xfffffe014391c680
calltrap() at calltrap+0x8/frame 0xfffffe014391c680
--- trap 0xc, rip = 0xffffffff829ec2ce, rsp = 0xfffffe014391c758, rbp = 0xfffffe014391c790 ---
avl_rotation() at avl_rotation+0x3e/frame 0xfffffe014391c790
zfs_rangelock_enter_impl() at zfs_rangelock_enter_impl+0x4e8/frame 0xfffffe014391c7f0
zfs_get_data() at zfs_get_data+0x15f/frame 0xfffffe014391c880
zil_commit_impl() at zil_commit_impl+0xe11/frame 0xfffffe014391c9e0
zfs_fsync() at zfs_fsync+0xc1/frame 0xfffffe014391ca20
VOP_FSYNC_APV() at VOP_FSYNC_APV+0x7b/frame 0xfffffe014391ca50
kern_fsync() at kern_fsync+0x191/frame 0xfffffe014391cac0
amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe014391cbf0
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe014391cbf0
--- syscall (95, FreeBSD ELF64, sys_fsync), rip = 0x8054699fa, rsp = 0x7fffdabd0aa8, rbp = 0x7fffdabd0ac0 ---
KDB: enter: panic
panic.txt0600001214070212621 7124 ustarrootwheelpage faultversion.txt0600006214070212621 7524 ustarrootwheelFreeBSD 12.2-RELEASE-p6 facd5d71c97(HEAD) TRUENAS
It seems to always happen at about same time - somewhere around 3.30 AM. I guess it’s related to some of the scheduled tasks that are scheduled between 2.00 and 5.00 AM, however I couldn’t figure it out yet.
The times it happened to me was 5:40am yesterday and at both 4:10am then 5:50am today. So seems to be at the timeframe you mentioned. I think that’s when Plex does its maintenance tasks.
I’ve got 24Gb of memory on my server. I have run the memtest86 on my server and it passed all the tests without any errors.
It is strange that whenever crash occurs the process in question is always plex media server hence the issue has been posted here.
I got another one of these crashes this morning at 6:30 a.m. after not having a crash for the last 12 days.
My server has 32 gigs of ECC ram that I tested for 4 days straight before I installed the OS on the server so I doubt there’s a fault with the memory.
The common thread here as stated is every kernel panic log points to Plex Media Server and seems to be doing the maintenance tasks as well.
Here is a link to someone posting the same issue on the TrueNAS forms but it does not look like they solved the issue either.
I don’t know much about TrueNAS or jails, but kernel issues are not typically caused by an app. These are usually OS related. As mentioned in that thread you linked, this can happen if you have a corrupted drive or bad media and Plex is trying to scan these files. Do you have the PMS logs from the last crash? If the scanner was running, it may show what file it was trying to access when it crashed.
This is the only logs from Plex right before the crash. This is from the file called “Plex Media Server.log”. All other log files did not have entries around the time of the crash. The crash happened at 06:34.
I have my doubts that it’s a corrupted media file as it crashed two days in a row and then there were no crashes for 12 days before it randomly crashed again. There are also no logs from the media scanner at the time.
Jul 20, 2021 06:31:09.506 [0x8086b9500] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:31:16.713 [0x808218e00] WARN - No matches for search.
Jul 20, 2021 06:31:18.232 [0x808218e00] WARN - No matches for search.
Jul 20, 2021 06:31:21.214 [0x808219d00] ERROR - Couldn’t copy file “/config/Plex Media Server/Metadata/Albums/a/074b79ae7e69958973224558b47c002e305fbcd.bundle/Contents/_stored/posters/com.plexapp.agents.localmedia_b8be5b0b17e60733f185086705460a99883ccb96” to “/config/Plex Media Server/Metadata/Albums/0/4728176eff9d655a1a6a76820c571ee34d65a12.bundle/Uploads/posters/com.plexapp.agents.localmedia_b8be5b0b17e60733f185086705460a99883ccb96”: File exists
Jul 20, 2021 06:31:22.436 [0x808218e00] WARN - Held transaction for too long (…/Library/MetadataItem.cpp:1093): 0.406250 seconds
Jul 20, 2021 06:31:27.324 [0x808219d00] WARN - SLOW QUERY: It took 257.812500 ms to retrieve 13 items.
Jul 20, 2021 06:31:48.012 [0x8086b9500] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:31:51.646 [0x80839c600] WARN - No matches for search.
Jul 20, 2021 06:32:04.025 [0x80e402500] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:32:17.566 [0x80e402500] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:32:29.326 [0x808219d00] WARN - No matches for search.
Jul 20, 2021 06:32:30.855 [0x80821a700] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:32:33.862 [0x808219d00] WARN - No matches for search.
Jul 20, 2021 06:32:34.802 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:32:39.173 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:32:42.931 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:32:43.476 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:32:45.965 [0x8086b9500] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:32:49.040 [0x80dbc6f00] WARN - Held transaction for too long (…/Library/MetadataItem.cpp:7141): 2.460938 seconds
Jul 20, 2021 06:32:49.628 [0x80d4b5e00] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:32:55.925 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:33:14.780 [0x80d096000] WARN - NAT: PMP, timed out waiting for response.
Jul 20, 2021 06:33:23.102 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:33:24.146 [0x806ba3000] WARN - Request for plug-in [com.plexapp.agents.plexmusic] but it doesn’t exist.
Jul 20, 2021 06:33:25.119 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:33:36.032 [0x808398500] WARN - No matches for search.
Jul 20, 2021 06:33:44.678 [0x808398500] WARN - No matches for search.
Doesn’t look like you have debug logging enabled for your server. There isn’t enough information without that. Please enable that and after the next crash grab the entire log package (i.e. zip file) and post it here.
I have turned debug logging on now, just waiting for the next crash.
To try an mitigate the issue I have moved my scheduled tasks to 10am-2pm and have Plex process shut down between the hours of 3am-7am as that is when the crashes were occurring.