Synology DS216+II
DSM 6.2.3-25426 Update 3
I had crashes on 1.21.1.3876, updated to 1.22.3.4392, and had another crash, so I decided to post the logs.
Crashes happen once in a few days, sometimes at night, sometimes during the day. I created Tautulli “Plex Down” notification to know when it happens.
I do not have on/off schedule on my DSM.
Plex Scheduled Tasks are set to 3pm and 6pm.
Tautulli notified me about Plex stop at 3:20 AM (on 1.22.3). No one used Plex at the time, so it must be a result from some background job. Also there’s nothing in the history of Sonarr/Radar during that time, so no new content was added.
Before the crash I had “Enable Plex Media Server debug logging” checked, and verbose set to off.
Manual archive of the folder before the Plex start: Logs 240421.zip (9.6 MB)
Just in case, now I disabled “Send crash reports to Plex” so I’d have a DMP file instead in the future. For now I’m waiting for another crash to post it.
It happened again today (25.04) around 18:58. Few hours before the crash I noticed DSM was using CPU (indicator in the red, with the Plex task on top of sorted tasks). ~4 hours after the crash I went to start Plex back, but noticed the CPU was still being used, went to see what’s using it - and it was Plex, even though its package was stopped as of few hours. It’s not supposed to be that way, right?
After that crash I started the PMS, and after my message here I opened iOS client (latest beta), and PMS crashed again (at 22:36) while the client was loading the content. I restarted Synology after that, Plex launched itself and I was able to watch from iOS client no issue. Added DMP for the new crash:
I think it crashed much less frequently on 1.21.1.3876 (and wasn’t crashing at all on some earlier version), now after the update, it became daily. But certainly, DSM is a part of it too. I know I’m not on its latest version, but I can’t update yet because some breaking API changes were introduced and I have to wait when it’s fixed in every community software that uses it.
This test sets up a new, temporary, “Test” server while keeping your existing server safe. It will help us figure out if the problem is somewhere in the PMS database or in DSM itself.
Here’s what I’d like you to do
Make certain latest PMS is installed
Stop PMS
Sign out of the Plex/web app (upper right corner), close the Plex/Web tab.
Open the Plex shared folder in FileStation.
Rename Library to Library.save (we’re going to keep what exists safe)
Thanks, I think I will be able to try it around the weekend. Does it matter if the libraries I create use the exact same agent etc? Is it better to write down my server and each library settings to recreate exactly, or it’s better to create a mostly default config?
Also, is there any use in uploading new dumps and logs from the current server, until I will be able to conduct the experiment?
It’s perfectly OK (actually desired) to recreate this test server as you would normally create it. It’s important to create a controlled test case while keeping track of everything done as it’s done. We need to figure out how we arrive at the failure point
I had an idea to check /var/log for times when dumps were created. I found out that each crash has a corresponding line in the kern.log:
2021-04-25T18:57:02+03:00 DiskStation kernel: [688897.142755] Plex Media Serv[12423]: segfault at 0 ip 000000000191b40c sp 00007f39025a02d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-25T22:36:01+03:00 DiskStation kernel: [702036.101081] Plex Media Serv[11689]: segfault at 0 ip 000000000191b40c sp 00007f75276c02d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-26T16:18:30+03:00 DiskStation kernel: [63514.354013] Plex Media Serv[27817]: segfault at 0 ip 000000000191b40c sp 00007f37177342d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-28T15:35:03+03:00 DiskStation kernel: [233705.269365] Plex Media Serv[31643]: segfault at 0 ip 000000000191b40c sp 00007f3187a222d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-28T23:20:09+03:00 DiskStation kernel: [261610.109185] Plex Media Serv[21697]: segfault at 0 ip 000000000191b40c sp 00007f1b774462d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-29T05:00:26+03:00 DiskStation kernel: [282027.021927] Plex Media Serv[8842]: segfault at 0 ip 000000000191b40c sp 00007f8fa3d102d0 error 4 in Plex Media Server[a30000+f5b000]
2021-04-29T16:21:37+03:00 DiskStation kernel: [322897.805329] Plex Media Serv[26050]: segfault at 0 ip 000000000191b40c sp 00007f72794c32d0 error 4 in Plex Media Server[a30000+f5b000]
With a faulting address of 0 and an error code of 4, we know this particular segfault is a read of a null pointer.
Do you have more info on that? Could you relay that to the engineering team if possible?
I’ve never experienced segfault in Linux before, so I’m not sure what to do with it. Is the issue specific to some address in the Plex code, or should I also contact Synology support? Maybe I should do a memtest?
Didn’t have the time to set it up yet, so it’s all past crashes on the regular server, including latest one. Does segfault specifically imply if the problem is with Plex or the host? Is the experiment still required to figure this thing out?
Since I have 10+ libraries and it could be up to 1+ day without crashes, by my calculations the experiment would take me at least a couple of weeks. So before starting it, I decided to try IFYates advice from here just in case. And it probably worked, at least I had no crashes for 5 full days (before it was almost every day).
The only place I could find info about the proper reinstall steps is here:
Since it’s titled as “FAQ 8: Uninstalling PMS - Reverting to a previous version” and I didn’t need to downgrade, I didn’t read this before. And I’d guess a lot of users also don’t try to reinstall it that way cause they’re too afraid of the “Uninstall” action, thinking it will remove their data or settings.
I think it would be useful to create a separate FAQ entry for “How to update/reinstall”, or add it as a troubleshooting step. But, in case similar issues may happen with the future versions, I think the best place to show a clean update routine on Synology would be here: https://www.plex.tv/media-server-downloads/#plex-media-server
You can rest assured, I take every precaution to safeguard users’ data on Synology
When you eventually migrate to DSM 7, you’ll see how much I do in the new packaging.
I keep everything in different areas just as it is in DSM 6.
The new menu mechanism gives the opportunity to confirm as well as ‘back up & change’ before making the final ‘commit’ action.
