Latest Version 1.31.1.6733 crashes my Thecus NAS N5810

Plex Media Server NAS & Devices
1

Server Version#:1.31.1.6733
Player Version#: Various

I’ve been running, somewhat to my surprise, the latest versions of Plex Media server on my Thecus NAS N5810, for quite some time now. Thecus has all but disappeared from the world, so there really isn’t any support or updates to be had. So I was rather shocked when the latest offerings from Plex for the Thecus actually worked!

For reference, my 5810 is running OS7 firmware version 3.02.08.10.

So, from Plex 1.25.8.5663 up until the 2nd most recent 1.31.0.6654, I haven’t had any issues. Opened up my Plex desktop client today, and I’m informed there’s a server update ready to download and install. I go ahead and do it, as I’ve done on the previous occasions.

Except this time, after the install of version 1.31.1.6733, my NAS started crashing. I don’t just mean Plex freezes up, the whole NAS locks up. Even the LCD on the front which rotates through the NAS settings, health, time date, connections, etc is frozen. I’ve never had that happen before. The only way out of it is to force-power it down, which is not great for a NAS with a RAID-5 array.

Anyway, I know it’s the Plex Server module, because I can deactivate it by logging in while the NAS is still finishing its boot process. Once deactivated, the NAS runs fine, no issues, same as it always was. As soon as I activate the Plex Server Module again, within a few minutes, BAM, NAS crash.

I tried rolling back to the previous version, but even after that was installed and running, the crash issue kept happening. I had a look at Plex Media Server downloads page, and there was actually an even newer version there (1.31.2.6739). Which struck me as a bit odd, since my Plex only just informed me about 1.31.1.6733. But, I thought, maybe it’s a known issue, and this is the fix. So I installed that one. No go, same problem.

I figure that I could probably fix this by uninstalling Plex completely, and then re-installing the older version. But, I’m likely to lose all my libraries, settings, collections, etc. It’s a fairly hefty collection, and it took a looooong time to fix and curate. I’d rather not go through the pain of that again.

If anyone can offer some help, I’d be very appreciative. As mentioned earlier, Thecus themselves have been MIA for a number of years now, so I’m unlikely to get help from that quarter. In any case, normal NAS functions are unaffected. It’s only Plex that has the issue.

For reference, I use the Plex client on various devices, namely my PC, an Android Tablet, and an XBox One (which doesn’t really count, most of my collection won’t play on the XBox until I transcode all the files). The version on my PC (which updated at the same time I updated the server version) is Version 1.65.1.3596. On my Android I’m running 9.14.0.37895. I don’t think the client versions really matter for this case, because the NAS crashes without any clients running. It only doesn’t crash if I deactivate Plex.

Here’s hoping someone has a simple fix.

Cheers,

d33j.

2

It could be that the NAS is getting overwhelmed (it does have an underpowered CPU) trying to perform analysis for credit detection, a feature that was currently released. So, you might try disabling that feature, if you haven’t done so already. You can find information on how to turn it off in the support article.

Otherwise, for the team to investigate the issue, you’d need to share Plex Media Server log files, grabbed after you encounter the issue, as an attachment in a new reply here.

3

Unless something fundamentally changed between Credit Detection in version 1.31.0.6654 to 1.31.1.6733, then I don’t see how, it was working already. The NAS IS underpowered, but that just means that it can’t do transcoding on the fly, and if I want to watch on other devices with different requirements, I have to do that myself. Also, if that’s all it was, then rolling it back should have fixed it… I think. I’ll give it a go.
And yeah, since initially posting this, I’ve backed up the Plex folder, and gone through the logs. There’s a lot of them, like nearly 180 of them, all generated within 3 seconds of each other. Most are tiny, and don’t contain much. The there’s about a dozen that are about 10MB in size and have anywhere from between 65,000 to 111,500 lines of log entries. Fortunately, I think I can exclude most of those, because they have names like “Scanner Credits” or “Chapter Thumbnails”. The interesting ones are just called “Plex Media Server”, and all the big ones (which are essentially all 111,000+ lines), just have this for the first 5 lines:

`Mar 04, 2023 20:12:03.465 [0x7f61fe8d0b38] INFO - Plex Media Server v1.31.2.6739-a87e876bd - Thecus N5810 x86_64 - build: linux-x86_64 thecus - GMT 08:00`

`Mar 04, 2023 20:12:03.466 [0x7f61fe8d0b38] INFO - Linux version: 3.02.08.10, language: en-US`

`Mar 04, 2023 20:12:03.466 [0x7f61fe8d0b38] INFO - Processor: 4-core Intel(R) Celeron(R) CPU J1900 @ 1.99GHz`

`Mar 04, 2023 20:12:03.466 [0x7f61fe8d0b38] INFO - Compiler is - Clang 11.0.1 (https://plex.tv 9b997da8e5b47bdb4a9425b3a3b290be393b4b1f)`

`Mar 04, 2023 20:12:03.466 [0x7f61fe8d0b38] INFO - /raid/data/module/PlexMediaServer/sys/Plex Media Server`

and then this for the remaing 110,995+ lines:

Mar 04, 2023 20:11:43.674 [0x7f61fc6b1b38] ERROR - Mp4AtomParser: Invalid atom header size: 0

Now, I have no idea what this means. It LOOKS like a memory address, followed by an error (note the memory register is also present in the preceding 5 lines as well). But that error is Greek to me. I’ve tried searching for it, but found very little, and nothing helpful. If that jogs yours, or anyone else’s memory, I’d love to hear about it. If that IS specifically something to do with Credit Detection, that would be very useful to know. The only thing that I can say that maybe speaks to it, is the process for going through all my files to do the credit detection was taking a long time (I’m not even certain if it finished). Maybe, MAYBE, it just ran out of resources, and is now stuck in a crash loop. Actually, now that I think about it… that kinda fits pretty much everything that happened. The update may either have just been badly timed and therefore a false positive, or exacerbated an issue that was already in the process of occuring. Ok… Imma investigate. Cheers chrisc

4

The credit detection feature was not introduced specifically with a Plex Media Server release. The functionality was present in some already-released versions and then we “flipped the switch” on our side to make it available to users. So, it’s certainly plausible based on the described behavior here that you started seeing issues when the feature was activated recently. And “rolling back” to a previous server release might not change anything, since that v1.31.0 release also included the functionality.

Again, we recommend disabling that feature and seeing if you no longer experience the same behavior.

As for the server logs, if you still have issues after disabling the credit detection feature, you would need to zip up all of the Plex Media Server.log—and the numbered Plex Media Server.X.log—files, then attach/share the zip here for the team to be able to investigate.

(Make sure that you enable the Settings > Server > General > Enable Plex Media Server debug logging option in the server settings to allow “debug” logging before reproducing the behavior and then grabbing the logs.)

5

An MP4 atom with a size of zero is an invalid atom and should treated as an error. It looks like their parser tries to ignore it and move on by going to the “end” of that atom and trying to parse further, but if the length is zero they just end up at the same file position and keep reading the same 0 length atom over and over in a tight loop until the log fills up and the server crashes.

Please just check your MP4 atom parsing code to verify that it stops trying to parse any file once it finds an atom with a length less than 8. That should solve the problem.

6

Note that I ran into the same problem with one of my videos. Zipping up the log files won’t help because all you see is the error. The log information that contains the name of the file that was being parsed or what atoms were successfully parsed is never saved to disk because the rest of the output happens so fast that it swamps the server until the NAS kills the process. You have to watch the PMS console and as soon as you see the stream of errors, quickly scroll back to see what file it was parsing before both the browser session and the PMS server crash. Doing this, I was able to temporarily rename 2 of my files (old files from when MP4 was new) so that it wouldn’t try to parse them again. After doing this twice and renaming those 2 files, I was able to have the server stay up.

This happened after upgrading from 1.29.2.something to the latest 1.32 for me. It continue to happen after I rolled back to 1.29.2.6364 because I think the cached info for the file was corrupted and it was trying to recompute it.

7

An MP4 atom with a size of zero is an invalid atom and should treated as an error. It looks like their parser tries to ignore it and move on by going to the “end” of that atom and trying to parse further, but if the length is zero they just end up at the same file position and keep reading the same 0 length atom over and over in a tight loop until the log fills up and the server crashes.

Please just check your MP4 atom parsing code to verify that it stops trying to parse any file once it finds an atom with a length less than 8. That should solve the problem.

I’m actually not certain how to do that :face_with_diagonal_mouth:. Can you give me some tips? Are there some commands in terminal I can enter to determine if that’s the case, and if so, how to implement that fix?

I should also point out that, when the server crashes, it’s because it’s essentially run out of memory. Prior to the Plex update that initiated all of this, Plex ran quite smoothly.

As of the update, I notice that, once the plex server is started on the NAS, the amount of memory drops off dramatically, seems to hold stable for a short while, but then ultimately fills up, and the server stops responding, where only a cold shutdown and start will fix it. If Plex isn’t started, the memory seems to hold steady.

The other thing of note is that, at about the time this started (although I’m not certain which came first, as I had kind of forgotten about it), is that the NAS was displaying a message on the front LCD saying “HBT LOG FULL!!!”, which may also be in some way responsible for the memory drop-off. But the message disappeared some time ago, and I don’t remember if I resolved that, or just dismissed it, or if I DID fix it, what action I performed!

I DO remember trying to get in touch with Thecus for help, but they told me they couldn’t even TELL me anything unless I paid them! I also don’t remember if it was the same NAS (I have 2, the N5810 is the Plex server, the other NAS is an N5500, which doesn’t support any version of Plex). Anyway…

I don’t know if any of that helps or is of relevance. I have since updated the Plex server to the latest version (1.32.2.7100), with no improvement. Then, in desperation uninstalled it, and reinstalled it. It seemed to stay up a bit longer after that, but then inevitably the same thing happened; the memory ran out, and the device crashed.

That’s where we are as of 23/05/2023. Sorry for the delay in returning to this for my part, but I’m juggling this with Uni, work, parenting, etc. Hoping the interested and commenting parties haven’t lost interest by my absence. Remaining hopeful and looking forward to hearing from someone… :grin:

D.

8

Addendum to the above: I went through my files and essentially removed any new ones that I had put into the library on or around the date of the initial failure (was actually a bit brutal, went above and beyond, but I wanted to be sure).

After removing them and restarting Plex, it looked like it was going to be fine for a while, there was a smallish dip in memory (to be expected), and it carried on ok for somewhere between 5 to 10 minutes, before the memory suddenly all got swallowed up and it crashed again!

I’m doing another sweep now to be sure I didn’t miss anything, but I’m fairly certain I got it all. But I notice you mentioned something else:

Having removed what I believe to be the offending files, is your recommendation to uninstall Plex completely, and then do a fresh install, which would then possibly remove the cached “bad” info? And without the offending file(s), the error should be corrected?

I did go through the logs, and came across this:

Mar 19, 2023 23:10:43.697 [0x7f5a5bf2db40] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.39.4, threadsafe=1
Mar 19, 2023 23:10:43.935 [0x7f5a5bf2db40] DEBUG - Analyzing media parts for item 168015 (Episode 3): 250443
Mar 19, 2023 23:10:43.949 [0x7f5a5bf2db40] DEBUG - [ID 267228] Media part analysis: /raid/data/Videos/TV Shows/M/Murder She Wrote (1984)/Murder, She Wrote 1984-1996 (Complete TV series in MP4 format)[114.82GB]/Murder, She Wrote - Movies/MSW - 2003 Movie (The Celtic Riddle).mp4
Mar 19, 2023 23:10:44.003 [0x7f5a5bf2db40] DEBUG - [MI] Opening input file: "/raid/data/Videos/TV Shows/M/Murder She Wrote (1984)/Murder, She Wrote 1984-1996 (Complete TV series in MP4 format)[114.82GB]/Murder, She Wrote - Movies/MSW - 2003 Movie (The Celtic Riddle).mp4"
Mar 19, 2023 23:10:44.003 [0x7f5a5bf2db40] DEBUG - [FFMPEG] - Opening '/raid/data/Videos/TV Shows/M/Murder She Wrote (1984)/Murder, She Wrote 1984-1996 (Complete TV series in MP4 format)[114.82GB]/Murder, She Wrote - Movies/MSW - 2003 Movie (The Celtic Riddle).mp4' for reading
Mar 19, 2023 23:10:44.003 [0x7f5a5bf2db40] DEBUG - [FFMPEG] - Setting default whitelist 'file,crypto,data'
Mar 19, 2023 23:10:44.004 [0x7f5a5bf2db40] DEBUG - [FFMPEG] - Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
Mar 19, 2023 23:10:44.004 [0x7f5a5bf2db40] DEBUG - [FFMPEG] - ISO: File Type Major Brand: isom
Mar 19, 2023 23:10:44.004 [0x7f5a5bf2db40] DEBUG - [FFMPEG] - Unknown dref type 0x206c7275 size 12
Mar 19, 2023 23:10:44.014 [0x7f5a5bf2db40] WARN - [FFMPEG] - STSC entry 87814 is invalid (first=0 count=0 id=0)
Mar 19, 2023 23:10:44.014 [0x7f5a5bf2db40] WARN - [FFMPEG] - STSC entry 87813 is invalid (first=0 count=0 id=0)

… and then those warnings continue on ad infinitum. There were also another few files in the same series which generated the same error. The issue didn’t clear after I removed them, which is why I went through and removed all the others mentioned above. But, if a reinstall is called for, maybe I had already removed the offending files?

D.

9

Ok, so, I MAY be jumping the gun, but I think it’s fixed! The Plex service has been up for close to 40 minutes now, and the memory on the NAS is holding steady, and using about a quarter of the total (which is a happy accident… I’ll get into that in a second), and I’m even playing media while typing this. So yay!

And, if it turns out that it IS fixed then absolute kudos to you flarbear, you hit the nail on the head! I finally worked out what and where you were talking about to sort out what file was doing what. Viewing the logs in the console, it hit that ERROR - Mp4AtomParser: Invalid atom header size: 0 directly after trying to parse a REALLY old mp4 file, that wasn’t even located in the main Videos section (it was in the Pictures folder, of all places). I deleted it, restarted Plex, and thus far, everything has been working solidly!

I don’t know if this had anything to do with the solution as well, but I also rolled back to 1.29.2.6364 ; just so happened to be the earliest version I have saved in my Downloads folder, so purely coincidentally that was the one I used. If I went into my archives I have earlier versions than that.

Anyway, it was one of the attempts I was using to try and fix the problem, prior to observing the console and determining which file was the cause. At that time it obviously didn’t do anything, Plex still kept getting caught in the mp4-atom parse error loop, draining the memory from the NAS, and crashing. But, after I removed the offending file, things did start working again. I’m not certain if the rollback to that version was necessary or helped, or if any rollback was required at all, just that it was one of the things that I did prior to correcting the problem.

As to why it seems to be running on a lot less memory; this is kind of weird. I was watching the console entries scroll past, when the error popped up. It was so fast I didn’t even have time to see what file was causing. The memory usage on the NAS monitor shot up. In a panic, so as to not have to restart the NAS again, I quickly deactivated Plex. The memory returned to a baseline less than it had been using prior to Plex starting previously! I did the same thing again unintentionally before deciding to use a screen recorder to capture the file details. It went to even less memory used than the previous time. It settled at about a quarter usage after I did it intentionally a couple more times. After I worked out which file was the culprit and removed it, Plex sat happily at that level.

Anyway, I’m going to keep an eye on it, but all things being equal, this thread might be able to be closed. I’ll report back if something goes wrong, and let you know in a few days if it’s all gone right!

Cheers,

D.

10

Just tying this one off; the Plex server has now been up for over 24 hours without issue. flarbear’s solution was bang on the money for this one, thank you very much good sir :blush::grin::blush:. And thank you kindly chrisc for your input, you helped start me looking in the right places to find the problem. Very much obliged to both of you.

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.