As follow up to my previous question,
I would like to see the resource status as shown by the Synology Resource app when PMS is running.
Of primary interest is Memory and swap. This is all visible from the “Overview” -> “Real Time”
You’ll also want to correct these filesystem permissions so Plex can read the files (use FileStation to gr ant Read). If permissions already allow reading, then I know what’s happening. You’ve exceeded the kernel’s default iNotify table size of 8192 directories. Do you have more than 8192 directories?
Sep 22, 2018 14:31:29.779 [0x7f40e311d0] ERROR - [Notify] Failed to add watch for /volume1/video/Anderes (13: Permission denied)
Sep 22, 2018 14:31:29.778 [0x7f8aab81d0] DEBUG - [Notify] Waiting for inotify notifications...
.
.
.
Sep 22, 2018 14:31:31.006 [0x7f40e311d0] ERROR - [Notify] Failed to add watch for /volume1/video/Serien (13: Permission denied)
.
.
.
Sep 22, 2018 14:31:31.624 [0x7f40e311d0] ERROR - [Notify] Failed to add watch for /volume1/video/Serien/Hawaii Five-0 (13: Permission denied)
Sep 22, 2018 14:31:31.624 [0x7f40e311d0] ERROR - [Notify] Failed to add watch for /volume1/video/Serien/Hawaii Five-0/Season 3 (13: Permission denied)
I suggest you go to the top-level directory, right click it,
Properties -> Permissions tab -> Apply to this folder, sub-folders, and files.
I am attempting to replicate your experience but cannot at this time.
Please respond at your earliest so I may continue.
Just out of curiosity, what causes the Synology ARM v8 build to be ~2.75 times bigger than the old builds?
As I already noted I’m on vacation with really bad internet so I can’t really do any tests.
I have nothing but VPN and SABnznd running on the DS118, the restart issues disappear when stopping PMS. I also have Sonarr/Radarr/Lidarr running on a Rpi with SABnzbd as a remote download client.
Regarding the filesystem problems Plex at installation has notified me because of this. Then I did what you with the permissions and with applying it to subfolders.
As the binaries are being built, after linking, the symbols are removed. There isn’t any need to carry this around in the executable because you would need a full SDK environment, with the source, to debug it.
When they fixed a build issue for one OS, stripping of symbols got broken.
The problem where they weren’t being stripped has been fixed for all the systems now.
I am expecting 
we will have an updated build tomorrow but won’t know until that team is back in the office (different time zone from me)
1 Like
Thanks Chuck! So, I’m not a dev, but out of curiosity, what all has changed in this new branch of builds compared to the previous versions (if you’re allowed to share)? Is it a huge list of things or a couple high impact items or both?
Just interested to understand a bit of the inner workings. My brother is a software engineer, so I am familiar with a very small part of the concepts and the terminology.
Installed on Debian, not seeing any issues so far. Any plans to update this to the new 1.13.8 builds? Also, could we get any more info on the actual performance/framework changes that were made and where you guys are looking to see better performance? IE should I be noticing that scrolling through libraries are faster? Or lower CPU usage? Or quicker scans? Etc.
Just so I can give better feedback.
Thanks for all the hard work!
We’ve overhauled the entire build system (as shared in the OP).
This new build system benefits everyone, including the dev team.
- They can build a single package (for them to use duing Unit Development and Testing) of their work.
- If I need to rebuild any release to correct a packaging (install script) problem, I can do so without forcing a new build for all of PMS and bumping the version (wasteful).
- Build times are shorter because of how it’s built.
- Package sizes are smaller (once this little glitch is pushed out). If you look in the Alpha Testing thread, you will see the build sizes there. Those were created before this bug was introduced. Package sizes will return to that size when we’re done.
- Last, but by far least, the compilers and libraries were updated. It’s quicker and more efficient.
4 Likes
Hi @tamas,
I’m using this version of Plex on QNAP TS128A with QTS 4.3.4.0695 and seems that Plex uses lots of memory: QTS must stop Plex due to high memory usage many times. One time Plex caused restart of NAS.
Have you seen this issue yet?
Thans!
I am using the PMS on my development TS128A.
You will find it takes about 500 MB of RAM (right now) with everything running, including DVR, but should not be causing a restart of QTS.
I would like to see the logs if they are still available (you only have an hour after such restart)
1 Like
Yeah, the freebsd port wouldn’t install without forcing it because of that bug. When I stripped the binaries it complained about one (not sure which cuz I did a strip *). Seems to be as stable as any other release though.
Hello again, today I was at home for one day so I was able to do some testing. I did following:
- reinstalled PMS to new build
- moved whole library from the video folder to a new media folder so it does not conflict with DS video
- changed library location on PMS too
- gave user „Plex“ permission over all of the folders and subfolders
Then I did some testing it worked quite well, I was wven able to do a 4k transcoded to 1080p stream, then I started synchronisation on my iPhone. This worked well except for some songs so I restarted the process and did some more testing with transcoding songs while watching a 4k movie and listening to songs. Well I was probably going a little bit too crazy with it so it crashed and was inaccsessible for a minute (reasonable with just 1 GB of RAM on DS118). After this minute it was good again and I let the transcoding of the synchronizing songs do it‘s things overnight. Today I woke up and the server was again, like I found out previously completely inaccessible. It didn‘t recover again like it did last night with my heavy testing, I had to pull the cable because I couldn‘t even regularly shut down because the physical button didn‘t cause any reaction as well.
So I restarted and now I grabbed the logs, I hope there is something important.
I also took a screenshot of the realtime cpu statistics as you suggested, they were made while watching a movie 1080p DirectStream with audio transcoding from AC3 to AAC.
Logs:
Plex Media (2.1 MB)
Screenshot of Resource Monitor:
Thank you for your effort into this difficult situation, I can probably answer some questions next couple of days but I will be in holidays. Once the server is inaccessible I can‘t reach it from far away so I really can‘t do much testing.
Can see a restart after 08:23am
Plex Media Server process started a Plex Media Scanner job (PID 23750)
Sep 29, 2018 08:23:02.556 [0x7f848551d0] DEBUG - Job running: EAE_ROOT='/volume1/Plex/tmp_transcoding/pms-1975c4d1-aea6-4d9a-95b1-a7263363431f/EasyAudioEncoder' FFMPEG_EXTERNAL_LIBS='/volume1/Plex/Library/Application\ Support/Plex\ Media\ Server/Codecs/1.4-29a678f-9-linux-aarch64/' X_PLEX_TOKEN='xxxxxxxxxxxxxxxxxxxx' '/volume1/@appstore/Plex Media Server/Plex Media Scanner' '--scan' '--refresh' '--section' '2'
Sep 29, 2018 08:23:02.809 [0x7f848551d0] DEBUG - Jobs: Starting child process with pid 23750
Nothing was logged after this
The Scanner job started but hung
Sep 29, 2018 08:23:28.733 [0x7fa66ee1d0] INFO - Plex Media Scanner v1.13.6.283-9e747cb50 - Synology DS118 aarch64 - build: linux-aarch64 - GMT 02:00
Sep 29, 2018 08:23:28.798 [0x7fa66ee1d0] INFO - Linux version: DSM 6.2.23739-2, language: en-US
Sep 29, 2018 08:23:28.798 [0x7fa66ee1d0] INFO - Processor Unknown architecture
Sep 29, 2018 08:23:28.798 [0x7fa66ee1d0] INFO - /volume1/@appstore/Plex Media Server/Plex Media Scanner --scan --refresh --section 2
Your resource monitor screenshot was after all this - would have been interesting to see what it was at 08:23 am / 08:24 am
The server was restarted at 08:26:46
The problem is, that you can‘t really know when the crash will happen, it happens sometimes usually a couple of hours after restart. The only message I get is a mail from Synology stating that my Synology lost connection to the internet because the NAS also acts as a DynDNS updater.
Is there a way of recording activities and taking a look at them after a crash?
Maybe should move the DNS elements to a different box/system if you believe it is a factor
If the failure is beyond plex Media Server itself then that would need investigating as a Synology problem
I do not know what system diagnostic tools exist but you probably need something like the windows sysinternals process monitor.
There may be some system logs
I will leave this to @ChuckPa to advise on
It has to be a PMS problem, because when I shutdown PMS it doesn‘t crash anymore.
I have been doing some testing myself on the ARMv8 systems.
- If PMS is going to “crash”, it is a hard crash. The process terminates. It does not recover in a minute or so. Neither the Synology nor QNAP PMS packages work this way. When it stops, it remains stopped until restarted.
- Due to the limited amount of memory (1GB), I find these small boxes use SWAP a great deal more. Swapping reduces a program’s performance to a bare crawl of normal.
- As previously stated, PMS uses about 1/2 of installed RAM at this point. This will decrease in final form but for now, please stop other processes first and/or reduce the number of active apps running on the unit before making “Crash” assertions.
It would help everyone if we could use the terminology correctly.
- Swapping - Running out of physical memory, hard drive(s) flailing, apps and system slow to respond (protracted / possibly web-browser-expiring slow) but still operate otherwise normally.
- Freezing - Otherwise quiet before but becoming unresponsive for no apparent reason. The frozen app may or may not recover.
- Crash - Processes terminate, no longer listed in the active processes list, and do not recover unless manually restarted.
1 Like
Using the above terminology clarification,
I ask for a recap of what’s being seen.
The most exact definition would be the crash or maybe something similar to swapping. So what I was experiencing was a sudden inaccessability. After the crash the DS118 isn’t able to be accessed through Synologys Web Interface, Plex, SABnzbd, Synologys VPN on the DS118. What I found out is that the only way to access it is after about 6-10 hours after crash through the DS Finder App on the iPhone. Then it more feels like swapping because it’s still a low chance to get connected and it’s slow and unresponsive. From there you might be able to perform a restart. Once it was possible to access Plex and Synology’s Web Interface. But thats pretty difficult, same applies to the DS Finder version.
That sounds like you’re “swapping its brains out” when DS Finder (a core DSM application using 8 MB in DSM) stops responding to network queries
Running PMS (500 MB),
VPN (50 MB ?)
SABnzb (32 MB ?) when not processing anything,
Is already a heavy commitment on memory.
The key to diagnosis here is watching the disk LED. If it’s flicking, then everything is running and there is no alternative but to wait.
After it does return, you really have no choice but to start limiting the activity for now until PMS memory requirements (production form) drop.
