On FreeBSD it appears that Plex Media Server doesn't properly implement kqueue to allow the "Update my library automatically" feature to work. It's a real hassle to lose this core feature. I've already set my "update my library periodically" to be hourly which is already considerably more-frequent than I'd like and triggers massive unnecessary server load, and I don't dare set it any more-frequently... however, it still means that users in my household can't upload a video and have it be immediately available to watch. They have to wait up to an hour without my intervention.
Â
Since FreeBSD is officially supported now it'd be nice if PMS properly utilized kqueue to detect filesystem changes and allow this feature to work like it does on other platforms. Thanks.
I’ve been looking into this in some spare time and have discovered that kqueue does have some severe limitations. TBH, FreeBSD desperately needs something better designed along the lines of Linux’s inotify or OS X’s fsevents as kqueue is quite the heavy hammer to throw at monitoring a directory tree for changes. The most notable limitation is that the process must open every single file and directory (read only) and keep them open to monitor their changes. There is a limitation in the number of open files allowed by a process which may be different for others. Since my system has much more RAM than normal, my limits are much higher than others. So could you help me in a quick feasibility check (for those with really large libraries)?
Check the allowed number of files per process: sysctl -a | grep files
Here the kern.maxfiles and kern.maxfilesperproc are hard upper limits and kern.openfiles is the number of files currently open.
Check the number of files/directories in your library: execute find . | wc -l in each of your library root paths and sum the results
In my system, the number of allowed open files numbers just below a million which is far beyond the number of files in my libraries. I want to know if the same is true for others.
@sremick said:
Have you tried posting for insight/assistance over at the FreeBSD forums?
I’m not sure what part you are asking about here. I’ve conducted experiments using kqueue on small directory structures and performed a large set of changes to determine what it can and cannot detect. For example, opening a directory and monitoring it for changes will not inform of changes to an existing file within that directory but instead only content additions/subtractions. This is one of the limitations of using kqueue for FS monitoring. It is almost certainly well known in the FreeBSD community. After all, a google search for kqueue file change yields a developer article from Apple talking about it: https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html Stated there is
If you are monitoring a large hierarchy of content, you should use file system events instead, however, because kernel queues are somewhat more complex than kernel events, and can be more resource intensive because of the additional user-kernel communication involved.
Note: file system events is available on OS X but not in FreeBSD.
If you are talking about the questions I posed, they wouldn’t really be of much help over there as my questions are installation specific. The kern.maxfiles and kern.maxfilesperproc values are set on startup dependent on the amount of RAM in the system. That’s not as much of a concern unless there is someone out there with very little RAM in their FreeBSD machine. Given the specs in your sig, I expect your limits are around 500,000. The real question is the size of people’s Plex library which is certainly not a FreeBSD question.
I’m trying to assess how feasible kqueue is for libraries before going further.
When I posed the question of a better FS notification system, the people over at forums.freebsd.com got pretty defensive of their kqueue/events.
I am not technically adept enough to explain why plex doesn’t use kqueue, but when given the scenario of a plex server, they seem to think kqueue would work just fine. Descriptor limits can be raised, etc.
Plex devs-
Can you please tell me why kqueue won’t work?
@aschmidtm said:
When I posed the question of a better FS notification system, the people over at forums.freebsd.com got pretty defensive of their kqueue/events.
link? If they have a good means for monitoring a large directory tree, I’d love to see it.
Plex devs-
Can you please tell me why kqueue won’t work?
Guessing what they said but there’s far more to the resource usage than just the open files. Because kqueue only tells you that something about an fd changed, not what or where, a file system monitoring tool must also map those FDs to paths, and if the fd is a directory, it must rescan the contents of the directory to find changes. It must keep this mapping up to date with every change. There is a tremendous amount of bookkeeping that must be done and must be exactly right (I know what’s involved as I’ve taken a few stabs at it already). We are talking code that’s around as large as the code for Linux, Windows, and MacOS platforms combined!
BTW, do you happen to know the size of your library? You can run find /path/to/libarry | wc -l to yield a pretty good approximation. Also, the the directory count is also useful in determining feasibility find -type d /path/to/libarry | wc -l
For anyone else seeing this thread, I don’t need any more examples of library sizes/maxfiles. I have obtained an example where one user has enough library items to exceed the allowed open files for the whole system. From some of the “build threads” available here, I know there are more. Given that user libraries can grow irrespective of the system’s RAM, this situation will become more prevalent, especially considering that lately users can add library types with a greater file count / size ratio (photos and music).
Given how detrimental exceeding the open file count can be, I’d be scared to release any implementation using this mechanism of file system change detection. I suppose that it could read the maxfiles values, but this is yet another step in the complexity on this platform that already greatly exceeds that of others.
Let’s be honest here: kqueue is designed to detect changes associated with an open file descriptor. Using this to detect changes in the file system is a bit of a hack. While it works on a limited scale, doing this on large file system trees is abusing the system to accomplish a task for which it was not designed. A proper file system change detection system needs to be implemented in the kernel at its VFS layer.
