Linux, inotify, and CIFS... Oh My!

[Kermee]

So after running Plex Media Server for several years now under Windows 10 Pro, I’ve moved my Plex Media Server back to Linux, specifically, Ubuntu 20.04 Server. Instantly, I noticed that scanning upon “changes to library folders are detected”, doesn’t quite work as I would like it to as it did in Windows 10. And the reason why is, all of my media is stored on a Synology NAS and mounted as a CIFS share under my Ubuntu 20.04 Server running Plex Media Server.

This wasn’t an issue with running Plex Media Server under Windows 10 and having my media on Samba/CIFS shares mounted in Windows. Any changes to my library folders were literally picked up instantly. So after a bit of digging around, when running Plex Media Server under Linux, it relies on the inotify API service to detect changes. However, since it’s a kernel-level subsystem, if the changes to the directory are NOT initiated directly by the same system that Plex Media Server resides on, the kernel has no clue something has changed. CIFS unfortunately exploits this shortfall in the following manner:

Scenario #1: I’m on my Synology via DSM on a web-browser. I open up File Station, upload some videos off my phone and then move them into a directory which Plex Media Server has as a Video library called “Home Videos”. In this case, Plex has NO IDEA there’s new content and doesn’t get picked up until my library scan interval hits.

Scenario #2: I plug my phone via USB into Ubuntu Server which Plex Media Server resides on and shows up as a mass-storage device where I can access my media files. I copy the videos off my phone directly onto the CIFS share off my Synology which is my Video library called “Home Videos”. In this case, Plex INSTANTLY picks up there’s new content and adds it immediately to my library.

So, in a nutshell, I’m wondering if anyone has tried attacking this issue with a workaround. I do NOT manage my Plex Media Server library directly on the same server itself. Usually, it’s either directly on my Synology or on another machine (i.e. I copy my videos or pictures to a notebook computer and then upload them to my Synology). Both cases, Linux inotify has no clue something has changed on the CIFS share.

Obviously, a work around is running a Library scan interval at 15 minutes is a work around but a horrible one. The nice thing about the “partial scan” is that it only scans and updates the changes vs. scanning ALL my libraries with thousands of directories, just to add a couple, or even just one thing to my library.

The other obvious work around is running Plex Media Server directly off my Synology, but for many reasons, I do not and will not go that route.

So if anyone, again, has any suggestions or work arounds, would be greatly appreciated. Thanks in advance.

EDIT: One thing that did cross my mind, was building a simple service which runs on the same server as Plex Media Server (Linux) that continuously monitors my Libraries located on CIFS mounts. And if it detects a new directory or a change to a directory, it creates a 0-byte file named foo in that directory which triggers an inotify event. Then deletes the file.

When I was testing earlier in Scenario #1, if I was SSH’ed into the Ubuntu server which Plex Media Server was running on, I go into the directory where there’s new content but hasn’t been picked up yet, if I did a touch foo in that directory, Plex Media Server almost immediately detects the new content and adds it to my library. Likely, because my action triggered inotify since it was done directly on the system.

[ChuckPa]

I think you misunderstand some of how the inotify (Inode-NOTIFY) service works.

First, You’re correct that

1. Windows notification and Linux notification of network shares is vastly different.
2. Linux can immediately detect changes to those inodes (directories) which it has direct visibility to (i.e. anything mounted locally)

What you may not be aware of:

1. Linux is an event-driven (software interrupt - callback) mechanism

2. Windows is polling based. It keeps track of the last modified time for each monitored item. It then (frequently) polls the date time stamp of each directory in that list. If a change is detected, it generates the callback to the application in the same way the Linux kernel does.

3. NFS doesn’t support notification by the server to the client of a mounted filesystem. SMB does (limited) but must be enabled manually.

4. To work around this limitation of the protocols, Windows decided to make this a polling-based event for network shares which runs very frequently (every few seconds). Linux did not opt to make network mounts ‘polling’ based. Windows does:
   a. For each item in the notify list
   b. perform a “stat()” (get status) library call to retrieve date/time stamp information
   c. If any of the registered trigger flags are satisfied, generate a call to the registered user application function.

Because of these limits, PMS itself has mitigation.

1. You can specify Periodic scanning of the library ( enable polling )
2. You can limit the amount of scanning by enabling “Partial Scan” mode.
3. When Partial-scan mode is enabled, PMS performs a similar “stat()” call like Windows does and then compares the current “Last Modified” time to what it has stored in the database. This method emulates all of the existing monitored flags (CLOSE_WRITE, CREATE, MODIFY, DELETE, or MOVE)
   a. CLOSE_WRITE - Changes to the directory are complete, e.g. The file being copied into the directory are done
   b. CREATE - A new file or directory was created
   c. MODIFY - The directory itself was modified (anything in the directory changed)
   d. DELETE - Something was deleted from the directory - rescan for changes
   e. MOVE - Something in the directory or the directory itself was renamed.

If you’re curious why changes were likely detected when you did that ‘touch’, it’s because of the “CREATE” event ( a new file created by touch) and then scanned everything. Seem about right?

My suggestion to mitigate this is to use PMS’s internal mechanism.

1. Set period scan (5 minutes?)
2. Enable partial scan.
3. Keep immediate notification enabled if you have local media else turn off.

[Kermee]

@ChuckPa, thank so much for the very comprehensive response. It’s very much appreciated!

Yes, for now, I’m using the mitigation techniques you mentioned. I think the “highest” setting for period scan is 15 minutes unless I’m missing another setting somewhere else.

Everything you explained to me makes perfect sense. I’ll just roll with the settings you noted. Thanks again!

[ChuckPa]

It’s unfortunate that NFS behaves that way.
Depending on the SMB server you have , it is possible for SMB (Samba server) to use that local kernel’s inotify function and translate it to SMB 3.x messages.

I don’t know how well that works. (I don’t have anything Windows here)

[Kermee]

Yeah. The Samba server is running on the Synology, which I assume is just plain 'ole Linux smbd close to stock or actually stock. The CIFS shares are mounted SMBv3 (verified with mount on the Plex Media Server):

(rw,relatime,vers=3.0,cache=strict,username=xxxxx,uid=xxxx,forceuid,gid=xxxx,forcegid,addr=xxx.xxx.xxx.xxx,file_mode=0775,dir_mode=0775,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1)

Synology has a great page which explains all of the different SMB options available for their CIFS server, but none mention anything that would help us.

https://www.synology.com/en-global/knowledgebase/DSM/help/DSM/AdminCenter/file_winmacnfs_win

Just to be clear, I haven’t made any changes on the CIFS/file-server side. It’s the same CIFS shares that were mounted to my Windows 10 machine which ran Plex Media Server which are now mounted to my Ubuntu box running Plex Media Server. Just in my particular instance, the Windows “polling” seemed to have worked more quickly to library changes on the CIFS shares.

[ChuckPa]

synology does tend to be minimalist.

Whatever we get…