Transcoder fails due to lack of Notify space (edited)

General Discussions
41

If I can find an Nvidia would that play nice?

42

Any Nvidia which is on their list.

43

Excellent, thanks!
Plex Media Server Logs_2021-06-11_23-58-37.zip (6.5 MB)

44

Eagle Eye is still hanging at 33%. I’ll grab the logs again once it fails and stops.

45

I see it. You have more than 8192 directories to monitor.

Jun 11, 2021 23:57:24.110 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/World of Dance (2017)" (28: No space left on device)
Jun 11, 2021 23:57:24.111 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/World of Dance (2017)/Season 03" (28: No space left on device)
Jun 11, 2021 23:57:24.189 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/Young Sheldon (2017)" (28: No space left on device)
Jun 11, 2021 23:57:24.190 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/Young Sheldon (2017)/Season 01" (28: No space left on device)
Jun 11, 2021 23:57:24.192 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/Young Sheldon (2017)/Season 02" (28: No space left on device)
Jun 11, 2021 23:57:24.218 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/The Good Witch" (28: No space left on device)
Jun 11, 2021 23:57:24.219 [0x7f2ce0fc2b38] ERROR - [Notify] Failed to add watch for "/home/chris/server/tv/Grey's Anatomy" (28: No space left on device)
Jun 11, 2021 23:57:25.965 [0x7f2ce1273b38] DEBUG - Auth: authenticated user 1 as christophersanders0126
Jun 11, 2021 23:57:25.965 [0x7f2ce054cb38] DEBUG - Request: [192.168.0.57:65498 (Allowed Network (Subnet))] GET /statistics/bandwidth?timespan=6 (11 live) TLS GZIP Signed-in Token (christophersanders0126)
Jun 11, 2021 23:57:25.967 [0x7f2ce1273b38] DEBUG - Completed: [192.168.0.57:65498] 200 GET /statistics/bandwidth?timespan=6 (11 live) TLS GZIP 1ms 1192 bytes (pipelined: 11)
Jun 11, 2021 23:57:27.999 [0x7f2ce1296b38] DEBUG - Auth: authenticated user 1 as christophersanders0126

No space left on device (this is the Notify table being full) Nothing to do with the transcoder. :slight_smile:

46

No space left on what device? My NAS isn’t full. I just extended the partition on the Plex server.

47

Now we find out how many you do have and add a buffer on top of that.

sudo find /home/chris/server -type d -print | wc -l

This will give us a number “N”

Now some basic calculator math to round up to the next 32768 block.

( ( (N + 32768) / 32768 ) + 1 ) * 32768)

  1. number of directories (N + 32768) / 32768 == number of “blocks”

  2. Add 1 (give ourselves another block)

  3. (Blocks * 32768) = Convert “blocks” back to individual directory count

When that’s done:

sudo echo 'fs.inotify.max_user_watches=YOUR_NUMBER_HERE' >> /etc/sysctl.conf
sudo sysctl -p
48

This ‘no space’ has NOTHING to do with storage. This is where you panicked the first time.

It is the size of the Linux kernel’s notify table. (in RAM)

Show me your math before you do anything.

49 
root@plex:/home/chris/server/movies# sudo find /home/chris/server -type d -print | wc -l
find: ‘/home/chris/server/tv/#recycle’: Permission denied
find: ‘/home/chris/server/music/#recycle’: Permission denied
find: ‘/home/chris/server/movies/#recycle’: Permission denied
9112
root@plex:/home/chris/server/movies#
50

Ok… that looks like a good number …

here we go

9112 + 32768 = 41880

41880 / 32768 = 1.27

We keep the integer part only

we are using 1 full block

Now to add 1

1 + 1 = 2

2 * 32768 = 65536

sudo echo fs.inotify.max_user_watches=65536 >> /etc/sysctl.conf
sudo sysctl -p

Now you are set quite a while

Restart Plex.

Let’s see what it doesn’t like next but we should be there. The EAE won’t fail now

Note: that 65536 table size uses 32 MB of memory (trivial)

51

Eagle Eye now plays! woohoo!

52

Good to go? :slight_smile:

53

It certainly looks like it. So the value that we just changed to 65536…what exactly is that defining and is it anything I need to be worried about changing again in the future?

54

The table which monitors directory changes (the kernel “notify” service) is what we just adjusted.

By default, it can monitor 8192 directories.

You have more than that.

To make sure you have enough room for growth, I bumped you up to a maximum number of 65536 directories which can be watched.

The reason this is significant is ----

  1. The transcoder needs one Notify slot
  2. The Easy Access Encoder (Dolby) also needs one.

When the table is full, the transcoder and EAE fail.

Only the admin can increase the table size

On your next VM or bare metal install.

Add that to /etc/sysctl.conf before you even install Plex and you’ll be trouble free — until you fill it again :wink:

55

If I run into this again can I edit it again to increase the maximum number of watched directories or is there a cap?

56

Edit /etc/sysctl.conf and increase the value.

to make active

  1. reboot
    -or-

  2. sudo sysctl -p

  3. restart plex

57

Perfect! Thank you!

You mentioned earlier about a RAM Disk. Should that be configured rather than transcoding on the RAID array?

58

You’re in a VM. You are losing far more in the VM than you would gain back.

I recommend not complicating things until your mastery level is higher.

59

Understood.

Chuck, thank you so much for sticking with me through all of this. You’re a Plex God!

60

God? —or just an old fart ? :thinking:

:rofl:

Have a great evening and weekend.