Server Inaccessible - Couldn't accept new TLS connection: Too many open files

Plex Media Server Desktops & Laptops
1

Server Version#: 1.18.3.2129 running on Ubuntu 16.04.3 LTS
Player Version#: using a variety of clients including iOS, Shield, Samsung + web

Hi,

As per the title of the post - every so often (weekly?) my Plex server goes inaccesible. When I check in the logs i get the following error

ERROR - Couldn’t accept new TLS connection: Too many open files

Any ideas on what might be causing this?

2

Need your DEBUG (not VERBOSE) log ZIP - containing activity just prior to the restart.

With that, I can see what drove it into that state.

3

Hi Chuck - have attacked the logs.

Timestamp for the error is

Jan 19, 2020 19:53:42.608 [0x7fab2f43c700] ERROR - Couldn’t accept new TLS connection: Too many open files
Plex Media Server Logs_2020-01-20_20-09-58.zip (4.1 MB)

Thanks!

4

It ran out of socket handles at 19:53:42.607 to be precise :slight_smile:

I was about to look and see how it recovered but the Plex Media Server.4.log file is missing.

All the other .4. rollovers are there.

Screenshot from 2020-01-20 17-57-10
Screenshot from 2020-01-20 17-57-10634×788 152 KB

5

I wasn’t able to get this file but the same issue just occurred. Have snagged every file from the logs folder. Server went inaccessible at Jan 28, 2020 17:16:35.436 [0x7fac95783700] ERROR - Couldn’t accept new TLS connection: Too many open files

Thanks!

Logs.zip (4.5 MB)

6

I think I’ve found it.

You have a flurry of sync activity. That activity is overloading your max network connections (128 by default). Normally it’s not a big deal but, with a default timeout of 60 seconds after being closed before the slot becomes available for other use, this does become a big problem.

We can either find out why there is such a flurry on the machine
-or-
customize the system’s configuration a bit to handle the higher load.

What is your preference?

7

Is this the same thing happening to other plex server unavailable issue ? maybe the others also ran out of connections ? is there a way to increase this max connections as some routers can handle much more ?

8

It’s not a router issue that I’m aware of from what the logs show me.
I see the flurry of activity. If the router were dropping, I’d see the disconnects which I don’t.

This is a non-permanent test you can perform. It is ONLY valid until you restart the system again.

  1. sudo cat /proc/sys/net/ipv4/tcp_fin_timeout
  2. I expect this value to be 60 (60 seconds)
  3. On the assumption it is 60, we will reduce by 1/2
  4. Reduce the value
  5. Restart PMS - Do not restart the machine
  6. Test

The entire session looks like this:

[chuck@lizum self.316]$ sudo cat /proc/sys/net/ipv4/tcp_fin_timeout
60
[chuck@lizum self.317]$ sudo sh
sh-5.0# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
sh-5.0# cat /proc/sys/net/ipv4/tcp_fin_timeout
30
sh-5.0#

This will give some measure as to whether you’re being overwhelmed with traffic or not. What this does is reduce the close wait time from 60 seconds to 30 seconds after any process is done with the connection. (The unused slots will become available again sooner).

If this results in no positive change after a period of testing, restart the machine to return to the Linux default values.

1 Like
9

Hey Chuck,

THat’s great. Ideally I’d like to understand what’s causing the overload of sync activity and so attack the problem at cause but am also happy to configure for higer maximum connections as a workaround.
Sync-wise - there shouldn’t be too much hapening as far as I know.

Thanks

10

I would too.

I feel the first step is to confirm whether or not it’s indeed an overload of the inbound requests to the server.

This run-time tweak does just that.

In the logs, I saw several sync requests generated at once. It looked like playlists were being pushed / updated (not clear in the logs) but a lot of traffic was flowing in that short time. The traffic isn’t so much a volume problem as a transactions/minute problem.

11

I’ve got a number (3-4) of ‘smart syncs’ e.g. Plex will sync particular shows when there’s a new episode which is unwatched. Could that be the cause?

12

I don’t have any playlists per se at all and therefore none to sync.

13

Don’t forget your shared users. You seem to have allowed the ‘Sync’ privilege to a few of them.

14

Thanks Otto - where can I see what they’ve set to sync?

15

Found it.

There’s 11 different sets of items syncing - ranging from single movies to one complete series of 25 items syncing.

There’s 4 sync sets remaining to complete including that large 25 item one which is at 39% and I think has been for a while

16

A pretty complete list is available in Tautulli. You can also remove Sync jobs with it.

17

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.