Non-responsive after 1.15.0.659 ubuntu upgrade

Something is causing network outage to plex.tv

Feb 12, 2019 23:30:31.443 [0x7f0c69bff700] DEBUG - Statistics: Flushing 15 expired bandwidth entries, 0 expired media entries.
Feb 12, 2019 23:30:36.820 [0x7f0c68bff700] DEBUG - EventSource: Failure in IdleTimeout (0 - Success).
Feb 12, 2019 23:30:36.820 [0x7f0c68bff700] ERROR - EventSource: Retrying in 30 seconds.
Feb 12, 2019 23:31:01.458 [0x7f0c617fe700] DEBUG - Statistics: Flushing 12 expired bandwidth entries, 0 expired media entries.
Feb 12, 2019 23:31:06.820 [0x7f0c68bff700] DEBUG - EventSource: Resolving 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct port 32400s
Feb 12, 2019 23:31:06.821 [0x7f0c68bff700] DEBUG - EventSource: Resolved 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct to 47.185.129.102
Feb 12, 2019 23:31:07.766 [0x7f0c68bff700] DEBUG - Auth: authenticated user 1 as hstorey219
Feb 12, 2019 23:31:07.766 [0x7f0c1b3ff700] DEBUG - Request: [192.168.1.152:40630 (Subnet)] PUT /myplex/refreshReachability (8 live) TLS Signed-in Token (hstorey219)
Feb 12, 2019 23:31:07.767 [0x7f0c1b3ff700] DEBUG - MyPlex: Requesting reachability check.
Feb 12, 2019 23:31:07.767 [0x7f0c1b3ff700] DEBUG - HTTP requesting PUT https://plex.tv/api/servers/23744339ec170ca42afad9f16585085626624b43/connectivity?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx&asyncIdentifier=26b9999d-6400-456e-bab4-0b2b35a152b9
Feb 12, 2019 23:31:08.296 [0x7f0c1b3ff700] DEBUG - HTTP 200 response from PUT https://plex.tv/api/servers/23744339ec170ca42afad9f16585085626624b43/connectivity?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx&asyncIdentifier=26b9999d-6400-456e-bab4-0b2b35a152b9
Feb 12, 2019 23:31:08.298 [0x7f0c68bff700] DEBUG - Completed: [192.168.1.152:40630] 200 PUT /myplex/refreshReachability (8 live) TLS 531ms 195 bytes (pipelined: 1)
Feb 12, 2019 23:31:08.352 [0x7f0c683fe700] DEBUG - Auth: authenticated user 1 as hstorey219
Feb 12, 2019 23:31:08.353 [0x7f0c15bfe700] DEBUG - Request: [192.168.1.152:40634 (Subnet)] GET /myplex/account (8 live) TLS Signed-in Token (hstorey219)
Feb 12, 2019 23:31:08.353 [0x7f0c683fe700] DEBUG - Completed: [192.168.1.152:40634] 200 GET /myplex/account (8 live) TLS 0ms 3186 bytes (pipelined: 1)
Feb 12, 2019 23:31:08.683 [0x7f0c1b3ff700] DEBUG - Request: [34.245.172.51:56118 (WAN)] GET /identity (9 live) TLS Signed-in Token (hstorey219)
Feb 12, 2019 23:31:08.684 [0x7f0c68bff700] DEBUG - Completed: [34.245.172.51:56118] 200 GET /identity (8 live) TLS 1ms 386 bytes (pipelined: 1)
Feb 12, 2019 23:31:09.090 [0x7f0c68bff700] DEBUG - EventSource: Got event [data] '<Message address="47.187.84.137" port="32400" asyncIdentifier="26b9999d-6400-456e-bab4-0b2b35a152b9" connectivity="1" command="notifyConnectivity"/>'
Feb 12, 2019 23:31:09.090 [0x7f0c68bff700] DEBUG - PubSub: Got notified of reachability: 1 for 47.187.84.137:32400
Feb 12, 2019 23:31:13.977 [0x7f0c68bff700] DEBUG - EventSource: Connected in 36 ms.
Feb 12, 2019 23:31:13.978 [0x7f0c68bff700] DEBUG - EventSource: Wrote data, reading reply.
Feb 12, 2019 23:31:14.030 [0x7f0c68bff700] DEBUG - EventSource: Read HTTP reply header.
Feb 12, 2019 23:31:14.030 [0x7f0c68bff700] DEBUG - EventSource: Successfully connected to 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct.
Feb 12, 2019 23:31:23.989 [0x7f0c68bff700] DEBUG - EventSource: Failure in ParseMessage (0 - Success).
Feb 12, 2019 23:31:23.989 [0x7f0c68bff700] ERROR - EventSource: Retrying in 15 seconds.
Feb 12, 2019 23:31:31.482 [0x7f0c61fff700] DEBUG - Statistics: Flushing 27 expired bandwidth entries, 0 expired media entries.
Feb 12, 2019 23:31:38.989 [0x7f0c683fe700] DEBUG - EventSource: Resolving 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct port 32400s
Feb 12, 2019 23:31:39.105 [0x7f0c683fe700] DEBUG - EventSource: Resolved 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct to 47.185.129.102
Feb 12, 2019 23:31:39.138 [0x7f0c68bff700] DEBUG - EventSource: Connected in 28 ms.
Feb 12, 2019 23:31:39.138 [0x7f0c68bff700] DEBUG - EventSource: Wrote data, reading reply.
Feb 12, 2019 23:31:39.192 [0x7f0c68bff700] DEBUG - EventSource: Read HTTP reply header.
Feb 12, 2019 23:31:39.192 [0x7f0c68bff700] DEBUG - EventSource: Successfully connected to 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct.
Feb 12, 2019 23:31:49.153 [0x7f0c68bff700] DEBUG - EventSource: Failure in ParseMessage (0 - Success).
Feb 12, 2019 23:31:49.153 [0x7f0c68bff700] ERROR - EventSource: Retrying in 15 seconds.
Feb 12, 2019 23:32:01.506 [0x7f0c5ebff700] DEBUG - Statistics: Flushing 37 expired bandwidth entries, 0 expired media entries.
Feb 12, 2019 23:32:04.153 [0x7f0c68bff700] DEBUG - EventSource: Resolving 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct port 32400s
Feb 12, 2019 23:32:04.155 [0x7f0c68bff700] DEBUG - EventSource: Resolved 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct to 47.185.129.102
Feb 12, 2019 23:32:04.185 [0x7f0c683fe700] DEBUG - EventSource: Connected in 26 ms.
Feb 12, 2019 23:32:04.185 [0x7f0c683fe700] DEBUG - EventSource: Wrote data, reading reply.
Feb 12, 2019 23:32:04.239 [0x7f0c683fe700] DEBUG - EventSource: Read HTTP reply header.
Feb 12, 2019 23:32:04.239 [0x7f0c683fe700] DEBUG - EventSource: Successfully connected to 47-185-129-102.233d5cc0f068488b9a41bf1e70916b47.plex.direct.
Feb 12, 2019 23:32:07.762 [0x7f0c683fe700] DEBUG - Auth: authenticated user 1 as hstorey219
Feb 12, 2019 23:32:07.762 [0x7f0c1b3ff700] DEBUG - Request: [192.168.1.152:40640 (Subnet)] PUT /myplex/refreshReachability (8 live) TLS Signed-in Token (hstorey219)
Feb 12, 2019 23:32:07.763 [0x7f0c1b3ff700] DEBUG - MyPlex: Requesting reachabilit

FWIW, I live on the east coast and we are having ice sttorms.

back down now if you wish.

I will continue investigating

Be safe!!! Let me know if you want more logs, I will leave it run overnight on 115. Right now at 1.27 gigs. See where is at in morning and will post it .

Thanks for your help!

1.27GB as in 1027 MB is normal.

agree, earlier today it was at 8 gigs, I should had saved the logs, I will let it run overnight and see, and report back.

When DVR / EPG runs, it will go up. There is so much changed that previous memory profiling norms don’t apply as much.

I checked that , nothing was running, not even streaming was going on :slight_smile:

even when defined, it will keep itself updated.

You’re right hstorey, that worked a treat - I see this isn’t the only post about v1.15 being broken on Linux systems
 At least this will keep me going for the time being.

Thanks

Morning ChuckPA,

Checked memory and it is at 4.4 gigs for the last 20 mins.

at 6:09 pm 5.25 gigs

Make certain you have Crashdump reporting enable.

Please do a kill -15 of the Plex pid

When you restart, it will upload.

Please capture the logs. Attach them in the PM I send you please

Sorry Chuck, how to you enable Crashdump?

Hi There,
ChuckPA and I did several test scenarios tonight. I did some screenshots of Plex running on 18.4.2 headless and Plex taking around 4 gigs of memory .
We turned off the DVR feature and rebooted the server. It has now stayed around several hundred megs.
Here is my ask, if you like to contribute in testing to see if there is a memory issue for sure, here is the action plan:

Take baseline screenshot, “top” is a good way of doing this (thanks Chuck:))
Let server run with DVR enable. run for around 24 hours and then take screenshot, See if memory has gone up without using the DVR feature.
Then disable DVR , reboot and take screenshot, 24 hours check and see if memory has gone up or not.
Post your findings on this thread.
Theory is that DVR subsystem may have a memory issue, but we need more people having memory growth to reproduce this symptom

P.S. ChuckPA is great to work with and so glad he assisted me with this issue, Thank you!

If I may add to that?

1.15.0.659 is the most recent version. Please test against this.

Hi Chuck,
After around 24 hours, 600-800 megs only :slight_smile: I believe it could be the DVR subsystem that is leaking.

That is a HUGE improvement.

Please allow it to continue for another 24.
If, at that time, it still holds true,
We will turn on DVR and recreate the escalation.

I will then submit and “Hot Sheet” it to engineering.

PS: How do you like top instead of htop with the simple colorization I set up?

Hi Chuck,

Yes like Top :), thanks for showing it to me!

Looks like server still doing better good, I will turn on DVR now for 24 hours

If you find that memory is going nuts at any point.

  1. Capture the logs (Download Logs)
  2. stop PMS
  3. Go into the Databases diretory
  4. Make a full tarball of if. We will then ask engineering what they need.