Ubuntu ran out of disk space, now Server Starts but is listed unavailable

[davideisendc]

Server Version#: 1.18.2.2041-3d469cb32
Player Version#:

Hi,

My server ran out of space due to a pesky app I didn’t notice running while on vacation. I can access port 32400 and it loads up properly, however the Plex Media Server itself reads unreachable.
Server Status Log:

● plexmediaserver.service - Plex Media Server
   Loaded: loaded (/lib/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-11-21 08:50:23 EST; 5min ago
  Process: 10948 ExecStartPre=/bin/sh -c /usr/bin/test -d "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" || /bin/mkdir -p "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" (code=exited, status=0/
 Main PID: 10957 (sh)
    Tasks: 60 (limit: 4915)
   CGroup: /system.slice/plexmediaserver.service
           ├─10957 /bin/sh -c  PLEX_MEDIA_SERVER_INFO_VENDOR="$(grep ^NAME= /etc/os-release | awk -F= "{print \$2}" | tr -d \" )"  PLEX_MEDIA_SERVER_INFO_DEVICE="PC"  PLEX_MEDIA_SERVER_INFO_MO
           ├─10972 /usr/lib/plexmediaserver/Plex Media Server
           ├─11001 Plex Plug-in [com.plexapp.system] /usr/lib/plexmediaserver/Resources/Plug-ins-3d469cb32/Framework.bundle/Contents/Resources/Versions/2/Python/bootstrap.py --server-version 1
           ├─11068 /usr/lib/plexmediaserver/Plex Tuner Service /usr/lib/plexmediaserver/Resources/Tuner/Private /usr/lib/plexmediaserver/Resources/Tuner/Shared 1.18.2.2041-3d469cb32 32600 /wai
           └─11098 Plex Plug-in [com.plexapp.agents.imdb] /usr/lib/plexmediaserver/Resources/Plug-ins-3d469cb32/Framework.bundle/Contents/Resources/Versions/2/Python/bootstrap.py --server-vers

Nov 21 08:50:23 plex systemd[1]: Starting Plex Media Server...
Nov 21 08:50:23 plex systemd[1]: Started Plex Media Server.

Nov 21, 2019 08:50:23.353 [0x7f3077325700] INFO - Plex Media Server v1.18.2.2041-3d469cb32 - Ubuntu PC x86_64 - build: linux-x86_64 debian - GMT -05:00
Nov 21, 2019 08:50:23.353 [0x7f3077325700] INFO - Linux version: 18.04.3 LTS (Bionic Beaver), language: en-US
Nov 21, 2019 08:50:23.353 [0x7f3077325700] INFO - Processor Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
Nov 21, 2019 08:50:23.353 [0x7f3077325700] INFO - /usr/lib/plexmediaserver/Plex Media Server
Nov 21, 2019 08:50:23.353 [0x7f3081220b80] DEBUG - BPQ: [Idle] -> [Starting]
Nov 21, 2019 08:50:23.353 [0x7f3081220b80] DEBUG - FeatureManager: Using cached data for features list
Nov 21, 2019 08:50:23.353 [0x7f3081220b80] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.26.0, threadsafe=1
Nov 21, 2019 08:50:23.369 [0x7f3081220b80] DEBUG - Running migrations. (EPG 0)
Nov 21, 2019 08:50:23.373 [0x7f3081220b80] DEBUG - ChangestampAllocator: initialized to 369510
Nov 21, 2019 08:50:23.373 [0x7f3081220b80] DEBUG - Opening 2 database sessions to library (com.plexapp.plugins.library.blobs), SQLite 3.26.0, threadsafe=1
Nov 21, 2019 08:50:23.375 [0x7f3081220b80] DEBUG - Running migrations. (EPG 0)
Nov 21, 2019 08:50:23.381 [0x7f3081220b80] DEBUG - HttpServer: Listening on IPv6 as well as IPv4.
Nov 21, 2019 08:50:23.381 [0x7f3081220b80] DEBUG - HttpServer: Listening on port 32400 with queue of 128 connections.
Nov 21, 2019 08:50:23.381 [0x7f3081220b80] DEBUG - HttpServer: Listening on port 32401 with queue of 128 connections.
Nov 21, 2019 08:50:23.385 [0x7f306e0b9700] DEBUG - Grabber: Cleaning up orphaned grabs.
Nov 21, 2019 08:50:23.385 [0x7f3081220b80] DEBUG - Media Provider: Registering provider com.plexapp.plugins.library
Nov 21, 2019 08:50:23.385 [0x7f3081220b80] DEBUG - Auth: Refreshing tokens inside the token-based authentication filter.
Nov 21, 2019 08:50:23.385 [0x7f306d8b8700] DEBUG - Relay: read 18 cached entries from hosts file
Nov 21, 2019 08:50:23.385 [0x7f3081220b80] DEBUG - Opening 1 database sessions to library (), SQLite 3.26.0, threadsafe=1
Nov 21, 2019 08:50:23.385 [0x7f306d0b7700] DEBUG - Auth: Refreshing tokens inside the token-based authentication filter.
Nov 21, 2019 08:50:23.385 [0x7f306d8b8700] DEBUG - HTTP requesting GET https://plex.tv/media/providers?X-Plex-Token=
Nov 21, 2019 08:50:23.385 [0x7f306e0b9700] DEBUG - Grabber: Cleaned up 0 decrepit directories in 0.0 sec.
Nov 21, 2019 08:50:23.386 [0x7f30579cb700] DEBUG - File "/usr/lib/plexmediaserver/Resources/Plug-ins-3d469cb32" changed: -1 => 2019-11-21 08:50:17 (1574344217).
Nov 21, 2019 08:50:23.386 [0x7f30579cb700] DEBUG - File "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-ins" changed: -1 => 2019-10-02 15:17:48 (1570043868).
Nov 21, 2019 08:50:23.386 [0x7f30579cb700] DEBUG - Scanning for plug-ins in "/usr/lib/plexmediaserver/Resources/Plug-ins-3d469cb32"
Nov 21, 2019 08:50:23.411 [0x7f30579cb700] DEBUG - Scanning for plug-ins in "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-ins"
Nov 21, 2019 08:50:23.411 [0x7f30579cb700] DEBUG - PluginRepository::setStartState: 1, startingSystem
Nov 21, 2019 08:50:23.411 [0x7f30579cb700] DEBUG - Starting plug-in /usr/lib/plexmediaserver/Resources/Plug-ins-3d469cb32/System.bundle.
Nov 21, 2019 08:50:23.412 [0x7f30579cb700] DEBUG - [com.plexapp.system] Setting plug-in to always running (daemon mode).
Nov 21, 2019 08:50:23.413 [0x7f30579cb700] DEBUG - Starting file watcher for com.plexapp.system
Nov 21, 2019 08:50:23.413 [0x7f30579cb700] DEBUG - Plugin: not setting user token for plugin
Nov 21, 2019 08:50:23.415 [0x7f30579cb700] DEBUG - Spawned plug-in com.plexapp.system with PID 11001
Nov 21, 2019 08:50:23.415 [0x7f30579cb700] DEBUG - [com.plexapp.system] Sending command: GET /:/prefixes^M
Nov 21, 2019 08:50:23.787 [0x7f306d8b8700] DEBUG - HTTP 401 response from GET https://plex.tv/media/providers?X-Plex-Token=
Nov 21, 2019 08:50:24.562 [0x7f30579cb700] DEBUG - Reading 486 bytes in the body, code is 200
Nov 21, 2019 08:50:24.562 [0x7f30579cb700] DEBUG - [com.plexapp.system] Plug-in running on port 35209.
Nov 21, 2019 08:50:24.562 [0x7f30579cb700] DEBUG -  * Plug-in handles prefix: /system
Nov 21, 2019 08:50:24.562 [0x7f30579cb700] DEBUG -  * Plug-in handles prefix: /player
Nov 21, 2019 08:50:24.562 [0x7f30579cb700] DEBUG - Read configuration for [com.plexapp.system], had 2 prefixes
Nov 21, 2019 08:50:24.563 [0x7f30579cb700] DEBUG - Done with 1 synchronous plug-in starts, starting the rest in parallel.
Nov 21, 2019 08:50:24.563 [0x7f30579cb700] DEBUG - PluginRepository::setStartState: 2, startingPlugins
Nov 21, 2019 08:50:24.563 [0x7f30579cb700] DEBUG - Updating the list of agents known by the system.
Nov 21, 2019 08:50:24.563 [0x7f30579cb700] DEBUG - [com.plexapp.system] Sending command over HTTP (GET): /system/agents
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - Image transcode cache directory: "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder"
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - Transcoder: Cleaning old transcode directories.
Nov 21, 2019 08:50:24.563 [0x7f30579cb700] DEBUG - HTTP requesting GET http://127.0.0.1:35209/system/agents
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 1 lo (127.0.0.1) (loopback: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 3 br0 (10.0.1.8) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 4 lxcbr0 (10.0.2.1) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 1 lo (::1) (loopback: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 2 enp3s0 (fe80::f639:9ff:fe33:d6df%enp3s0) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 3 br0 (2601:140:xxx:xxx:xxx) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 3 br0 (fe80::f639:9ff:fe33:d6df%br0) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 6 vethONGQT8 (fe80::fc87:3eff:fede:fae6%vethONGQT8) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG -  * 8 vethJDM7MR (fe80::fc86:dfff:feb5:4013%vethJDM7MR) (loopback: 0)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - Creating NetworkServices singleton.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkServices: Initializing...
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f30561c8700] DEBUG - Network change for advertiser.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.563 [0x7f30561c8700] DEBUG - NetworkService: Setting up multicast listener on 0.0.0.0:32414
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Creating new service.
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Got notification of changed network (first change: 1)
Nov 21, 2019 08:50:24.563 [0x7f3081220b80] DEBUG - NetworkService: Quick dispatch of network change.
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - Network change for advertiser.
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Setting up multicast listener on 0.0.0.0:32410
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - Network change for advertiser.
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Setting up multicast listener on 0.0.0.0:32412
Nov 21, 2019 08:50:24.564 [0x7f3081220b80] DEBUG - Starting HTTP server.
Nov 21, 2019 08:50:24.564 [0x7f3081220b80] DEBUG - Running server...
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Network change for browser (polled=0), closing 0 browse sockets.
Nov 21, 2019 08:50:24.564 [0x7f3081220b80] DEBUG - Waiting for server condition to be signaled.
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Setting up multicast listener on 0.0.0.0:32413
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Browsing on interface 127.0.0.1 on broadcast address 127.255.255.255 (index: 0)
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Browsing on interface 10.0.1.8 on broadcast address 10.0.1.255 (index: 1)
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Browsing on interface 10.0.2.1 on broadcast address 10.0.2.1 (index: 2)
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Network change for browser (polled=1), closing 0 browse sockets.
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Browsing on interface 127.0.0.1 on broadcast address 127.255.255.255 (index: 0)
Nov 21, 2019 08:50:24.564 [0x7f30561c8700] DEBUG - NetworkService: Browsing on interface 10.0.1.8 on broadcast address 10.0.1.255 (index: 1)
Nov 21, 2019 08:50:24.564 [0x7f30559c7700] DEBUG - Job running: '/usr/lib/plexmediaserver/Plex Tuner Service' '/usr/lib/plexmediaserver/Resources/Tuner/Private' '/usr/lib/plexmediaserver/Resources/Tuner/Shared' '1.18.2.2041-3d469cb32' '32600' '/waitmutex'
Nov 21, 2019 08:50:24.564 [0x7f30551c6700] DEBUG - HttpServer: Set up a thread pool with 2 threads.
Nov 21, 2019 08:50:24.564 [0x7f30559c7700] DEBUG - Jobs: Starting child process with pid 11068
Nov 21, 2019 08:50:24.564 [0x7f30551c6700] DEBUG - HTTP requesting GET https://plex.tv/api/v2/user/privacy?X-Plex-Token=
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Request: [127.0.0.1:43222 (Loopback)] GET /:/plugins/com.plexapp.system/messaging/function/X0J1bmRsZVNlcnZpY2U6QWxsU2VydmljZXM_/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMApyMAo_ (5 live) GZIP
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale hu
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale cs_CZ
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale ar
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale ro
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale fi
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale et_EE
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale no
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale pl
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale lt
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale sv
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale af
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale ja_JP
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale ko
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale nl_BE
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale ru
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale he
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale de
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale nl
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale it
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale zh_CN
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale hr_HR
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale da
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale en
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale fr
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale pt_BR
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale hu_HU
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale es
Nov 21, 2019 08:50:24.565 [0x7f303ffff700] DEBUG - Translator: Found locale sr
Nov 21, 2019 08:50:24.565 [0x7f303f7fe700] DEBUG - Request: [127.0.0.1:43224 (Loopback)] GET / (5 live) GZIP
Nov 21, 2019 08:50:24.565 [0x7f303e7fc700] DEBUG - Request: [127.0.0.1:43226 (Loopback)] GET / (5 live) GZIP
Nov 21, 2019 08:50:24.567 [0x7f303ffff700] DEBUG - [com.plexapp.system] Sending command over HTTP (GET): /:/plugins/com.plexapp.system/messaging/function/X0J1bmRsZVNlcnZpY2U6QWxsU2VydmljZXM_/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMApyMAo_
Nov 21, 2019 08:50:24.567 [0x7f303ffff700] DEBUG - HTTP requesting GET http://127.0.0.1:35209/:/plugins/com.plexapp.system/messaging/function/X0J1bmRsZVNlcnZpY2U6QWxsU2VydmljZXM_/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMApyMAo_
Nov 21, 2019 08:50:24.567 [0x7f303effd700] DEBUG - Request: [127.0.0.1:43228 (Loopback)] GET /servers (5 live) GZIP
Nov 21, 2019 08:50:24.567 [0x7f306e8ba700] DEBUG - Completed: [127.0.0.1:43224] 200 GET / (5 live) GZIP 2ms 1754 bytes
Nov 21, 2019 08:50:24.567 [0x7f306f0bb700] DEBUG - Completed: [127.0.0.1:43226] 200 GET / (5 live) GZIP 2ms 1754 bytes
Nov 21, 2019 08:50:24.567 [0x7f306e8ba700] DEBUG - Completed: [127.0.0.1:43228] 200 GET /servers (5 live) GZIP 2ms 483 bytes
Nov 21, 2019 08:50:24.601 [0x7f303f7fe700] DEBUG - Request: [127.0.0.1:43236 (Loopback)] GET / (5 live) GZIP

Any ideas on what could have happened?

I tried uninstalling and reinstalling to no avail. Also tried updating to a newer version. Tried repairing the corrupted database. Not really sure what else to do. Server is a little to big to just nuke it.

Thanks.

[davideisendc]

Checked also this page and it didn’t work either:

"No soup for you!" error after disk ran full

Ok… let’s make the VM PMS forget who it is and go through “reclaiming”. It won’t know you’re the owner at first. You will re-associate it with your Plex account. It will think it is in a first-run setup mode , which we will skip through. We’ll keep your data intact. If the server is sharing to anyone, you’ll need to share again. (unless we do text editor surgery – your choice) If you have custom settings in Settings - Server (any of the categories), you’ll need review and make certain it…

Any ideas?

[ChuckPa]

When PMS fails to return after a disk full event, the most common cause is because it became dissociated.

If the server instance is in a VM, and the VM is NAT (as most are), then SSH-tunnel into the VM is the easiest method to regain “console” level access to it again.

It’s impossible to have any ideas without seeing info about what’s failing.

[davideisendc]

Hi, I am into the console. Can see the logs, run SSH etc… ran all the steps above and listed in: "No soup for you!" error after disk ran full . It still is reporting no soup for you and won’t let me reassociate it…

[ChuckPa]

1. Sign the browser out of Plex/web (Upper right corner)
2. Exit the browser
3. Start the browser (yes this is silly)
4. Open incognito window through the ssh tunnel to the server’s 127.0.0.1