Two updates have killed my Plex setup

Yesterday I updated my Ubuntu Server to 16.04.1 then I updated my Plex updated to 1.0.3.2461
Then my problems started. Plex stopped working as the Plex install failed. I tried uninstalling it, re-installing, still the same Plex failed to install. Below are what the reports say.

(Reading database ... 135911 files and directories currently installed.)
Preparing to unpack .../plexmediaserver_1.0.3.2461-35f0caa_amd64.deb ...
Removed symlink /etc/systemd/system/multi-user.target.wants/plexmediaserver.service.
Unpacking plexmediaserver (1.0.3.2461-35f0caa) over (1.0.3.2461-35f0caa) ...
Setting up plexmediaserver (1.0.3.2461-35f0caa) ...
OK
Job for plexmediaserver.service failed because the control process exited with error code. See "systemctl status plexmediaserver.service" and "journalctl -xe" for details.
dpkg: error processing package plexmediaserver (--install):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for systemd (229-4ubuntu7) ...
Processing triggers for gnome-menus (3.13.3-6ubuntu3.1) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu5) ...
Processing triggers for mime-support (3.59ubuntu1) ...
Processing triggers for bamfdaemon (0.5.3~bzr0+16.04.20160701-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...

systemctl reads

Loaded: loaded (/lib/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
   Active: inactive (dead) (Result: exit-code) since Sun 2016-07-31 09:47:49 BST; 1min 16s ago
  Process: 11186 ExecStartPre=/bin/sh -c /usr/bin/test -d "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" || /bin/mkdir -p "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" (code=exited, status=216/GROUP)

Jul 31 09:47:44 Andy-server systemd[1]: Starting Plex Media Server for Linux...
Jul 31 09:47:44 Andy-server systemd[1]: plexmediaserver.service: Control process exited, code=exited status=216
Jul 31 09:47:44 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.
Jul 31 09:47:44 Andy-server systemd[1]: plexmediaserver.service: Unit entered failed state.
Jul 31 09:47:44 Andy-server systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.
Jul 31 09:47:49 Andy-server systemd[1]: plexmediaserver.service: Service hold-off time over, scheduling restart.
Jul 31 09:47:49 Andy-server systemd[1]: Stopped Plex Media Server for Linux.
Jul 31 09:47:49 Andy-server systemd[1]: plexmediaserver.service: Start request repeated too quickly.
Jul 31 09:47:49 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.

journalctl-xe reads

    > -- Unit plexmediaserver.service has finished shutting down.
    > Jul 31 09:47:38 Andy-server systemd[11184]: plexmediaserver.service: Failed at step GROUP spawning /bin/sh: No such process
    > -- Subject: Process /bin/sh could not be executed
    > -- Defined-By: systemd
    > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    > -- 
    > -- The process /bin/sh could not be executed and failed.
    > -- 
    > -- The error number returned by this process is 3.
    > Jul 31 09:47:38 Andy-server systemd[1]: Starting Plex Media Server for Linux...
    > -- Subject: Unit plexmediaserver.service has begun start-up
    > -- Defined-By: systemd
    > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    > -- 
    > -- Unit plexmediaserver.service has begun starting up.
    > Jul 31 09:47:38 Andy-server systemd[1]: plexmediaserver.service: Control process exited, code=exited status=216
    > Jul 31 09:47:38 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.
    > -- Subject: Unit plexmediaserver.service has failed
    > -- Defined-By: systemd
    > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    > -- 
    > -- Unit plexmediaserver.service has failed.
    > -- 
    > -- The result is failed.
    > Jul 31 09:47:38 Andy-server systemd[1]: plexmediaserver.service: Unit entered failed state.
    > Jul 31 09:47:38 Andy-server systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.
    > Jul 31 09:47:44 Andy-server systemd[1]: plexmediaserver.service: Service hold-off time over, scheduling restart.
    > Jul 31 09:47:44 Andy-server systemd[1]: Stopped Plex Media Server for Linux.
    > -- Subject: Unit plexmediaserver.service has finished shutting down
    > -- Defined-By: systemd
    > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

/lib/systemd/system/plexmediaserver.service reads

[Unit]
Description=Plex Media Server for Linux
After=network.target

[Service]
Environment="PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/var/lib/plexmediaserver/Library/Application Support"
Environment=PLEX_MEDIA_SERVER_HOME=/usr/lib/plexmediaserver
Environment=PLEX_MEDIA_SERVER_MAX_PLUGIN_PROCS=6
Environment=PLEX_MEDIA_SERVER_TMPDIR=/tmp
ExecStartPre=/bin/sh -c '/usr/bin/test -d "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" || /bin/mkdir -p "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}"'
ExecStart=/bin/sh -c 'LD_LIBRARY_PATH=/usr/lib/plexmediaserver /usr/lib/plexmediaserver/Plex\ Media\ Server'
Type=simple
User=plex
Group=plex
Restart=on-failure
RestartSec=5
StartLimitInterval=60s
StartLimitBurst=3

[Install]
WantedBy=multi-user.target

Hopefully someone can help as it is driving me nuts trying to fix it.

let me see /etc/systemd/system/plexmediaserver.service and/or /etc/systemd/system/plexmediaserver.service.d/over*.conf

What customizations have you performed on your installation?

No customisations that I know of, I am sure I only did a standard installation. Neither of the files that you asked for exists. The only one in /etc/systemd/system/ is plexmediaserver.service.prev

[Unit]
Description=Plex Media Server for Linux
After=network.target

[Service]
Environment="PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/var/lib/plexmediaserver/Library/Application Support"
Environment=PLEX_MEDIA_SERVER_HOME=/usr/lib/plexmediaserver
Environment=PLEX_MEDIA_SERVER_MAX_PLUGIN_PROCS=6
Environment=PLEX_MEDIA_SERVER_TMPDIR=/tmp
ExecStartPre=/bin/sh -c '/usr/bin/test -d "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" || /bin/mkdir -p "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}"'
ExecStart=/bin/sh -c 'LD_LIBRARY_PATH=/usr/lib/plexmediaserver /usr/lib/plexmediaserver/Plex\ Media\ Server'
Type=simple
User=plex
Group=plex
Restart=on-failure
RestartSec=5
StartLimitInterval=60s
StartLimitBurst=3

[Install]
WantedBy=multi-user.target

Rename the prev file.

sudo mv /etc/systemd/system/plexmediaserver.service.prev /etc/systemd/system/plexmediaserver.service
sudo systemctl daemon-reload

Next, show me the contents of /etc/os-release

I am thinking your OS distro is one of those the startup unification appears to have some trouble with. The OS release information will confirm this for me and also give me what I need to make sure it doesn’t happen again.

If it balks at systemctl then I know for certain it got confused.

I ran the sudo systemctl daemon-reload and it didn’t complain. Contents of /etc/os-release

NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial

Did PMS restart after renaming and then reloading the daemon with the normal systemctl start plexmediaserver ?

systemctl start plexmediaserver

`Job for plexmediaserver.service failed because the control process exited with error code. See “systemctl status plexmediaserver.service” and “journalctl -xe” for detail

systemctl status plexmediaserver.service

Loaded: loaded (/etc/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
   Active: inactive (dead) (Result: exit-code) since Sun 2016-07-31 18:52:55 BST; 1min 33s ago
  Process: 14473 ExecStartPre=/bin/sh -c /usr/bin/test -d "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" || /bin/mkdir -p "${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}" (code=exited, status=216/GROUP)

Jul 31 18:52:49 Andy-server systemd[1]: Stopped Plex Media Server for Linux.
Jul 31 18:52:49 Andy-server systemd[1]: Starting Plex Media Server for Linux...
Jul 31 18:52:49 Andy-server systemd[1]: plexmediaserver.service: Control process exited, code=exited status=216
Jul 31 18:52:49 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.
Jul 31 18:52:49 Andy-server systemd[1]: plexmediaserver.service: Unit entered failed state.
Jul 31 18:52:49 Andy-server systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.
Jul 31 18:52:55 Andy-server systemd[1]: plexmediaserver.service: Service hold-off time over, scheduling restart.
Jul 31 18:52:55 Andy-server systemd[1]: Stopped Plex Media Server for Linux.
Jul 31 18:52:55 Andy-server systemd[1]: plexmediaserver.service: Start request repeated too quickly.
Jul 31 18:52:55 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.

journalctl -xe

-- Unit plexmediaserver.service has finished shutting down.
Jul 31 18:52:44 Andy-server systemd[14471]: plexmediaserver.service: Failed at step GROUP spawning /bin/sh: No such process
-- Subject: Process /bin/sh could not be executed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- The process /bin/sh could not be executed and failed.
-- 
-- The error number returned by this process is 3.
Jul 31 18:52:44 Andy-server systemd[1]: Starting Plex Media Server for Linux...
-- Subject: Unit plexmediaserver.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit plexmediaserver.service has begun starting up.
Jul 31 18:52:44 Andy-server systemd[1]: plexmediaserver.service: Control process exited, code=exited status=216
Jul 31 18:52:44 Andy-server systemd[1]: Failed to start Plex Media Server for Linux.
-- Subject: Unit plexmediaserver.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit plexmediaserver.service has failed.
-- 
-- The result is failed.
Jul 31 18:52:44 Andy-server systemd[1]: plexmediaserver.service: Unit entered failed state.
Jul 31 18:52:44 Andy-server systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.
Jul 31 18:52:49 Andy-server systemd[1]: plexmediaserver.service: Service hold-off time over, scheduling restart.
Jul 31 18:52:49 Andy-server systemd[1]: Stopped Plex Media Server for Linux.
-- Subject: Unit plexmediaserver.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

PMS won’t install properly either since updates

I have gone back and double checked what you reported to me.

The message Failed at step GROUP means the Group= line is in error. Based on what you’re showing me Group=plex is correct unless there is no group entry plex in /etc/group.

If you cannot even install properly, there is something else going on which is preventing the installer (runs as root) from creating/checking group plex

At this point, uninstall and purge PMS. Your library will remain intact in /var/lib/plexmediaserver unless you manually delete it

When you reinstall, do not start it before checking the ownership of /var/lib/plexmediaserver

ls -la /var/lib/plexmediaserver should show it’s owned by plex:plex.

If it is not then sudo chown -R plex:plex /var/lib/plexmediaserver

Also check, just for sanity, the amount of free space for /var df /var Everything will go sideways if it’s full

As above, I have checked what you posted and there is no group called plex. Looks like we may be getting somewhere.

 ls -la /var/lib/plexmediaserver
total 12
drwxrwxrwx  3 andy andy 4096 Oct  9  2014 .
drwxr-xr-x 68 root root 4096 Jul 30 18:03 ..
drwxrwxrwx  3 andy andy 4096 Oct  9  2014 Library
andy@Andy-server:~$ sudo chown -R plex:plex /var/lib/plexmediaserver
[sudo] password for andy: 
chown: invalid group: ‘plex:plex’

Andy, Did you try to configure Plex to run under your username or leave it under plex?

Not that I am aware of. Yesterday server said there was an upgrade to 16.04.1 from my previous version, I installed that, then I did the plex update to version 1.0.3.2461 and it hasn’t worked since as the Plex update wouldn’t install as per above.

Then perhaps you were a bit overzealous with the ‘chown’ command at some point. Either way, this is not good and will need serious manual cleaning up

everything we do from here below needs to be done as ‘root’ so get a root shell up and be very careful. a bad mistake here can trash the machine

sudo sh in your terminal window

Do the normal Ubuntu remove and purge (dpkg and apt-get IIRC). Make sure PMS is completely gone.

userdel plex    # to remove as much as possible normally
grep plex /etc/passwd   #check for the plex user

If it shows, use kate /etc/passwd or gedit /etc/passwd and delete the entire line containing plex. If you’ve deleted anything, save and exit to commit the change.

now repeat this for /etc/group

grep plex /etc/group   #check for the plex group

If it shows, use kate /etc/group or gedit /etc/group and delete the entire line containing plex. Again, if you’ve deleted anything, save and exit to commit the change.

At this point, if you’ve done everything correctly, PMS will be completely gone from your machine.

A normal system installation should run. Do this while still in the root shell

Do NOT start Plex.

After successful installation (do not proceed if errors),

cd /var/lib
chown -R plex:plex plexmediaserver

If there are errors here, your system installation itself is seriously compromised.

All the above worked with no errors. Do you want me to try and start it?

I tried it, it’s working. Thank you so much for your help ChuckPa :slight_smile:

Rebuilding my libraries now, what a relief.