1.15.6.1079 not scanning directory and other problems [HELP: it's basically not working any more]

1.15.6.1079 not scanning directory and other problems [HELP: it’s basically not working any more]

Server Version#: 1.15.6.1079
Player Version#: 3.100.1
A couple of weeks ago I updated my plex-server installation form “https://downloads.plex.tv/repo/deb public main” and since then I haven’t been able to have a server up and running properly.

As soon as I updated my server stopped working, the daemon process would keep crashing.
not finding any info on the crash where I would expect a daemon to write it’s problems (the usual suspects in /var/logs/) I took the simple path and just deleted all the content of " /var/lib/plexmediaserver" and restarted the server which showed up, as expired, as brand new and ready to configure.

I proceeded dully configuring and re-adding all my directories to the library and let the raspberrypi churn for a couple of days. When I checked back, with great surprise, I found my server inaccessible, and checking the daemon’s status would report it as crashed.
Still not able to find anything of use in /var/logs I proceed yet again zapping the whole plexmediaserver directory and readded all my libraries.

Came back after some days to find my libraries were “mostly” reconstituted: some missing media and a lot of movies/shows missing their pictures and metadata.
I thought perhaps I gave it too much to chew on all at once so I restarted the daemon again just to see it crashing and not coming back up again. Not very fun.

Not finding yet again any useful pieces of information in /var/log I proceeded to venture all the way down to “/var/lib/plexmediaserver/Library/Application Support/Plex Media Server” where I found a log directory (standard locations exists so people can find things fast and not perhaps at the 3rd attempt) and after digging in those logs I found that perhaps looked like some kind of DB corruption as it was complaining about sqlite.
I followed the instruction I found on this page:m https://support.plex.tv/articles/201100678-repair-a-corrupt-database/ the server came back up, but my libraries were still a mess.
Deleted everything and proceded reconfiguring everything again and let it alone for yet some other days just to cam back to the same scenario.
Checking “/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases” I noticed there seemed to be some DB backups. I tried restoring them but all I obtained was a server with messy libraries.
I guess you can picture my frustration as I proceded deleting everything another time and re-configuring the server.

This time I added only 2 of the 4 directories I usually add, as my doubt of been feeding too much stuff was still alive. Let it scan everything an once it was done I was disappointed again by the result:

  • Home keeps showing the following error: “There was an unexpected error loading the dashboard Please visit our forums if you continue to experience problems”
  • the Movie library is missing most of the meta-data and pictures as well as apparently all the media in mkv format.
  • the Shows library says: “There are no items in this library” despite re-adding the directory and scanning multiple times.

I meticulously checked files and directories permissions (even tho NOTHING has changed from when it used to work just fine. I didn’t even add a single file) but they seem al correct.
Digging int he logs I now don’t find any additional error, and when I try re-adding the “shows” library none of “Plex Media Scanner.log” nor “Plex Media Server.log” report any scan error.

I’m defeated and a bit frustrated. Not knowing where to bang my head after 10 good days of this pointless fight I ask here if anyone has any idea of what is going on or what could cause this horrible experience.

I would provide tarballs with logs but as you probably suspect from my tale they’re long gone multiple times in my tries, and I already addressed any error I could find (the only one was actually the corrupted DB) setting both “Enable Plex Media Server debug logging” and “Enable Plex Media Server verbose logging” on.

one more thing:
not only is not working, it’s not even failing consistently as with every try I would face a slightly different broken scenario.

you didn’t seem to specify what specific OS you are running, and whether or not its virtualized.

Oh, pardon me, I thought “linux” “raspeberry” and the source of the deb could be enough to indicate where I’m running it:

root@raspberrypi:~ # uname -a
Linux raspberrypi 4.19.42-v7+ #1219 SMP Tue May 14 21:20:58 BST 2019 armv7l GNU/Linux
root@raspberrypi:~ # cat /etc/issue
Raspbian GNU/Linux 9 \n \l

no docker, directly installed on the system

Does this apply?

not really: the HD is HFS+, but most of all: it worked just fine for a couple of years, my troubles began after switching deb repo and installing the latest version. Unfortunately, I don’t remember which version I was running before as updates happen via cron for convenience.

What confuses me is not finding any error on the missing media:
If, for example, I add the Shows Library while tailing both the server and media scanner logs, the 1st one reports just general http communication bits, and the latter is silent so no real help finding out what’s happening :frowning:

Please verify debug logging is enabled and VERBOSE logging is off (Settings - Server - General - Show Advanced)

Next, rescan everything
When this finishes, download the full set of logs and attach.

Regarding the tip I showed, formatting doesn’t matter as much as mount point in the file system does and whether or not /etc/fstab is performing the mount or Nautilus is automounting and blocking access.

Permission denied errors should be visible in the logs.

Logs.bz2.tar (1.5 MB)

(it’s a tar.bz2, I had to do a switcheroo on the name because of the web form)

So here we are:

  • zapped all the plex data dirs
  • reconfigured my server as new
  • added all my directories
  • waited for scan to be done
  • the libraries were missing pictures again but most if not all my media seemed to be there (which is already an achievement considering the other tries)
  • rebooted my pi (for unrelated reasons)

And this is the result:

root@raspberrypi:/mnt/bigstore # systemctl status plexmediaserver.service
● plexmediaserver.service - Plex Media Server
   Loaded: loaded (/lib/systemd/system/plexmediaserver.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2019-06-04 18:16:51 UTC; 12min ago
 Main PID: 2795 (code=exited, status=255)
      CPU: 255ms

Jun 04 18:16:46 raspberrypi systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.
Jun 04 18:16:51 raspberrypi systemd[1]: plexmediaserver.service: Service hold-off time over, scheduling restart.
Jun 04 18:16:51 raspberrypi systemd[1]: Stopped Plex Media Server.
Jun 04 18:16:51 raspberrypi systemd[1]: plexmediaserver.service: Start request repeated too quickly.
Jun 04 18:16:51 raspberrypi systemd[1]: Failed to start Plex Media Server.
Jun 04 18:16:51 raspberrypi systemd[1]: plexmediaserver.service: Unit entered failed state.
Jun 04 18:16:51 raspberrypi systemd[1]: plexmediaserver.service: Failed with result 'exit-code'.

Now, indulging in my annoying side, this is where I’d expect to find some info about why prevented it from starting, perhaps a little more informative than a: Failed with result ‘exit-code’. :stuck_out_tongue:

I’m finding files in your scanner logs which aren’t recognized as video files

Example:

Jun 04, 2019 14:52:29.649 [0x74aba000] VERBOSE -     * Taking directory 'Vision of Escaflowne' (104) out of the map (26 left)
Jun 04, 2019 14:52:29.664 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/# instructions-AS2RC #.txt"
Jun 04, 2019 14:52:29.664 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 01.mkv.AS2RC"
Jun 04, 2019 14:52:29.664 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 02.mkv.AS2RC"
Jun 04, 2019 14:52:29.665 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 03.mkv.AS2RC"
Jun 04, 2019 14:52:29.665 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 04.mkv.AS2RC"
Jun 04, 2019 14:52:29.665 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 05.mkv.AS2RC"
Jun 04, 2019 14:52:29.665 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 06.mkv.AS2RC"
Jun 04, 2019 14:52:29.665 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 07.mkv.AS2RC"
Jun 04, 2019 14:52:29.666 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 08.mkv.AS2RC"
Jun 04, 2019 14:52:29.666 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 09.mkv.AS2RC"
Jun 04, 2019 14:52:29.666 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 10.mkv.AS2RC"
Jun 04, 2019 14:52:29.666 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 11.mkv.AS2RC"
Jun 04, 2019 14:52:29.666 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 12.mkv.AS2RC"
Jun 04, 2019 14:52:29.667 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 13.mkv.AS2RC"
Jun 04, 2019 14:52:29.667 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 14.mkv.AS2RC"
Jun 04, 2019 14:52:29.667 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 15.mkv.AS2RC"
Jun 04, 2019 14:52:29.667 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 16.mkv.AS2RC"
Jun 04, 2019 14:52:29.667 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 17.mkv.AS2RC"
Jun 04, 2019 14:52:29.668 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 18.mkv.AS2RC"
Jun 04, 2019 14:52:29.668 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 19.mkv.AS2RC"
Jun 04, 2019 14:52:29.668 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 20.mkv.AS2RC"
Jun 04, 2019 14:52:29.668 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 21.mkv.AS2RC"
Jun 04, 2019 14:52:29.668 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 22.mkv.AS2RC"
Jun 04, 2019 14:52:29.669 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 23.mkv.AS2RC"
Jun 04, 2019 14:52:29.669 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 24.mkv.AS2RC"
Jun 04, 2019 14:52:29.669 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 25.mkv.AS2RC"
Jun 04, 2019 14:52:29.669 [0x74aba000] DEBUG - Adding file for scanner: "/media/Library/Video/Anime/Vision of Escaflowne/Escaflowne 26.mkv.AS2RC"

AS2RC is not a recognized extension. It’s adding it to the scanner because the regex (regular expression) was triggered.

But your biggest issue is the Database is now corrupted.

Jun 04, 2019 18:16:46.404 [0x7329a450] INFO - Plex Media Server v1.15.6.1079-78232c603 - Raspbian GNU/Linux PC armv7l - build: linux-armv7hf_neon debian - GMT 00:00
Jun 04, 2019 18:16:46.404 [0x7329a450] INFO - Linux version: 9 (stretch), language: en-GB
Jun 04, 2019 18:16:46.404 [0x7329a450] INFO - Processor Arm
Jun 04, 2019 18:16:46.405 [0x7329a450] INFO - /usr/lib/plexmediaserver/Plex Media Server
Jun 04, 2019 18:16:46.400 [0x73c37000] DEBUG - BPQ: [Idle] -> [Starting]
Jun 04, 2019 18:16:46.400 [0x73c37000] VERBOSE - BPQ: delaying processing 120 second(s)
Jun 04, 2019 18:16:46.403 [0x73c37000] DEBUG - FeatureManager: Using cached data for features list
Jun 04, 2019 18:16:46.405 [0x73c37000] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.26.0, threadsafe=1
Jun 04, 2019 18:16:46.408 [0x73c37000] ERROR - SQLITE3:0x7eba3608, 11, database corruption at line 66053 of [bf8c1b2b7a]
Jun 04, 2019 18:16:46.408 [0x73c37000] ERROR - SQLITE3:0x7eba3608, 11, database disk image is malformed in "PRAGMA cache_size=2000"
Jun 04, 2019 18:16:46.410 [0x73c37000] ERROR - Database corruption: sqlite3_statement_backend::prepare: database disk image is malformed for SQL: PRAGMA cache_size=2000
Jun 04, 2019 18:16:46.410 [0x73c37000] ERROR - Error: Unable to set up server: sqlite3_statement_backend::prepare: database disk image is malformed for SQL: PRAGMA cache_size=2000 (N4soci10soci_errorE)

I recommend going into “/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases”

In there:

  1. Delete the *.wal and *.shm files
  2. copy the most recent com.plexapp.plugins.library.db backup to assume the primary role of the db itself.
  3. Make certain the ownership is plex:plex before restarting

OMG, some files got encrypted by some mining shell :sweat:#fail
well, it did minor damage, nothing I can’t recover.

For what the DB is concerned: yeah I did that already a number of times but as soon as I restart the server the DB gets corrupted again :frowning:

even tho those AS2RC files are suspicious, I don’t think it might be relates as that thing was able to mess like less than 50 files so I don’t think it was working since very long and some how I’m more inclined to see a connection with the server upgrade.

No way to prove either way based on what you’ve shown me.
I’ve had people swear up and down it was a Plex upgrade when they had forgotten something they changed. It’s not important what caused it. It is important we find the cause and fix it

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.