Synology 1821+ / DSM 7.1-42661 U1 / Plex 1.26.2.x to 1.27.0.x db corrupt / library access

[LordCochise]

Folks,

Having an issue trying to upgrade from 1.26.2 to 1.27.0.x on a Synology 1821+ with DSM 7.1. I’m at the point where if I have to nuke the db and start over, that’s not the end of the world but it’d be nice if there’s something i missed…

This box has been running Plex for some time, first major issue upon upgrading to 1.27.0.x (one of the earlier Plex-pass builds). It will upgrade and run ok, but multiple libraries will report as ‘something went wrong’ when trying to access, as well as individual episodes / movies if I can get to them via collections / recommended / etc rather than browsing the library itself. Uninstalling 1.27.0.x and going back to 1.26.2 consistently re-allows those libraries / media to be accessed normally.

Have tried optimizing database, performing PRAGMA integrity check as well as the console recovery command, both at the 1.26.2 level AND at the 1.27.0 level, both appear to return the same results (and while the recovery process completes, it does not seem to correct whatever the issue is.

This is the output of the PRAGMA check, first on 1.26.2, then the recovery process, then another PRAGMA check on 1.27.0.5897

The relevant part of Plex Media Server.log upon installing / starting 1.27.0.x is the following:

Jun 13, 2022 18:03:55.026 [0x7f117e9d6b38] INFO - Plex Media Server v1.27.0.5897-3940636f2 - Synology DS1821+ x86_64 - build: linux-x86_64 synology-dsm7 - GMT -04:00
Jun 13, 2022 18:03:55.026 [0x7f117e9d6b38] INFO - Linux version: DSM 7.1.42661-1, language: en-US
Jun 13, 2022 18:03:55.026 [0x7f117e9d6b38] INFO - Processor: 8-core AMD Ryzen Embedded V1500B
Jun 13, 2022 18:03:55.026 [0x7f117e9d6b38] INFO - Compiler is - Clang 11.0.1 (https://plex.tv 9b997da8e5b47bdb4a9425b3a3b290be393b4b1f)
Jun 13, 2022 18:03:55.026 [0x7f117e9d6b38] INFO - /volume1/@appstore/PlexMediaServer/Plex Media Server
Jun 13, 2022 18:03:55.024 [0x7f11821bc0d0] DEBUG - BPQ: [Idle] → [Starting]
Jun 13, 2022 18:03:55.025 [0x7f11821bc0d0] DEBUG - FeatureManager: Using cached data for features list
Jun 13, 2022 18:03:55.026 [0x7f11821bc0d0] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.35.5, threadsafe=1
Jun 13, 2022 18:03:55.077 [0x7f11821bc0d0] DEBUG - Running migrations. (EPG 0)
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Running forward migration 202204252330.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 0.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 1.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 2.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 3.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 4.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 5.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 6.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 7.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 8.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 9.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 10.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 11.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 12.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 13.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 14.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 15.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 16.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 17.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 18.
Jun 13, 2022 18:03:55.078 [0x7f11821bc0d0] DEBUG - Captured session 19.
Jun 13, 2022 18:03:55.092 [0x7f11821bc0d0] ERROR - SQLITE3:0x80000001, 779, index corruption at line 91877 of [1b256d97b5]
Jun 13, 2022 18:03:55.092 [0x7f11821bc0d0] ERROR - SQLITE3:0x80000001, 779, statement aborts at 73: [update metadata_item_settings set last_viewed_at = iif(typeof(last_viewed_at) in (‘integer’, ‘real’), last_viewed_at, strftime(‘%s’, last_viewed_at, ‘utc’)), last_skipped_at = iif(type
Jun 13, 2022 18:03:55.092 [0x7f11821bc0d0] ERROR - Exception inside transaction (inside=1) (/data/jenkins/server/3532656870/Library/DatabaseMigrations.cpp:254): sqlite3_statement_backend::loadOne: database disk image is malformed
Jun 13, 2022 18:03:55.094 [0x7f11821bc0d0] ERROR - Exception thrown during migrations, aborting: sqlite3_statement_backend::loadOne: database disk image is malformed

I’m used to SQL Server more than SQLite but if I can run a few SQL statements to correct it, I can do that; seems the issue is just some metadata indexes referring to last watched / view / etc, which I’m plenty ok with blowing out rather than nuking the entire db.

I normally update the PMS version with whatever the new plex pass beta version is, give it time to do any schema updates etc. so this wasn’t a stopped-halfway sort of thing, and I have the scheduled tasks set for the default early-morning times so I don’t believe there was anything running at the time of update, but can’t guarantee that 100%.

I should add that while PMS has been running on this synology since early 2021, though it’s been set up to do regular backups every 3 days, it stopped at 11/08/21, though the blobs.db backups still seem to be continuing (perhaps that change was expected around that time?) This is what the db directory looks like currently:

I did not have the -shm and -wal files in this directory prior to running the above recovery (except for the DLNA ones which were already there); the library.blobs ones have stayed static since the recovery op, but the library.db ones seem to keep re-appearing now, even if they’re deleted.

This is the first time I’ve had to go to this level for any kind of PMA recovery, so any suggestions are welcome; my metadata really doesn’t have a lot of customizations though so if any recovery op is more than, say, 30 mins or so of work, I’ll just nuke it and start fresh in 1.27.0.x…

[pl_5309]

PRAGMA check only checks for errors kind of like DBCC checkdb. You would need to do the remaining steps to actually repair the database. Since it has been corrupt since 11/08/2021, not sure you will get a clean db at the end.

Some discussion on the repair process and suggestion for doing a .recover instead. Version 1.27.0.5849 - kills Media server - #30 by shiloh100

[ChuckPa]

There is disparity here in that you don’t have any DB backups from 2022.

If it stopped making backups – I suspect a problem with the permissions in the PlexMediaServer shared folder itself were damaged somehow. That can be addressed a bit later.

For the first step.

1. Stop Plex
2. Remove the com.plexapp.plugins.libary.db -shm and -wal files (these are caches)
3. COPY com.plexapp.plugins.library.db-2021-10-30 to another folder somewhere.
4. In that other folder, rename it to com.plexapp.plugins.library.db (this is how we get around a DSM 7 behavior issue)
5. Now copy it back to the Databases folder and OVERWRITE the one which is there.
6. Right-click com.plexapp.plugins.library.db and confirm ownership is still PlexMediaServer and PlexMediaServer has full read/write permissions.
7. Now start the server and, using the Syno health widget, wait for the CPU to drop to idle again.

This will take some time for it to take the old db and upgrade it (schema migrations).

When the CPU is again idle, Open PMS and Scan files & Refresh Metadata to get the DB caught up with current.

If this process fails, you’ll need to nuke and start over.

[LordCochise]

I did a recover after the first PRAGMA check, and redid the check afterwards to see if there was any difference (there was not)

[LordCochise]

Chuck, I’m sure you get it a lot, but thanks, and that process worked. I believe I may have done the ‘Upgrade Libraries’ processes to use the new metadata agents around the time the backups stopped, not sure if that may have played a factor or not. After restoring the db, i did have to re-upgrade the libraries / refresh metadata afterwards, which all seemed to pass through fine.

Libraries / media are all accessible in 1.27.x as expected. Thanks again!

[ChuckPa]

@LordCochise

I suggest keeping an eye on it making DB backups.

If it doesn’t , we’ll get into it.