PMS Failing to start - Database corruption: sqlite3_statement_backend::loadOne: database or disk is

Plex Media Server NAS & Devices
1

Server Version#: 1.40.5.8854-f36c552fd
Player Version#: N/A
<If providing server logs please do NOT turn on verbose logging, only debug logging should be enabled>

Plex Media Server has been running fine for about a year, pushed an update through the App Manager yesterday and now it won’t work

When I start the app it will run an xml page with this status for a few minutes which will then just fail

<Response code="503" title="Maintenance" status="Plex Media Server is currently running database migrations.">

I have tried ChuckPA’s DBrepair tool and that is reporting no issues

Automatic Check,Repair,Index started.

Checking the PMS databases
Check complete.  PMS main database is OK.
Check complete.  PMS blobs database is OK.

Exporting current databases using timestamp: 2024-08-24_11.40.23
Exporting Main DB
Exporting Blobs DB
Successfully exported the main and blobs databases.
Start importing into new databases.
Importing Main DB.
Importing Blobs DB.
Successfully imported databases.
Verifying databases integrity after importing.
Verification complete.  PMS main database is OK.
Verification complete.  PMS blobs database is OK.
Saving current databases with '-BACKUP-2024-08-24_11.40.23'
Making repaired databases active
Repair complete. Please check your library settings and contents for completeness.
Recommend:  Scan Files and Refresh all metadata for each library section.

Backing up of databases
Backup current databases with '-BACKUP-2024-08-24_11.46.01' timestamp.
Reindexing main database
Reindexing main database successful.
Reindexing blobs database
Reindexing blobs database successful.
Reindex complete.
Automatic Check, Repair/optimize, & Index successful.

I have tried backup databases from when it was working fine and still the same issue ( I have done this manually and using Chuck’s tool

Are you sure you want to restore a previous database backup (Y/N) ? Y
Checking for a usable backup.
Database backups available are:  2024-08-21 2024-08-18 2024-08-15 2024-08-09
Checking database 2024-08-21
Found valid database backup date: 2024-08-21
Use backup '2024-08-21' ? (Y/N) ? N
Checking database 2024-08-18
Found valid database backup date: 2024-08-18
Use backup '2024-08-18' ? (Y/N) ? Y
Saving current databases with timestamp: '-BACKUP-2024-08-24_11.48.33'
Copying backup database 2024-08-18 to use as new database.
Copy complete. Performing final check
Database recovery and verification complete.

df -h is showing more than enough space

Filesystem      Size  Used Avail Use% Mounted on
rootfs          287M   60M  227M  21% /
tmpfs           310M  1.9M  309M   1% /tmp
/dev/md0        2.0G  345M  1.5G  19% /volume0
/dev/loop0      951K   16K  915K   2% /share
/dev/md1         17T   12T  4.9T  70% /volume1
cgroup          310M     0  310M   0% /sys/fs/cgroup
overlay          17T   12T  4.9T  70% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/09ab5f65f9ece352b0e3204fe3a222fd8009017d8ceaa95b2c9bd5a937e59c99/merged
overlay          17T   12T  4.9T  70% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/833fc67b5b45a2f5e55ddcf0dbf2728c8678ba6187fea5df4fa56c231a724e1f/merged
overlay          17T   12T  4.9T  70% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/1187b3fd21d689bd7e7a35d25eb142117922106b40c0184eacadaa0231c880e5/merged

I have tried deleting the db files (I backed them up) and letting it start again but no luck.

I have reinstalled through the app manager, I can’t see anyway to install an older version manually but I don’t know if that would help

The only thing that has changed is I pushed the update

Does anyone have anything else I can check or try to get this working again?

Plex Media Server (8).log (55.7 KB)

2

sqlite3_statement_backend::loadOne: database or disk is full means that the disk plex is using is full, free up some space or make the drive larger.

3

I have 1 volume which has just under 5TB free I can’t see where or why it would report that

Based on the logs the App central entries in my df -h are Plex which have the 5TB available

Screenshot_2024-08-24-18-47-38-59_33c485448b01a70faac917a22ce09db9
Screenshot_2024-08-24-18-47-38-59_33c485448b01a70faac917a22ce09db9527×948 43.6 KB

4

Can you post a complete copy of your server logs? They may show something that your excerpt didnt.

5

I included the full server log in the post are there other logs I can grab?

6

You included 1 log file out of anywhere from 20-100 log files that the server generates. If you see the yellow server logs link it explains what they are and where you can find them and how to upload a zip of those logs.

7

Here’s the full log folder compressed. Appreciate the help I couldn’t find anything abnormal other than what I’ve posted when checking

Logs.zip (309.3 KB)

8

Ok looks like we are working on a NAS can you check the free space on /volume1/Plex/Library (I am going to make the assumption that the error you are getting is a red herring if there is space there) If there is space available can you check your database and attempt a repair using GitHub - ChuckPa/PlexDBRepair: Database repair utility for Plex Media Server databases

9

Tried that already and it reported no issues.

Automatic Check,Repair,Index started.

Checking the PMS databases
Check complete.  PMS main database is OK.
Check complete.  PMS blobs database is OK.

Exporting current databases using timestamp: 2024-08-24_11.40.23
Exporting Main DB
Exporting Blobs DB
Successfully exported the main and blobs databases.
Start importing into new databases.
Importing Main DB.
Importing Blobs DB.
Successfully imported databases.
Verifying databases integrity after importing.
Verification complete.  PMS main database is OK.
Verification complete.  PMS blobs database is OK.
Saving current databases with '-BACKUP-2024-08-24_11.40.23'
Making repaired databases active
Repair complete. Please check your library settings and contents for completeness.
Recommend:  Scan Files and Refresh all metadata for each library section.

Backing up of databases
Backup current databases with '-BACKUP-2024-08-24_11.46.01' timestamp.
Reindexing main database
Reindexing main database successful.
Reindexing blobs database
Reindexing blobs database successful.
Reindex complete.
Automatic Check, Repair/optimize, & Index successful.

I tried one of the backup DBs using the tool and manually just renaming them but nada.

It has 5 TB available and it has no quotas on space/files

df -h Library/
Filesystem      Size  Used Avail Use% Mounted on
/dev/md1         17T   12T  4.9T  70% /volume1

repquota: Mountpoint (or device) /share/Plex not found or has no quota enabled.
10

@ChuckPa Thoughts?

11

What is this please ?

12

MOST IMPORTANT ==

Aug 24, 2024 14:49:15.332 [548097122336] ERROR - Error: Unable to set up server: sqlite3_statement_backend::loadOne: database or disk is full (N4soci10soci_errorE

Disk is full.

The database is NOT malformed else it would say so.

Is this on Volume1 ?

13

Yes it’s on volume 1
Df -h in last response on library (I was in the Plex path already) shows its in /dev/md1 which is on volume on

That was the response to repquota, I didn’t set any up but was double checking none were set by something

I didn’t include the command just the output

14

Did quotas accidentally get applied to the Plex / PlexMediaServer shared folder ?

They should be unlimited.

If quota applied, it would show as Full.

15

There are no quotas applied it was just another thing I was checking as nothing else is making sense.

Just to confirm:
The volume has just under 5TB free, there are no and were not quotas applied to that or any folder.
I have tried the tool and everything reported as fine
I’ve tried backups and had the same error
I’ve deleted the DB’s to let them start from scratch and same issue.

Plex was working fine on the 22nd, I pushed the update offered for Plex by the App manager on the 23rd and it has since been doing this

16

How are your inodes df -i ?

17

Free

df -i
Filesystem         Inodes  IUsed      IFree IUse% Mounted on
rootfs              73238   2780      70458    4% /
tmpfs               79359      7      79352    1% /tmp
/dev/md0           131072   6751     124321    6% /volume0
/dev/loop0            544     17        527    4% /share
/dev/md1       1098358784 408545 1097950239    1% /volume1
cgroup              79359     11      79348    1% /sys/fs/cgroup
overlay        1098358784 408545 1097950239    1% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/1187b3fd21d689bd7e7a35d25eb142117922106b40c0184eacadaa0231c880e5/merged
overlay        1098358784 408545 1097950239    1% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/09ab5f65f9ece352b0e3204fe3a222fd8009017d8ceaa95b2c9bd5a937e59c99/merged
overlay        1098358784 408545 1097950239    1% /volume1/.@plugins/AppCentral/docker-ce/docker_lib/overlay2/833fc67b5b45a2f5e55ddcf0dbf2728c8678ba6187fea5df4fa56c231a724e1f/merged

I started plex again and no increase in the inode usage whilst it’s trying to startup, still the same behaviour

18

Do you have anything else we can check? I can’t see any reason for it to report as full

19

@chrisbarber252

I’ve asked my team member (who’s more familiar with ASUSTOR than I) to take a look.

My thought is: Move PMS to another drive and see if it works. That’s how I would test Synology but don’t know if you can do that with moving App data around on ASUSTOR

20

Unfortunately as it’s a 2 drive nas in Raid 1 I have no where to move it to.

I’ve completely wiped out all of the plex install files including dbs etc (not ideal) and setting it up fresh. no issues so far it’s currently performing the scan but will come back if the issue comes up again