DBRepair development

I don’t have many other joys in life! :slight_smile: :slight_smile:

Re your first post there,

How did you create the DB? Did you set the page_size in the DB before loading it up?

Also, this is very precarious to get the validation correct.
I don’t want to drop /bin/sh compatibility if I can help it.

I was using speedtest1 which is included in the SQLite src. SQLite: speedtest1.c

It creates a fresh DB with the specific parameters each time you run it.

So I was changing pagesize and cachesize to make the table in that post:

rm testfile.db ; ./speedtest1 --size 200 --journal WAL --pagesize 1024 --cachesize 2000 testfile.db | grep TOTAL

I admit I didn’t try with a much larger “size”. Using --size 200 the DB was about 25MB.

Ok… I’ll go do some more experimenting.

Working on the math right now to verify power of two

[chuck@lizum ~.2008]$ a=40960
[chuck@lizum ~.2009]$ while [ $a -ge 2 ]
> do
> echo $a
> a=$((a / 2))
> done
40960
20480
10240
5120
2560
1280
640
320
160
80
40
20
10
5
2
[chuck@lizum ~.2010]$
1 Like

You really can just check for a few values, I think.
(I left 2048 out because it’s a silly choice.)

but how do you fix the value if incorrect ?
Select next higher ?

**EDIT: I figured out a way

Division loop until given value = 2
– while dividing if [ $value % 2 ] == 1 (then not a multiple)
– BARK AT USER :rofl:
– select a value higher than the given value from list of known safe values

2 Likes

Bark at the user about their poor life decisions? :slight_smile:

I would prefer a computer say “Hey that’s not valid” and exiting, or maybe “Hey that’s not valid so I’m using the default”.

I kinda get annoyed when they say “Hey that’s not valid so I’m using a different value I picked out.”

1 Like

Oh well. 1.03.01 time or just update and re-issue?

Documentation change:

Constraints:

  1. Must be a power of 2, with 1024 (2^10) as the Plex default. (per SQLite 3.12.0 documentation).
    Any invalid value provided will be rounded up to the next integral value (1024, 2048, 4096 … 65536)
    This may or may not be the value you intended. Caution is advised.

  2. May not exceed 65536 (per SQLite 3.12.0 documentation).
    Any value exceeding 65536 will be truncated to 65536.

@Volts

[chuck@lizum ~.2036]$ ./DBtest  0
DBtest 0
[chuck@lizum ~.2036]$ ./DBtest  1
DBtest 1
output: Setting Plex SQLite page size (1024)
writelog: SetPageSize - Setting Plex SQLite page_size: 1024
[chuck@lizum ~.2037]$ ./DBtest  65537
DBtest 65537
writelog: SetPageSize - DBREPAIR_PAGESIZE too large. Reducing.
output: Setting Plex SQLite page size (65536)
writelog: SetPageSize - Setting Plex SQLite page_size: 65536
[chuck@lizum ~.2038]$ ./DBtest  16383
DBtest 16383
output: Setting Plex SQLite page size (16384)
writelog: SetPageSize - Setting Plex SQLite page_size: 16384
[chuck@lizum ~.2039]$ ./DBtest  40960
DBtest 40960
output: SetPageSize - DBREPAIR_SIZE not a power of 2 between 1024 and 65536. Value selected = 65536.
writelog: SetPageSize - DBREPAIR_PAGESIZE not a power of 2. New value selected = 65536
output: Setting Plex SQLite page size (65536)
writelog: SetPageSize - Setting Plex SQLite page_size: 65536
[chuck@lizum ~.2040]$
1 Like

Here is the new error handling.

Please let me know how it looks

  • Both a non-1024 value and non-power of 2 after rounding up.
Checking the PMS databases
Check complete.  PMS main database is OK.
Check complete.  PMS blobs database is OK.

Exporting current databases using timestamp: 2024-01-17_21.02.19
Exporting Main DB
Exporting Blobs DB
Successfully exported the main and blobs databases.  Proceeding to import into new databases.
Importing Main DB.
WARNING: DBREPAIR_PAGESIZE (3071) not a multiple of 1024. New value = 3072.
ERROR: DBREPAIR_SIZE (3072) not a power of 2 between 1024 and 65536. Value selected = 4096.
Setting Plex SQLite page size (4096)
Importing Blobs DB.
WARNING: DBREPAIR_PAGESIZE (3071) not a multiple of 1024. New value = 3072.
ERROR: DBREPAIR_SIZE (3072) not a power of 2 between 1024 and 65536. Value selected = 4096.
Setting Plex SQLite page size (4096)
Successfully imported databases.
Verifying databases integrity after importing.
Verification complete.  PMS main database is OK.
Verification complete.  PMS blobs database is OK.
Saving current databases with '-BACKUP-2024-01-17_21.02.19'
Making repaired databases active
Repair complete. Please check your library settings and contents for completeness.
Recommend:  Scan Files and Refresh all metadata for each library section.

  • Not multiple of 1024 but rounding makes it compliant (1024)
Exporting current databases using timestamp: 2024-01-17_21.07.41
Exporting Main DB
Exporting Blobs DB
Successfully exported the main and blobs databases.  Proceeding to import into new databases.
Importing Main DB.
WARNING: DBREPAIR_PAGESIZE (1023) not a multiple of 1024. New value = 1024.
Setting Plex SQLite page size (1024)
Importing Blobs DB.
WARNING: DBREPAIR_PAGESIZE (1023) not a multiple of 1024. New value = 1024.
Setting Plex SQLite page size (1024)
Successfully imported databases.
Verifying databases integrity after importing.
Verification complete.  PMS main database is OK.
Verification complete.  PMS blobs database is OK.
Saving current databases with '-BACKUP-2024-01-17_21.07.41'
Making repaired databases active
Repair complete. Please check your library settings and contents for completeness.

Thanks to all of you who have no personal life for helping me :wink:

ALL:

https://github.com/ChuckPa/PlexDBRepair/releases/tag/v1.03.01

Please let me know how the updated purge performance is.
My testing shows it about 10x faster on a Synology DS418j (ARMv8)

1 Like

The “Prune” description is almost perfect now.
“Prune (remove) old image files (jpeg,jpg,png) from PhotoTranscoder cache”

I would suggest changing to:
“Prune (remove) image files (jpeg,jpg,png) older than 30 days from PhotoTranscoder cache”
Using the env variable for # of days (if set)

Should Prune be run before or after “2 - automatic”, or does it matter?

Doesn’t matter – separate data locations

1 Like

what did i do wrong? This ran fine on monday. Updated to the latest version this morning using the udpate feature in the utility and now get this.

Error relocating /lib/openhook.so: backtrace_symbols: symbol not found
Error relocating /lib/openhook.so: backtrace: symbol not found
Error relocating /lib/openhook.so: __syslog_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __snprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __asprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __strcpy_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __memset_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __fprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __sprintf_chk: symbol not found
[2024-01-18 10.19.13] Check complete. PMS main database is damaged.
Error relocating /lib/openhook.so: backtrace_symbols: symbol not found
Error relocating /lib/openhook.so: backtrace: symbol not found
Error relocating /lib/openhook.so: __syslog_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __snprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __asprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __strcpy_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __memset_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __fprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __sprintf_chk: symbol not found
[2024-01-18 10.19.13] Check complete. PMS blobs database is damaged.
[2024-01-18 10.19.13]
[2024-01-18 10.19.13] Exporting current databases using timestamp: 2024-01-18_10.19.13
[2024-01-18 10.19.13] Exporting Main DB
Error relocating /lib/openhook.so: backtrace_symbols: symbol not found
Error relocating /lib/openhook.so: backtrace: symbol not found
Error relocating /lib/openhook.so: __syslog_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __snprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __asprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __strcpy_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __memset_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __fprintf_chk: symbol not found
Error relocating /lib/libapparmor.so.1: __sprintf_chk: symbol not found
[2024-01-18 10.19.13] Error 127 from Plex SQLite while exporting com.plexapp.plugins.library.db
[2024-01-18 10.19.13] Could not successfully export the main database to repair it. Please try restoring a backup.
[2024-01-18 10.19.13] Repair failed. Automatic mode cannot continue. Please repair with individual commands

It’s in the README.md

@djfriday13

You got bit by the DSM apparmor update.

Near the bottom of the README, I show you how to run it from Task Scheduler now.

It’s a PITA (you need to run as yourself with ‘sudo’ instead of directly as root)

@djfriday13

I did it this way in the README. (you end up with /etc/sudoers as:)

chuck@ds418:~$ sudo tail -5 /etc/sudoers

# Include user-defined sudoers
#includedir /etc/sudoers.d

chuck ALL=(ALL) NOPASSWD: ALL
chuck@ds418:~$ 

gotcha. ya i guess after i updated then updated me schedule task to include prune i bet something happened.

trying to follow the read me i have a couple questions.

  1. the new user does it matter what priviledges that user has in dsm or are we fixing that with the script ran in step 2?
  2. the script being ran in step 2, can it be ran from any user or does it need to be ran by the newly created user?
  3. upon reboot of the nas, does any script need to be ran or will this setup continue to work after reboot?

If you create this one new Task in task scheduler, which you run when things break after DSM updates,

Screenshot from 2024-01-18 10-40-09

Script contents:

#!/bin/bash
#
# This script grants the given syno username (your username)
#  the ability to elevate to 'root' privilege for use with DBRepair.sh
#
# Set your Syno username here (no spaces)
MyUsername=chuck

# Confirm username exists
if [ "$(id $MyUsername)" = "" ]; then
  echo ERROR:  No such user \'$MyUsername\'
  exit 1
fi

# Remove old record 
sed -i s/^${MyUsername}.\*$// /etc/sudoers

# Add myself to sudoers
echo "$MyUsername" 'ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

This will persist until DSM updates again and wipes it out

Once you’ve set it up.

  1. Run DBRepair.sh task as the username you’ve assigned.
  2. In that DBRepair task, use sudo /path/to/DBRepair.sh stop auto purge start exit
    (or similar options)

Screenshot from 2024-01-18 10-45-55

Right, but I’m suggesting that the Prompt for “21” reflect what is set in the ENV variable . Just a minor comment.