DBRepair development

Hello,

This is likely out of the scope of this tool but it’s the best place I could think to ask and maybe someone knows of a tool or how to safely do this.

My DB still contains a bunch of items from an old deleted library. I currently have library IDs 1-5 and 9 in active use but in the DB itself, there’s still a lot of stuff with library ID 7.

I imagine I could delete the things that are listed with the ID directly but I’m sure there’s stuff elsewhere that would relate to it without the library ID. Is there a tool that can scrub that information from the DB properly or is there a known best way to do so?

Thanks in advance.

FYI…

DBRepair-v0dot7-dev.sh: 1661: Syntax error: end of file unexpected (expecting “fi”)

EDIT: Upon a very a quick look it appears the DoStop function has some problems with it.

Looking good.

Are you updating the Windows version too?

Thanks a lot!

@anon5074910

Thank you! fixed that and one you missed :slight_smile: LOL

@Yaracuy

What you currently see there is the Linux “Auto” mode.

The initial cut is based on the perceived desire of most Windows users (as I’ve seen) to “Click it, watch it run, and be done”, it seemed most logical to make “one size fits all” (preset ‘auto’ mode).

If anyone wants to help and/or show HOW to do menus and branching in Windows BAT files, I’m up for learning

2 Likes

Thanks!

My knowledge of Windows BAT files is very limited. Sorry :anguished:

I’ve run manually the current version of your Windows BAT and it works great. Maybe there is nothing to improve :smiley:

@Yaracuy

Maybe you can talk them into improving Windows ?

:rofl:

1 Like

ALL:

Here’s the roadmap I’ve laid out.

  1. This step (v0.7) is where the menus get better and I put in the underpinning for improving automation (auto mode is step 1)

  2. Next step (v0.8),
    – consider persistent settings for utility.
    – consider controlling script behavior based on ‘settings’
    – consider add new command ‘backup’ with backup saved to user-specified location.
    – consider restore external backups if PMS backups fail based on user settings

:point_up: #2 here is potentially messy and out of scope for a shell tool like this.

It might be time for python but that’s a whole new dependency I hadn’t planned on.
Also is the SH/BASH issue. Most have bash default now but some do have /bin/sh limitations. What’s best for everyone here?

1 Like

Folks,

I’m going to take a plunge, drive a stake into the ground and offer “v1.0.0 BETA 1” for your consideration.

I do this because it feels mature at this level of development

I am bumping to 1.0 because most of the code has been rewritten (consistent with software development rules about bumping major version number)

Documentation is coming along. The README is still rough but new content is better structured.

I expect cosmetic tweaks here and there.

I also expect tweaking docker container start/stop features. Please let me know/

Host Supported:

  1. Apple (MacOS)
  2. ASUSTOR
  3. Docker containers via ‘docker exec’ command (inside the running container environment)
    – Plex,inc.
    - Linuxserver.io
    - BINHEX
    - HOTIO
    - Podman (libgpod)
  4. Linux workstation & server
  5. Netgear (OS5 Linux-based systems)
  6. QNAP (QTS & QuTS)
  7. Synology (DSM 6 & DSM 7)
  8. Western Digital (OS5)

The menu

For clarity, each command’s name is ‘quoted’.

      Plex Media Server Database Repair Utility (_HostType_)
                       Version v1.0.0

  Select

      1 - 'stop' PMS (if available)
      2 - 'automatic' database check, repair/optimize, and reindex in one step.
      3 - 'check' database
      4 - 'vacuum' database
      5 - 'repair' / 'optimize' database
      6 - 'reindex' database
      7 - 'start' PMS (if available)
      8 - 'import' viewstate (Watch history) from another PMS database
      9 - 'replace' current database with newest usable backup copy (interactive)
     10 - 'show' logfile
     11 - 'status' of PMS (Stop/Run and databases)
     12 - 'undo' - Undo last successful command

     99 -  exit

  Enter command # -or- command name (4 char min) :

Use of Command Name is recommended. Command numbers will change as new capabilities are added.

Recommended usage:

sudo bash
./DBRepair.sh  stop auto status start exit
  1. Stop PMS
  2. Perform automatic maintenance
  3. Show status (PMS and DBs)
  4. Start PMS
  5. Exit

Internal changes

  1. Inline code restructured to functions
  2. New command line parser
  3. Each database operation now guarantees ample free space exists prior to starting.
  4. Each command checks PMS run-state prior to beginning (PMS might have restarted)
  5. Log file date-time stamp improved
  6. Scripted mode shows date/time stamp as each command executes

Sample session

bash-4.4# ./DBRepair.sh stop check vacuum reindex start exit
 
 
 
      Plex Media Server Database Repair Utility (Synology (DSM 7))
                       Version v1.0.0
 
 
[03.11.55] Stopping PMS.
[03.11.55] Stopped PMS.
 
[03.11.55] Checking the PMS databases
[03.13.30] Check complete.  PMS main database is OK.
[03.13.31] Check complete.  PMS blobs database is OK.
 
[03.13.31] Backing up databases
[03.13.31] Backup current databases with '-BKUP-2023-02-25_03.13.31' timestamp.
[03.13.38] Vacuuming main database
[03.14.31] Vacuuming main database successful (Size: 398MB/398MB).
[03.14.31] Vacuuming blobs database
[03.14.32] Vacuuming blobs database successful (Size: 1MB/1MB).
[03.14.32] Vacuum complete.
 
[03.14.33] Checking the PMS databases
[03.15.25] Check complete.  PMS main database is OK.
[03.15.25] Check complete.  PMS blobs database is OK.
[03.15.25] Backing up of databases
[03.15.25] Backup current databases with '-BKUP-2023-02-25_03.14.32' timestamp.
[03.15.30] Reindexing main database
[03.16.15] Reindexing main database successful.
[03.16.15] Reindexing blobs database
[03.16.15] Reindexing blobs database successful.
[03.16.15] Reindex complete.
 
[03.16.15] Starting PMS.
[03.16.29] Started PMS
 
bash-4.4# 
4 Likes

This is using it on my main DB

[chuck@glockner ~.2005]$ sudo ./DBRepair-beta.sh  stop auto start exit
 
 
 
      Plex Media Server Database Repair Utility (Ubuntu 20.04.5 LTS)
                       Version v1.0.0 - development
 
 
[19.36.06] Stopping PMS.
[19.36.06] Stopped PMS.
 
[19.36.06]  
[19.36.06] Checking the PMS databases
[19.36.45] Check complete.  PMS main database is OK.
[19.36.45] Check complete.  PMS blobs database is OK.
[19.36.45]  
[19.36.45] Exporting current databases using timestamp: 2023-02-25_19.36.06
[19.36.45] Exporting Main DB
[19.37.19] Exporting Blobs DB
[19.37.22] Successfully exported the main and blobs databases.  Proceeding to import into new databases.
[19.37.22] Importing Main DB.
[19.39.06] Importing Blobs DB.
[19.39.06] Successfully imported data from SQL files.
[19.39.06] Verifying databases integrity after importing.
[19.39.44] Verification complete.  PMS main database is OK.
[19.39.44] Verification complete.  PMS blobs database is OK.
[19.39.44] Saving current databases with '-BKUP-2023-02-25_19.36.06'
[19.39.44] Making imported databases active
[19.39.44] Import complete. Please check your library settings and contents for completeness.
[19.39.44] Recommend:  Scan Files and Refresh all metadata for each library section.
[19.39.44]  
[19.39.44] Backing up of databases
[19.39.44] Backup current databases with '-BKUP-2023-02-25_19.39.44' timestamp.
[19.39.45] Reindexing main database
[19.40.57] Reindexing main database successful.
[19.40.57] Reindexing blobs database
[19.40.57] Reindexing blobs database successful.
[19.40.57] Reindex complete.
[19.40.57] Automatic Check,Repair/optimize,Index successful.
 
[19.40.57] Starting PMS.
[19.40.57] Started PMS
 
[chuck@glockner ~.2006]$

Got a few cosmetic things to do.

2023-02-25 19.35.45 - ============================================================
2023-02-25 19.35.45 - Session start: Host is Ubuntu 20.04.5 LTS
2023-02-25 19.36.06 - ============================================================
2023-02-25 19.36.06 - Session start: Host is Ubuntu 20.04.5 LTS
2023-02-25 19.36.06 - Stop    - PASS
2023-02-25 19.36.45 - Check   - Check com.plexapp.plugins.library.db - PASS
2023-02-25 19.36.45 - Check   - Check com.plexapp.plugins.library.blobs.db - PASS
2023-02-25 19.36.45 - Check   - PASS
2023-02-25 19.37.22 - Repair  - Export databases - PASS
2023-02-25 19.39.06 - Repair  - Import - PASS
2023-02-25 19.39.44 - Repair  - Verify main database - PASS (Size: 1742MB/1680MB).
2023-02-25 19.39.44 - Repair  - Verify blobs database - PASS (Size: 1MB/1MB).
2023-02-25 19.39.44 - Repair  - Move files - PASS
2023-02-25 19.39.44 - Repair  - PASS
2023-02-25 19.39.44 - Repair  - PASS
2023-02-25 19.39.45 - Reindex - MakeBackup com.plexapp.plugins.library.db - PASS
2023-02-25 19.39.45 - Reindex - MakeBackup com.plexapp.plugins.library.blobs.db - PASS
2023-02-25 19.39.45 - Reindex - MakeBackup - PASS
2023-02-25 19.40.57 - Reindex - Reindex: com.plexapp.plugins.library.db - PASS
2023-02-25 19.40.57 - Reindex - Reindex: com.plexapp.plugins.library.blobs.db - PASS
2023-02-25 19.40.57 - Reindex - PASS
2023-02-25 19.40.57 - Reindex - PASS
2023-02-25 19.40.57 - Auto    - PASS
2023-02-25 19.40.57 - Start    - PASS
2023-02-25 19.40.58 - Exit    - Delete temp files.
2023-02-25 19.40.58 - Session end.
2023-02-25 19.40.58 - ============================================================
[chuck@glockner Databases.2011]$ 

Looks good based on my testing.

As I was curious yesterday I ran the following test …

  1. setup a new test plex server and added my movies, tv and music libraries turning off all heavy cpu scheduled tasks
  2. these libraries consist of 3,652 movies, 31,683 episodes and 79,570 tracks
  3. after initial scan, second force refresh of metadata, analyse of media and optimise of db the plex db was 501mb in size

Ran your tool and it took the db down to 459mb.
2023-02-25 11.05.04 - Repair - Verify main database - PASS (Size: 501MB/459MB).

So reduction of 8% just from a standing still starting point. Awesome :slight_smile:

One piece of feedback. Inside the DBRepair.log you use 2023-02-25 11.05.04 as the timestamp but on the console you use time [11.05.04]. Would you consider using the date & time also on console [2023-02-25 11.05.04]. I personally like seeing the full timestamp.

Thanks again!

This thing is getting all professional and stuff. Nifty!

@ChuckPa PM me (and @anon5074910 if he wants to chat too) I want to talk about a couple SQLite details with you. Wondering about the order of operations here.

1 Like

Beta 2

For your consideration.

Fixes:

  1. Not all temp files were located in Temp directory
  2. Importing Viewstate did not account correctly for DB’s Unique constraint
  3. Cosmetic cleanup of Console and Logs.
  4. Console and Logfile now use same full date/time stamp.

DBRepair-beta2.tar (60 KB)

3 Likes

I’m trying to run this on the Mac Mini and I’m pretty much tearing out my remaining hair. I’m getting “no such directory” in Terminal. That, and enabling administrator, which is not going as I expect.

1 Like

Same here, gave up after an hour.

Running Ventura 13.2.1 on Intel iMac, has anyone any ideas. No such directory…

1 Like

Ran into a similar problem on Windows when the Plex media server data directory was moved to a non-default location.

See below for where DBRepair is looking for things on a Mac.

  • Any chance you have PMS installed somewhere other than Applications?
  • Do the settings for AppSuppDir and DBDIR match up with your system?
    # Where is the software
    PLEX_SQLITE="/Applications/Plex Media Server.app/Contents/MacOS/Plex SQLite"
    AppSuppDir="$HOME/Library/Application Support"
    DBDIR="$AppSuppDir/Plex Media Server/Plug-in Support/Databases"
    PID_FILE="$DBDIR/dbtmp/plexmediaserver.pid"
    LOGFILE="$DBDIR/DBRepair.log"
    LOG_TOOL="logger"

I’m about ready to nuke the installation and start over, honestly. It’ll take far less time than trying to do this.

1 Like

So we keep things clean here, I’ve dropped my Mac Mini’s wake discussion from the thread.

I’m sure you understand the family wanting some private time.

:rofl:

I am working on the tool and will be addressing what we learned here tonight.

If I can get SSHD working, there’ll be life after death.

Edit: permission check resolved.

2 Likes

Corrections applied:

  1. Does not require “root” permission to run
    – Only needs R/W access to the databases

  2. Start / Stop requires root permission to work – (System level function)

  3. Other things I was looking at have been removed.

Possible issues:

  1. If PMS not in default location on MacOS, this is not handled.
    – Need more information about how implemented to accommodate

DBRepair-beta3.tar (60 KB)

1 Like

I finally gave up. Moved the database files elsewhere, started over. I was a bit baffled, to be honest — the main files were ~2.3Gb, which is about what the database files averaged in the far smaller server.

No issues starting the Mac server, either. So…my guess is that something screwed up the files. I should take a look at the TM backups and see how big they are.

FYI.

You beaut

Cheers