Ever since the 1.25.xxx builds, I’ve been experiencing server crashing daily and multiple times a day. I’m getting really tired of family members complaining that the Plex is down… again. Rebooting the docker brings everything back up, but this is getting ridiculous.
Looking at the docker image logs for Plex, I’m seeing a ton of: Sqlite3: Sleeping for 200ms to retry busy DB spam until the connection to the server is closed out.
I’m running out of ideas on how to resolve this issue.
Database is on an NVMe SSD drive with plenty of room available on the drive.
The system RAM (32GB) is good as well.
Clients range from web browsers, Roku, game consoles, and smart TV apps.
The only new element to the server has been drive upgrades one at a one, and those drives have tested good after 72 hours of zeroing.
I’m attaching my logs and hope someone can help as Plex has worked like a champ for many years!
In the logs you posted before, I see shutdowns at these times:
Plex Media Server.1.log:Mar 06, 2022 22:06:37.450 [0x14c80e949b38] DEBUG - Shutting down with signal 15 (Terminated)
Plex Media Server.4.log:Mar 06, 2022 15:04:17.606 [0x153fbe247b38] DEBUG - Shutting down with signal 15 (Terminated)
Signal 15 is a normal “System asked me to stop” message, implying that something else on the system requested Plex stop.
Does Docker (or Unraid) provide any logging about these events?
I also see a few of these:
Plex Media Server.5.log:Mar 06, 2022 02:38:26.333 [0x153fb7e97b38] ERROR - Thread: Uncaught exception running async task which was spawned by thread 0x153fbe2c9b38: sqlite3_statement_backend::loadOne: database is locked
What storage is this on? And what version of Unraid? Unraid has a history of filesystem damage where the FUSE-based filesystem was unsafe for databases.
One workaround was to move the Plex data directory to a cache disk, bypassing the FUSE/mergerfs problems.
Unraid 6.9.2
The Plex data directory is on an NVMe 256GB cache drive.
The Docker logs are pretty vanilla unfortunately:
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 40-chown-files: executing...
[cont-init.d] 40-chown-files: exited 0.
[cont-init.d] 45-plex-claim: executing...
[cont-init.d] 45-plex-claim: exited 0.
[cont-init.d] 50-gid-video: executing...
[cont-init.d] 50-gid-video: exited 0.
[cont-init.d] 60-plex-update: executing...
No update required
[cont-init.d] 60-plex-update: exited 0.
[cont-init.d] 90-custom-folders: executing...
[cont-init.d] 90-custom-folders: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
Starting Plex Media Server.
[services.d] done.
Sqlite3: Sleeping for 200ms to retry busy DB.
Connection to 184.105.148.81 closed by remote host.
Connection to 184.105.148.88 closed by remote host.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Sqlite3: Sleeping for 200ms to retry busy DB.
Connection to 184.105.148.89 closed by remote host.
Connection to 184.105.148.88 closed by remote host.
Sqlite3: Sleeping for 200ms to retry Dolby, Dolby Digital, Dolby Digital Plus, Dolby TrueHD and the double D symbol are trademarks of Dolby Laboratories.
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 01-migrations: executing...
[migrations] started
[migrations] no migrations found
[cont-init.d] 01-migrations: exited 0.
[cont-init.d] 02-tamper-check: executing...
[cont-init.d] 02-tamper-check: exited 0.
[cont-init.d] 10-adduser: executing...
usermod: no changes
Should I move the existing /plex folder in /appdata over to the newly created appdata folder on the cache drive? Or should I use another method to retain the database?
To few details - need to know if this is split on your site:
Do you use a physical cache/pool (SSD, M.2 NVMe) for system data like appdata? What’s the host path? /mnt/user/appdata/plex?
Do you use the Unraid array for your content? What’s the host path? E.g. /mnt/user/Music?
What are your Plex container path mappings? I doubt that Appdata:/mnt/user/appdata/plex is the only path mapping. What are the mappings for your content (music)?
Usually there is no need to move things around. Path mappings are there to avoid just that.
I do use a physical cache drive ((SSD, M.2 NVMe) for Plex (appdata) and the other 4 Dockers I have running. I do my transcodes in the system memory, which there is plenty of (32GB)
Nothing has changed for years as far as my software configuration. Just the hardware has changed with hard drive size bumps and the growing size my very large library.
Here is my pathing as requested.
I am leaning towards the issue being with Plex and the use of SQLite.
Does anyone from the Plex Development Team know if there is plans or any testing going on with using a different database other than SQLite? I’d love to test if there build available outside of the public beta channel.
I assume cache is really on cache and not on a pool and /mnt/cache/appdata will work:
Please change the /mnt/user/appdata/… value to /mnt/cache/appdata/… then and hit Apply. Thats all. No need to move things around. As you are using Media:/mnt for your content this won’t hurt. It’s a test that may help. It does avoid FUSE for the SQLite DB. This combination showed some problems in the past.
Your container config is nearly identical to mine. I’m using 6.9.2 and Linuxservers Plex container. My DB is over 2 GB in size and rock solid. There might be a difference: I stop all containers every night, take backups and restart them. I use my own scripts in the User Scripts plugin for that.
In fact I use Unraid completely without User Shares - just Disk Shares.
It has some nice effects when using more than a handful of disks. Just change within Plex Libraries from e.g. /mnt/user/Movies to /mnt/disk1/Movies, /mnt/disk2/Movies and so on.
Plex can handle that - no problem at all. But the effect is: Plex addresses an individual disk and not the FUSE layer. Depending on your memory and cached directories other disks may spun up just to find out where the requested content is. This does not happen when using Disk Shares.
Very interesting and thanks for the tip HTPC_Lover!
This seriously should be a huge notice or important note flagged for all the Plex Docker images in my opinion. All the research into this crashing and I totally did not come across this FUSE
layer clash information.
What I wrote is based on my own experience. I’m using server-grade hardware and RAM size. Not the latest motherboard, CPU, iGPU, …
My guess is: Many problems show up on the latest hardware, low RAM, desktop-grade hardware (Laptop?) to drive server-grade features (KVM?). And not to forget: BTRFS - I avoid it whereever possible …
