Plex Web App - Very Slow Dashboard Loading - Anyone Else Experiencing or Know Why?

General Discussions
1

Problem

Within the last week, perhaps two (and roughly coinciding with upgrading my server to the 1.9.x series, though I am not sure if it is in any way related) the Plex Web App’s Dashboard has begun loading horribly slowly and erratically, often taking over 30 seconds for all the columns to display. There are also a number of JavaScript console errors that are triggered while it is fetching the different data sources and performing other operations.

Video

I recorded a screencast of my Chrome browser window (with the developer console open to show activity and errors) when loading app.plex.tv/desktop, which can be viewed at youtu.be/o2MhQANd7fE. This showcases the problem described above.

Environment

I’m not positive what environment information would be most important so I’ve provided details about the server and client systems and browser. If any additional information would be helpful, please let me know and I will provide it.

System (Server, Headless)

Make:          Custom Build
OS:            Linux (Ubuntu)
Version:       17.04 (zesty)
Kernel:        4.10.0-35.39
CPU:           Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
Cores/Threads: 4/8
Memory:        32GB DDR4 (2400MHz) [4 x 8GB]
Graphics:      None                   (Dedicated)
Graphics:      Intel HD Graphics P530 (Integrated)
Plex Server:   1.9.1.4272-b207937f1
Filesystem:    Btrfs [Samsung SSD 256GB]                     (System, Plex Databases)
Filesystem:    ZFS   [Western Digital HDD 6TBx9 30TB Raidz2] (Storage, Media, Plex Metadata)
Mounts: ...
  /dev/sdi2            on /     type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@)
  /dev/sdi1            on /boot type ext4  (rw,relatime,data=ordered)
  /dev/sdi4            on /home type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@home)
  pool                 on /pool                    type zfs (rw,relatime,xattr,noacl)
  pool/backup          on /pool/backup             type zfs (rw,xattr,noacl)
  pool/download        on /pool/download           type zfs (rw,xattr,noacl)
  pool/media           on /pool/media              type zfs (rw,noatime,xattr,noacl)
  pool/opt             on /pool/opt                type zfs (rw,xattr,noacl)
  pool/projects        on /pool/projects           type zfs (rw,xattr,noacl)
  pool/torrent         on /pool/torrent            type zfs (rw,xattr,noacl)
  pool/plexmediaserver on /var/lib/plexmediaserver type zfs (rw,relatime,xattr,noacl)
  /dev/sdi4            on /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support type btrfs (rw,relatime,ssd,space_cache,subvolid=282,subvol=/@plex-plugin-support)

System (Client, Laptop)

Make:          Lenovo Thinkpad P70
OS:            Linux (Ubuntu)
Version:       17.04 (zesty)
Kernel:        4.10.0-35.39
CPU:           Intel(R) Xeon(R) E3-1505M v5 CPU @ 2.80GHz
Cores/Threads: 4/8
Memory:        32GB ECC DDR4 (2133MHz) [2 x 16GB]
Graphics:      NVIDIA GM204GLM Quadro M4000M 4GB [Driver: 381.22] (Dedicated)
Graphics:      Intel HD Graphics P530                             (Integrated)
Filesystems:   Btrfs [Samsung NVMe SSD 512GB] (System [Primary, Linux])
Filesystems:   NTFS  [SanDisk SSD 1TB]        (System [Secondary, Windows])
Filesystems:   Btrfs [Samsung NVMe SSD 256GB] (Storage)
Filesystems:   Btrfs [Seagate SSHD 2TB]       (Storage)

Browser (Client)

Name:       Chrome Browser
Version:    63.0.3218.0 (Official Build) dev (64-bit)
Revision:   8634c0eac009323c45c362e9308429d8f1e1976b-refs/heads/master@{#502516}
OS:         Linux
JavaScript: V8 6.3.157
Flash:      27.0.0.134
2

Would you please do the following:

  1. Stop PMS
  2. Start PMS
  3. Open your browser to the dashboard
  4. Settings - Server - Help - Download Logs
  5. Post the ZIP file it gives you here. I will take a look
3

@ChuckPA Thanks for the quick response! I’ve included a copy of the server logs zip for your review.

4

Thank you.

As suspected, your PMS internal database is severely fragmented.

Sep 22, 2017 00:39:07.334 [0x7f5b417fe700] WARN - SLOW QUERY: It took 240.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:07.416 [0x7f5b26bfb700] WARN - SLOW QUERY: It took 220.000000 ms to retrieve 75 items.
Sep 22, 2017 00:39:07.417 [0x7f5b343fe700] WARN - SLOW QUERY: It took 250.000000 ms to retrieve 75 items.
Sep 22, 2017 00:39:07.957 [0x7f5b263fa700] WARN - SLOW QUERY: It took 220.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:08.003 [0x7f5b217fe700] WARN - SLOW QUERY: It took 260.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:08.066 [0x7f5b1ffff700] WARN - SLOW QUERY: It took 300.000000 ms to retrieve 75 items.
Sep 22, 2017 00:39:08.133 [0x7f5b417fe700] WARN - SLOW QUERY: It took 270.000000 ms to retrieve 75 items.
Sep 22, 2017 00:39:09.777 [0x7f5b4c3ff700] WARN - SLOW QUERY: It took 250.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:09.824 [0x7f5b26bfb700] WARN - SLOW QUERY: It took 400.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:09.857 [0x7f5b25bf9700] WARN - SLOW QUERY: It took 300.000000 ms to retrieve 40 items.
Sep 22, 2017 00:39:10.332 [0x7f5b283fe700] WARN - SLOW QUERY: It took 400.000000 ms to retrieve 1 items.
Sep 22, 2017 00:39:10.714 [0x7f5b4c3ff700] WARN - SLOW QUERY: It took 220.000000 ms to retrieve 15 items.

The steps to remedy and prevent this are:

  1. Hover over Libraries in the left pane to expose the ellipsis .
  2. Click it
  3. Click “Optimize Database” (This cleans up the DB)
  4. Go to Settings - Server - Scheduled Tasks
  5. Enable the task to optimize the database every week.
5

@ChuckPA I actually already have that enabled in my settings and have since I installed PMS originally on this machine. I went ahead and manually ran the optimization regardless, but this hasn’t resolved my issue.

The way I currently have everything set up, the main PMS home directory is stored on the massive ZFS storage partition, which is relatively slow, but the database files themselves are stored on my SSD. Here is a listing of the relivant mounts:

/dev/sdi2            on /     type btrfs (rw,relatime,ssd,space_cache,subvol=/@)
/dev/sdi1            on /boot type ext4  (rw,relatime,data=ordered)
/dev/sdi4            on /home type btrfs (rw,relatime,ssd,space_cache,subvol=/@home)
pool/media           on /pool/media              type zfs (rw,noatime,xattr,noacl)
pool/plexmediaserver on /var/lib/plexmediaserver type zfs (rw,relatime,xattr,noacl)
/dev/sdi4            on /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support type btrfs (rw,relatime,ssd,space_cache,subvol=/@plex-plugin-support)

The /dev/sdi device is a 256GB Samsung 850 Pro SSD. As you can see above, the

/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support

path containing the database files is mounted to the SSD, whereas everything else in /var/lib/plexmediaserver is on the slower disk media.

Are there other files in /var/lib/plexmediaserver that I should move to the SSD as well? I can’t place the entire path on the SSD as, with the thumbnails and other metadata, it is too large.

Thoughts? Thanks for any recommendations.

6

How large is your media library in total items ? (a find blah-blah -type f -print | wc -l will count that quickly)

If you navigate to /var/lib/plexmediaserver/Libary/Application Support/Plex Media Server/Plug-in Support/Databases, you can access the db itself and perform a count also. Doing so will tell us exactly how many records are in it ( one possible source of the slowness – list length )

sudo sqlite3 com.plexapp.plugins.library.db
select count(*) from media_streams;
select count(*) from media_items;

I am suggesting checking this because SQLite3 is an indexed sequential file with a sql front end. It’s not a true relational database.

Knowing how much raw data must be managed is the next step to take

7

@robfrawley said:
The way I currently have everything set up, the main PMS home directory is stored on the massive ZFS storage partition, which is relatively slow,

The /dev/sdi device is a 256GB Samsung 850 Pro SSD. As you can see above, the

/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support

path containing the database files is mounted to the SSD, whereas everything else in /var/lib/plexmediaserver is on the slower disk media.

IMHO it is no wonder the page builds relatively slow, considering that the database file doesn’t contain the pictures.
These must be collected from your “relatively slow” ZFS storage partition.

8

@OttoKerner The pictures and other media metadata shouldn’t affect the SQL queries and wouldn’t have any bearing on the SLOW QUERY warnings, AFAIK. I wouldn’t be surprised or concerned if thumbnails and other media loaded slowly (which it sometimes does), but I am concerned when the actual text and relations data (retrieved from the SQLite databases) causes page loads to hang and the server starts logging SLOW QUERY warnings. From what I can tell, the Plex web application behaves similarly to any Ajax application I’ve written: the images are handled independently of any text metadata retrievals. Are these assumptions incorrect?

@ChuckPA I’ve gone ahead and run the SQLite queries you suggested; I’ve also provided some additional information about the disk space used by different media types.

SQLite Query Results

The following are the SQLite query results…

select count(*) from media_streams;

Result: 89,113

select count(*) from media_items;

Result: 42,307

File Count for Media

The following is the number of files per media type…

find /pool/media/video/tv -type f -print | wc -l

Number of TV Episodes: 8,063

find /pool/media/video/tv-recorded -type f -print | wc -l

Number of DVR TV Recordings: 581

find /pool/media/video/movie -type f -print | wc -l

Number of Movie Titles: 1,152

find /pool/media/music -type f -print | wc -l

Number of Music Files: 9,909

File Size for Media

The following are the different media group sizes…

du -h --max-depth=0 /pool/media/video/movie

Size of Movie Titles: 2.0TB

du -h --max-depth=0 /pool/media/video/tv

Size of TV Episodes: 4.7TB

du -h --max-depth=0 /pool/media/video/tv-recorded

Size of DVR TV Recordings: 1.2TB

du -h --max-depth=0 /pool/media/music

Size of Music Files: 90GB

And finally, here is the output of zfs list which describes the mount point sizes:

# sudo zfs list
pool                  27.5T  8.73T  3.05M  /pool
pool/backup           19.1T  8.73T  19.1T  /pool/backup
pool/download         17.0G  8.73T  17.0G  /pool/download
pool/media            8.13T  8.73T  8.10T  /pool/media
pool/opt               107M  8.73T   107M  /pool/opt
pool/plexmediaserver   149G  8.73T   147G  legacy
pool/projects         7.36G  8.73T  7.36G  /pool/projects
9

Rob,
Thanks for that info. It’s as I suspected you’d find. It’s not the disk i/o for the images. It truly is the DB query time walking down those tables.

I’m going to take this and ask the DB guys how to proceed next with this.

10

@ChuckPA Thanks; I look forward to any information the DB guys may forward you.

As a side note, I would absolutely love for Plex to support multiple database backends, as many other applications do. That way, for example, SQLite could be used by default (an appropriate choice for most users), and something like MySQL or PostgreSQL could be selected for those that require it. Is anything like this on a feature request list already? Or even possibly in the development map? Just curious.

11

That’s a standing feature request. If anything, I would suspect Mariadb because it’s the public fork of mysql.

You know , where it gets tricky is in the DB abstraction layer. I don’t know what is done there but <shrug> maybe?

12

I started using MariaDB over MySQL a few years back, and ever since trying it, MariaDB has become my default choice for all web-applications I create for clients (I’m a freelance developer). First, because it works great and performs well, and second because I’ve grown to distrust Oracle, the current shepherd for MySQL. Glad to hear they are looking into it, at the very least!

13

@ChuckPA Any update from the DB guys on how to proceed moving forward? Am still experiencing the issue.

14

@robfrawley - ChuckPA asked me to take a look at your issue. I don’t believe it is a DB issue. The slow query messages ChuckPA pointed out earlier were not for your main database, but for one of the other supporting databases that Plex uses, which is still on your ZFS drive.

From what I see, it just appears that your system is slow delivering all the images it needs from your ZFS drive. If you moved everything over to your SSD drive, things would probably be faster.