Plex web slow item loading

Changing posters on this brand new install doesn’t affect loading times as significantly as it did on the older one however the time for the XML to load went from <100ms to 1.5s. So still a large increase relatively.

The only change between refreshing the XML page was updating the poster.

I’m happy to forward this DB as well, if it helps.

I am recreating now.

I need a controlled test bed

1 Like

Thanks for the update @ChuckPa

I checked the taggings table for this newly scanned in database and it doesn’t seem like the tag_id is static. On this database the tag_id for the artwork is 107225 and 107224.

So 100 and 101 don’t seem to mean anything in particular.

Reporting progress so far.
This will take better part of a day to complete given USB3 SSD for Database and NFS. :frowning:

root@testbed:Databases# date
Tue Oct 24 05:27:46 UTC 2023
root@testbed:Databases# sqlite3 com.plexapp.plugins.library.db 'select count(*) from media_parts;'
366618
root@testbed:Databases# date
Tue Oct 24 05:27:52 UTC 2023
root@testbed:Databases#  ll -h
total 1.3G
drwxr-xr-x 2 plex plex 4.0K Oct 23 21:50 ./
drwxr-xr-x 7 plex plex   96 Oct 23 20:05 ../
-rw-r--r-- 1 root root 1.5K Oct 23 21:50 DBRepair.log
-rw-rw-r-- 1 plex plex 1.1M Oct 23 21:50 com.plexapp.plugins.library.blobs.db
-rw-rw-r-- 1 plex plex  32K Oct 23 21:50 com.plexapp.plugins.library.blobs.db-shm
-rw-rw-r-- 1 plex plex    0 Oct 23 21:50 com.plexapp.plugins.library.blobs.db-wal
-rw-rw-r-- 1 plex plex 902M Oct 24 05:33 com.plexapp.plugins.library.db
-rw-rw-r-- 1 plex plex 672K Oct 24 05:33 com.plexapp.plugins.library.db-shm
-rw-rw-r-- 1 plex plex 333M Oct 24 05:33 com.plexapp.plugins.library.db-wal
root@testbed:Databases#

At this time,

  1. All movies are loaded (9539)
  2. Series loaded = 3267

Plex/web page load time, using i7-8809G (NUC8) = 3-4 seconds.

The database loading is complete.

Test system loaded by:

  1. extracting file from media_parts in DB provided by @carrousel5301
  2. Dummy files created to satisfy scanner and input processing
  3. New PMS server created using the pathnames extracted, folded into Movies and TV sections

Here are the results:

  1. Back to the Future → Get Info → View XML – 3 seconds maximum.

( I would show this video but this information requires @carrousel5301 's permission. I consider the filenames and structure to be Personal Info )

  1. Physical DB files
root@testbed:Databases# ls -lah
total 1.1G
drwxr-xr-x 3 plex plex 4.0K Oct 24 17:13 .
drwxr-xr-x 7 plex plex   96 Oct 23 20:05 ..
-rw-r--r-- 1 root root 2.9K Oct 24 17:13 DBRepair.log
-rw-rw-r-- 1 plex plex 1.1M Oct 24 17:13 com.plexapp.plugins.library.blobs.db
-rw-rw-r-- 1 plex plex  32K Oct 24 17:13 com.plexapp.plugins.library.blobs.db-shm
-rw-rw-r-- 1 plex plex    0 Oct 24 17:13 com.plexapp.plugins.library.blobs.db-wal
-rw-rw-r-- 1 plex plex 1.1G Oct 24 21:36 com.plexapp.plugins.library.db
-rw-rw-r-- 1 plex plex  64K Oct 24 21:54 com.plexapp.plugins.library.db-shm
-rw-rw-r-- 1 plex plex  18M Oct 24 21:54 com.plexapp.plugins.library.db-wal
drwxr-xr-x 2 root root 4.0K Oct 24 17:12 dbtmp
root@testbed:Databases# sqlite3 com.plexapp.plugins.library.db 'select count(*) from media_parts;'
436867
root@testbed:Databases# date
Tue Oct 24 21:55:06 UTC 2023
root@testbed:Databases#
  1. General responsiveness of Plex/web
    ( All metadata & DB on USB3 SSD )

Discussion points to consider:

  1. Resultant size of the DB using default settings for everything
  2. PMS DB damage from 3rd party tools ?

Is there any answer as to why just changing a simple poster is making the file basically unplayable? Carousels example above shows what it’s doing in the background on just a simple poster change from the Edit screen.

In PM discussion with @carrousel5301 ,

  1. I have permission to share the pathnames
  2. I will also replicate a condition asserted to cause slowness.

I just replicated changing a single poster makes things get funny…
but only on selected items.

I changed Toy Story 3 without issue.
I changed Toy Story 4 and it went stupid.

I’m running this down now

1 Like

I would like to ask folks for assistance.

In my container, I observe two scanner processes running.
There should be one.

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                  
 243799 plex      20   0   79868  39544  23420 R  98.0   0.1   0:10.78 Plex Media Scan                                          
 243940 plex      20   0   78248  41540  27172 R  18.4   0.1   0:00.09 Plex Media Scan                                          
 203712 plex      20   0  730540 511224  37156 S  12.2   0.8 136:22.94 Plex Media Serv                                          
      1 root      20   0  167236  12932   8548 S   0.0   0.0   0:02.12 systemd        

When there are none, everything works ok.
When there’s one, there’s a slight pause opening to change the poster.
When there are two, it gets slow.

Before I consider this an anomaly of this system being less than 24 hours old, can anyone replicate (observe) this ?

Also, If you wait, you will get one of two results:

  1. It will eventually complete without error and goes back to normal
    -or-
  2. “There was an error editing…”

I do see 2 scan processes when actively scanning a library but that is the only time.

ps aux | grep /usr/lib/plexmediaserver/Plex
root         298 28.1  0.1 205520 100660 ?       Ssl  12:38   0:27 /usr/lib/plexmediaserver/Plex Media Server
root         369  0.0  0.0  40284 11792 ?        Sl   12:38   0:00 /usr/lib/plexmediaserver/Plex Tuner Service /usr/lib/plexmediaserver/Resources/Tuner/Private /usr/lib/plexmediaserver/Resources/Tuner/Shared 1.31.2.6810-a607d384f 32600
root        1156  0.0  0.1 127388 75236 ?        RNl  12:39   0:00 /usr/lib/plexmediaserver/Plex Media Scanner --analyze --log-file-suffix  Analysis --no-thumbs --item 30244
root        1159  0.0  0.0  91940 39532 ?        DNl  12:39   0:00 /usr/lib/plexmediaserver/Plex Media Scanner --match --type 1 --log-file-suffix  Matcher --files /movies-4k/Tag (2018)...

thanks… Just wanting to confirm what I see.
Still working on this one.

There is definitely a “size limit” issue somewhere. That’s what I’m trying to resolve.

I have a 998 series test machine which does not exhibit this problem.
(about 200,000 episodes)

well, but where is the size limit? I mean I have a library of only 250 movies and it is also slow/exhibits slow loading… Again if you also need my DB - I´m happy to provide it. :slight_smile:

Thank you.

250 movies + ??? TV episodes ?

There is something going on – that’s what I’m trying to find.

– Naming ?
– How it’s built ?
– References between library sections?
– Multiple versions of the file?

1 Like

What is the purpose of the additional artwork urls in the taggings table for movie items? Why does deleting them resolve the problem?

This issue has only ever affected the Plex Movie Agent, the TV agent is fine and so is the legacy movie agent.

In previous experiments I only scanned in my movie libraries and the issue persisted.

250 movies in one library and 920(ish) in a different library, that do have “references” between them. One lib is a lib containing 4K versions of the movies and the 2K files are in the other lib.

Not many TV shows, some other libs only contain trailers and other vids. (that should not have any cross-references)

I don’t know why they put things in there. I’m not a DB/Server expert on the design.

If this can be confined to the Plex Movie Agent, and recreated with a small DB.
(My initial test of 20 movies did not recreate the problem) then we have something for the engineer to work with.

I’ve looked at tag counts in individual movies so far. There is no obvious correlation.

My focus is to figure out how to easily recreate the problem with the smallest dataset and submit it for eval & resolution.

1 Like

There is a direct correlation to modifying a poster, those fields getting added to the taggings table and the movie not taking 15+ seconds to play.

If I manually delete those extra fields for a “broken” movie playback starts instantly.

Like in this post - Plex web slow item loading - #37 by carrousel5301

For instance, on my database where nearly every movie was non-functional running this sqlite fixed the issue immediately.

sqlite3 com.plexapp.plugins.library.db "DELETE FROM taggings WHERE tag_id IN (SELECT tag_id FROM taggings WHERE text LIKE '%image.tmdb.org%' OR text LIKE '%artworks.thetvdb.com%' OR text LIKE '%metadata-static.plex.tv%');"

The same amount of episodes are in this database. The ONLY difference is removing the excess entries in the taggings table.

I suppose this might only present itself on a somewhat large library, as even with dozens of useless rows that table is still relatively small with only 20 items.

Thank you for doing that deep dive into the DB.

IMHO, this is definitely excessive for your database HOWEVER given th

root@testbed:Databases# sqlite3 com.plexapp.plugins.library.db 'select count(*) from taggings;'
2868796
root@testbed:Databases# 

I’m going to see what happens when I clear table taggings.
This is great info. thank you.

1 Like

Just as a further example, using Toy Story 4 again as an example. There are over 160 different art URL’s for various backgrounds etc. (This is AFTER changing the poster in the UI).

Toy Story 4 will take 15+ seconds to load, if it does at all.

If you delete these 160 art entries, nothing changes with the item visually but playback is nearly instant.

Here’s the contents of taggings for the metadata_item_id 366205. None of these seem neccesary. Deleting them causes no apparent issues and restores load times.

Toy Story 4 would load most of the time.
Other times, “There was an error editing…”

In looking at the DB, not sure if it’s how I created it or not, the tag_id is 165.

When I run counts on this database:

select count(*) FROM taggings WHERE tag_id IN (SELECT tag_id FROM taggings WHERE text LIKE '%image.tmdb.org%' OR text LIKE '%artworks.thetvdb.com%' OR text LIKE '%metadata-static.plex.tv%');
251004
sqlite>

That’s 10% of the taggings table total count.

I have taken this info to engineering and asked for more guidance on how to isolate further for them OR if what I have here is enough