Regeneration of `intro markers` after moving plex media server from one System to Another

Plex Media Server Desktops & Laptops
1

Server Version#: Version 1.22.0.4163 (Fedora 33 64bit)
Player Version#: Version 4.53.0

I moved by plex media server from one dedicated hardware system to another over the weekend. Overall process was fine, I’ve done this may times over my years of using plex so am very familiar with this steps. The process I use for this is → https://support.plex.tv/articles/201370363-move-an-install-to-another-system

My OS remains the same between source and target system but the one thing I did change was the location of the mount points for my media. I moved it from /mnt/xxx mounts to /data/xxx mounts. Permissions etc… are all the same.

As per the doc I added the new mounts to the libraries (doing one library at a time) and then let the system scan, pickup the new locations etc… Once complete empty trash, optimise db etc… all done and no errors. Again, as I say all was well and worked flawlessly, on deck, watched content, history etc… all came over okay and as I expected and my users watched content after I completed the move.

Now to the wrinkle in the sand… I typically check my logs daily to see what background task fired overnight and what might have failed and one of the checks I do is for intro markers and it reported …

[mcurran@dokuro Scripts]$ grep “intro markers” /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Logs/Plex\ Media\ Server*.log
/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Logs/Plex Media Server.log:Mar 14, 2021 07:00:42.347 [0x7f5bd636e640] DEBUG - The butler generated 316 season intro markers
[mcurran@dokuro Scripts]$

You can see that 316 intro markers was regenerated which really surprised me. These were generated previously and should have been brought over to the new server as part of the restore (I restored everything under /var/lib/plexmediaserver/Library when moving to the new server. BTW, chapter thumbnails seem to be the same and are being regenerated.

This looks like a bug so wanted to create a thread for it. In the bigger scheme of things for me the system will eventually complete all this re-work but really it would be nice to not have to do this again, especially for intros as this is a hugely intensive task on the server.

Plex Media Server Logs_2021-03-14_10-26-07.zip (4.0 MB)

1 Like
2

I think I potentially could have a problem which I need someone from the plex team to confirm.

My com.plexapp.plugins.library.blobs.db was approx 905M before I moved to my new server and now is 1.5GB and still growing. Intro detection is still in progress but I’m worried I’m getting duplicate entries due to a bug after moving my system. Is there anyway to confirm ?

3

Because you change the paths, the media_parts table was effectively replaced with different entries. The intro detection uses data stored at the part level and since you now have completely new parts, it has to regenerate them. The blobs database you pointed out is where some of this is stored.

1 Like
4

Thanks for getting back to me.

Okay, so that explains why intro detection is being regenerated but how about the size ? Will there be clean-up done after these scheduled tasks complete ? It kinda looks like its being added rather than replaced.

5

When you empty the trash of the old parts, it well clean up the data associated with the old files.

6

Mmmm, I’d done that already.

7

@gbooker02 as an fyi… I decided to bite the bullet and nuke my plex setup and start from scratch. I know I will lose my play history, on deck and unplayed settings but I can manually restore them. I’d just be more happy to resolve this cleanly than hoping the com.plexapp.plugins.library.blobs.db would clean up.

Once all the background tasks fully complete, intro, chapters etc… I’ll report back what size my db’s ended up at after a clean install. This should take approx 1 week based on the amount of my content. Might be a good data point to have for reference.

8

You could simply keep the com.plexapp.plugins.library.db file and blow away the rest. That way at least most “played” statistics will survive.

9

Mmmm, that’s interesting.

So could I basically delete everything under /var/lib/plexmediaserver/Library – startup plex, claim server, stop plex and then restore my com.plexapp.plugins.library.db only and start plex again. Then do a scan, refresh data etc… (all the usual plex stuff).

10

Yep.
It will retain your library setup and the items in there. But many of the metadata like thumbnails and loudness info will be lost.

1 Like
11

Thanks. I may have asked this before but I cannot find it if I did. Anyhow, is there a way to query how much is left do for the specific background scheduled tasks ?

Example, I can see --analyze-deeply is currently running and has been for a number of hours. Is there a way for me to check that x is complete and y is left to be done from a total of z ? I can then estimate how long it may take to complete on my hardware.

Same question for the tasks such as intro detection and chapter thumbnails.

12

You can try this: bif.stats.tar.gz (840 Bytes) to view progress of generating the video preview thumbs in a terminal window.
This is by far the slowest of the three, so it will be most significant.

For intro scan, try this SQL query

select distinct(metadata_items.parent_id) from metadata_items join media_items on metadata_items.id=media_items.metadata_item_id join media_parts on media_items.id=media_parts.media_item_id where metadata_items.metadata_type=4 and media_parts.extra_data not like '%pv^%3Aintros=%' escape '^' and metadata_items.id not in ( select distinct(metadata_items.id) from metadata_items join media_items on metadata_items.id=media_items.metadata_item_id join media_parts on media_items.id=media_parts.media_item_id where media_parts.extra_data like '%pv^%3Aintros=%' escape '^' ) order by metadata_items.added_at

it will return the seasons which have no intros detected (yet).
Keep in mind that not all seasons might have detectable intros (e.g. specials season with OVA’s etc.)

The following will return just a number, if that is all you want:

select count(distinct(metadata_items.parent_id)) from metadata_items join media_items on metadata_items.id=media_items.metadata_item_id join media_parts on media_items.id=media_parts.media_item_id where metadata_items.metadata_type=4 and media_parts.extra_data not like '%pv^%3Aintros=%' escape '^' and metadata_items.id not in ( select distinct(metadata_items.id) from metadata_items join media_items on metadata_items.id=media_items.metadata_item_id join media_parts on media_items.id=media_parts.media_item_id where media_parts.extra_data like '%pv^%3Aintros=%' escape '^' ) order by metadata_items.added_at

I don’t know a way to query the progress for chapter thumbs, but this is the fastest of the 3, so it won’t matter that much, IMHO.

2 Likes
13

thank you @OttoKerner, this is helpful. It took approx 2 days to complete the deep analysis work on all my media. I was manually adjusting the scheduled time of the background tasks so I could see how it was going.

Anyhow, chapter thumbnails is in progress now (after I enabled it in my library settings) and its zipping through it. Will then move onto intros which I know will take time.

Again, thanks for the info and help. I’m we are okay now on this topic and can be closed if you wish.

14

More of an FYI than anything else. After a full rebuild of my server my databases are back to the sizes I would expect them to be (blobs.db being now approx half the size it was before).

Plex Media Server Database Sizes:
919M	/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.blobs.db
220M	/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db

For a timing perspective this is the approx times is took my system to complete the butler tasks. I monitored each of these and enabled each one by one until complete keeping note of their times and progress in the logs…

NOTE: deep analysis took approx 40 hours
NOTE: chapter thumbnails took approx 4 hours
NOTE: intro markers took approx 40 hours

My system is:

OS: Ubuntu 20.04.2 LTS x86_64
CPU: Intel i7-9750H (12) @ 4.500GHz
GPU: NVIDIA GeForce RTX 2080 Mobile
Memory: 32029MiB
Movies: 2640
TV: 417 series with approx 23k episodes

In summary: I suspect there is some duplication or cleanup failure happening when intro detection happens for a second time on a system as com.plexapp.plugins.library.blobs.db pretty much doubled. I’ll let you folks decide if you want to pursue this any further.

1 Like
15

Please wait a full week before giving the final judgment. Some cleanup maintenance jobs only run once a week.

2 Likes
16

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.