I know Thumbnail Previews use storage, but 1.4TB might be a bit excessive

Plex Media Server
,
1

Server Version#: 1.20.1.3252
Player Version#: Web 4.42.1

It seem that every time I enable Thumbnail Previews, the process seems to work fine and then it create a huuuuge TMP file and one of the “.bundle” directories fills my Cache/AppData (Unraid) Drive. The most recent offending bundle is 1.4TB (something in its /content/tmp folder I can’t easily open or view) whereas other bundles are around 50MB

1.4T 0afcxxxxxxx.bundle
46M c4f9axxxxxxx.bundle

$: ...Media/localhost/f/0afcxxxxxx.bundle/Contents/Indexes# ls -al
total 0
drwxrwxrwx 1 nobody users          6 Sep 11 03:50 ./
drwxrwxrwx 1 nobody users        146 Jul 19 18:23 ../
drwxr-xr-x 1 nobody users 2784689688 Sep 11 10:37 tmp/

Looking at the Media Scanner Analysis logs, I do see some errors where FFMPEG is erroring out analyzing media

Invalid data found when processing input

I’m not sure if these files are the ones causing problems or if there’s something else going on.

Is there a way to match the Bundles to what Library or File(s) that’s causing this to happen?
If you’d like additional logs, let me know.

Thanks for looking.

2

fyi if you use ls -lah it will display sizes in mb/gb/tb, like the bundle list.

3

I started an ls -al on the /tmp directory in the bundle and its been trying to figure it out for the last hour now

4

bundles are often hundreds/thousands of small sub folders/files, which can take a long time to enumerate.

5

My Metadata is also grow to big for my taste
the Metadata / Movie is about 12.39 GB
and the Metadata /TV Shows 6,64 GB
Plex Media server last version (plexpass) OS macOS Catalina
I remove all the previews and trailers

6

I have a pretty huge library with video/preview thumbs enabled. I have a nearly full 1tb ssd drive only for plex data.

/dev/sdz btrfs 932G 842G 90G 91% /dataplex

my media folders all seem pretty close together though, no giant TB folders.

tekno@proximo:~$ du -hsc /dataplex/pms/Library/Application\ Support/Plex\ Media\ Server/Media/localhost/*
40G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/0
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/1
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/2
40G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/3
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/4
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/5
42G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/6
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/7
40G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/8
42G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/9
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/a
41G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/b
40G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/c
42G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/d
40G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/e
42G     /dataplex/pms/Library/Application Support/Plex Media Server/Media/localhost/f
646G    total

also my server is linux though, not mac

7

I believe that log entries related to index-sd.bif generation are in the main Plex Media Server.log file.

ls is much slower (it pokes every file) if you’ve got it aliased to something like ls --color. Try unalias ls if you’re using bash/zsh.

For very big directories it can be (non-intuitively) faster to do a find blah.bundle/tmp -ls instead of ls itself.

8

Thanks for the other du/ls suggestions, however, they never returned results after waiting 3+ hours.

The du -sh on the <PlexDIR>/Media/localhost folder is what helped me to initially find the bundle in the “f” folder; that 1.4T abomination; every other “alphanumerical” folder was sub 10G.

In the offending bundle, there was no identifiable details about what it corresponded to and the single thumbnail image that was present was a black box; as if whatever it tried to process caused it to fail and zip bomb the directory.

I gave up on trying to figure out what’s in this tmp folder; and went for the delete; that too fails. Any attempt to try andrm -rf’ing it (both the tmp/ folder and at the .bundle level causes the the whole system to lock up. I bounced the box (uncleanly, need to do a parity check now) and had all the Dockers and VMs turned off to :crossed_fingers: to hopefully have nothing touching that directory or file, and the whole thing still hangs and nothing’s deleted. I’m not sure why, but when the rm is run the machine hits 92% RAM usage of the 64 GB installed and the system isn’t happy about it.

After 10+ hours of trying to figure this out, I said f-it and formatted the cache/appdata drives; that’s what backups are for, right? I lose some playback history, but hey, my Plex folder is only 120GB (backup was from Tues and nothing significant has been added and thumbnails were re-enabled about 2-3 weeks ago).

I’m up and running again but WTF?

This isn’t the first time I’ve had Plex with Thumbnail Previews enabled seem to just go off the rails whenever it wanted and fill my drive. This issue only seems to be worse after upgrading from a 500 GB drive to 2TB; on that old drive, the max size Plex could get was 500 GB and I could easily delete the offending tmp folder.

  • Any suggestions?
  • Stop trying to have the shiny and keep thumbnails turned off?
  • Do I Tdarr my library to see if it borks on something?
  • Is there a file size limiter option in Plex so that it stops trying to make an infinite sized file?
9

I don’t think you are the first or only person who has had a similar issue, but I don’t know if any specific bug or problem isolated/identified.

My assumption is there is a bad file(s), or a file that the thumbnail generator doesn’t like.

if you know/knew what the offending bundle was, you can monitor it if it starts to get large again, and hopefully stop before it gets out of hand.

I’m not a person who can do it, but I’m sure that someone here can probably trace the bundle back, through the database, to figure out which file(s) may be having problems.

Once the problem file(s) can be identified and isolated, and copy/sample made that can consistently replicate the problem, then perhaps a solution can be identified and implemented.

edit: i don’t think you mentioned what file system it is, but I wouldn’t rule out something corrupted at the file system level, especially if some of those commands locked up or stalled the system.

since you already formatted the file system, its too late to run a fsck on it to see if any other errors turned up.