HELP - Plex Media Server/Cache/PhotoTranscoder is 577G!

[rkv]

Server Version#: 1.32.7.7621
Player Version#: not relevant
os: Ubuntu 20.04.6 LTS

I have tinkered with the settings, changed time schedule and restarted server. The Cache simply continues to grow. It seems to be all entries since sometime in 2021.

This seems to be related to issues huge-size-of-phototranscoder-folder and a feature request back in 2020, including at least one claim of a bug being fixed. However, I seem to be experiencing this.

Will deleting PhotoTranscoder subdirectories break things? or entries older than 30 days? What is the recommended course of action? If I manually delete is there anything else I need to do for PMS to be happy with the deletions? Are such manual deletions SAFE for me to make.

My 577G PhotoTrancoder directory has 2,749,006 entries comprising:
257 directories containing:
2,580,008 .jpg files
2,962 .jpeg files
27,124 .png files
138,334 .ppm files

I am only using Plex for video. No photos or music.

What is the recommended course of action?

My Server Settings

du and ls output related to my /var/lib/plexmediaserver/Library/Application Support/Plex Media Server directory

`
┌─(/var/lib/plexmediaserver/Library/Application Support/Plex Media Server)
└─(09:45:06)──> du -BG -d1 . | sed -r “:r;s/\b[0-9]{1,3}G\b/0&/;tr” | sort | sed -e “s/^000/ /” -e “s/^00/ /” -e “s/^0/ /” | sed -E “s/^([0-9])([0-9]{3})G/\1,\2G/”
1G ./Codecs
1G ./Crash Reports
1G ./Diagnostics
1G ./Drivers
1G ./Logs
1G ./Plug-ins
1G ./Scanners
1G ./Updates
3G ./Plug-in Support
14G ./Media
31G ./Metadata
577G ./Cache
624G .
┌─(/var/lib/plexmediaserver/Library/Application Support/Plex Media Server)
└─(09:45:47)──> du -BG -d1 ./Cache | sed -r “:r;s/\b[0-9]{1,3}G\b/0&/;tr” | sort | sed -e “s/^000/ /” -e “s/^00/ /” -e “s/^0/ /” | sed -E “s/^([0-9])([0-9]{3})G/\1,\2G/”
1G ./Cache/CL-ICDs
1G ./Cache/cl-icds-linux-x86_64
1G ./Cache/fontconfig
1G ./Cache/OCSP
1G ./Cache/Transcode
1G ./Cache/va-dri-linux-x86_64
577G ./Cache/PhotoTranscoder
577G ./Cache

┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:07:19)──> ls -1 * | wc -l
2749006
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:07:35)──> find . -type d | wc -l
257
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:08:17)──> find . -type f -name “.jpg" | wc -l
2580008
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:08:51)──> find . -type f -name ".jpeg” | wc -l
2962
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:09:02)──> find . -type f -name “.png" | wc -l
27124
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:09:56)──> find . -type f -name ".ppm” | wc -l
138334

┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:27:10)──> ll | head -10
total 205M
drwxrwxr-- 2 plex plex 816K Dec 4 03:51 00
drwxrwxr-- 2 plex plex 828K Dec 4 07:02 01
drwxrwxr-- 2 plex plex 816K Dec 4 07:02 02
drwxrwxr-- 2 plex plex 824K Dec 4 07:02 03
drwxrwxr-- 2 plex plex 812K Dec 4 01:40 04
drwxrwxr-- 2 plex plex 808K Dec 4 00:17 05
drwxrwxr-- 2 plex plex 796K Dec 4 07:02 06
drwxrwxr-- 2 plex plex 800K Dec 4 07:02 07
drwxrwxr-- 2 plex plex 824K Dec 3 23:33 08
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:27:33)──> ll | tail -10 drwxrwxr-- 2 plex plex 780K Dec 4 00:36 f6
drwxrwxr-- 2 plex plex 816K Dec 4 00:20 f7
drwxrwxr-- 2 plex plex 836K Dec 3 23:29 f8
drwxrwxr-- 2 plex plex 844K Dec 3 23:31 f9
drwxrwxr-- 2 plex plex 808K Dec 4 01:40 fa
drwxrwxr-- 2 plex plex 804K Dec 3 23:19 fb
drwxrwxr-- 2 plex plex 836K Dec 3 23:36 fc
drwxrwxr-- 2 plex plex 808K Dec 3 23:36 fd
drwxrwxr-- 2 plex plex 824K Dec 4 07:02 fe
drwxrwxr-- 2 plex plex 824K Dec 4 00:20 ff
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:27:42)──> ls -lshptN 01/* | head -10
44K -rw-r–r-- 1 plex plex 43K Dec 4 07:02 01/01ee7aa3a41fc67069fd6f85c04c42b20101baaf.png
388K -rw-r–r-- 1 plex plex 385K Dec 4 01:40 01/018350948b79f2be615154d27dbb19c4cb5ca4a9.jpg
4.0K -rw-r–r-- 1 plex plex 711 Dec 4 00:20 01/01423f14708cb91c2eda267d4a58a2e7c049623c.jpg
16K -rw-r–r-- 1 plex plex 16K Dec 3 23:32 01/01497b328cfa71a5c9378a0e2685688566572ea9.jpg
500K -rw-r–r-- 1 plex plex 499K Dec 3 23:32 01/01e093256551f37b2f8c16e328ab289be44f6ab8.jpg
20K -rw-r–r-- 1 plex plex 18K Dec 3 23:13 01/01cfb02af0ffe03a6c23877fafbb03fa197b8279.jpg
152K -rw-r–r-- 1 plex plex 149K Dec 3 23:04 01/0184f8125a2cd826d30f4e3d1b6b76f987bd1eb0.jpg
152K -rw-r–r-- 1 plex plex 149K Dec 3 23:03 01/0138c878cdd51d1db1071df0065393414faf0b5d.jpg
152K -rw-r–r-- 1 plex plex 149K Dec 3 22:53 01/011dd6ddd786c37bdb83b8923db2f40d537da4d0.jpg
20K -rw-r–r-- 1 plex plex 18K Dec 3 22:53 01/01415447aed70fe7521d24b99cabc83ac909b4de.jpg
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:28:05)──> ls -lshptNr 01/* | head -10
1.4M -rw-r–r-- 1 plex plex 1.4M Nov 16 2021 01/01d2a3e8dfb3bbda9e0158a42fb516f2df8ed83f.jpg
396K -rw-r–r-- 1 plex plex 395K Nov 16 2021 01/016ac45315f32e7bb3607ecf9afe1f5f64662c36.jpg
88K -rw-r–r-- 1 plex plex 86K Nov 16 2021 01/017cbf16f72cba004c854e17a34eb3f206a0fba7.jpg
16K -rw-r–r-- 1 plex plex 14K Nov 17 2021 01/017441bf3afe474de435cee909dfdae129ec8d6c.jpg
20K -rw-r–r-- 1 plex plex 17K Nov 17 2021 01/01a285a0ecc54e9daf8a96f9c12659d1b55297d9.ppm
524K -rw-r–r-- 1 plex plex 522K Nov 17 2021 01/019f39812e7d5dbc7ecbb6e3c46b3f15ee3952bc.jpg
20K -rw-r–r-- 1 plex plex 17K Nov 17 2021 01/01c6b871b4fbcd3f6e2b41eb2f9ff619685578e3.ppm
72K -rw-r–r-- 1 plex plex 71K Nov 17 2021 01/017a0a74e8829063d752d5f2ea7a27f60dbfc537.jpg
136K -rw-r–r-- 1 plex plex 136K Nov 17 2021 01/016b632236d4ee68547186e82eca8f5ffca6b297.jpg
20K -rw-r–r-- 1 plex plex 17K Nov 17 2021 01/017ff1b05d9d614ba8d463ea1e99f2b49e9c9c35.ppm
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:28:13)──> ls -lshptN ff/* | head -10
4.0K -rw-r–r-- 1 plex plex 945 Dec 4 00:20 ff/ffb2887219de7fb366cc6df44ca9c4fe68b4fbe4.jpg
152K -rw-r–r-- 1 plex plex 149K Dec 3 23:24 ff/ff2f09d96d2073c30cc06f7893b2ec5ac22d7821.jpg
20K -rw-r–r-- 1 plex plex 18K Dec 3 23:21 ff/ff0c2f3cac28b9a3434caed551b375094ae6df3f.jpg
20K -rw-r–r-- 1 plex plex 18K Dec 3 23:03 ff/ff50bb8b00748cf354946878b77260270cba84db.jpg
592K -rw-r–r-- 1 plex plex 591K Dec 3 22:02 ff/ffe4b2442cccc6e8932e4c9f37cedac91652c50e.jpg
20K -rw-r–r-- 1 plex plex 19K Dec 3 21:48 ff/ff9ce336088162b54df197439c0b394b2be9159d.jpg
592K -rw-r–r-- 1 plex plex 591K Dec 3 21:44 ff/ff0a2b1815c3fdd9df7b0074394d0e57f16e4c10.jpg
188K -rw-r–r-- 1 plex plex 185K Dec 3 20:53 ff/ff5e3d73def477f27d8067bdcfa0a0db8271ebb2.jpg
112K -rw-r–r-- 1 plex plex 109K Dec 3 18:08 ff/ffe2c7cf6dc4e2bbbd2b7a3cb28bd1832b05c42f.jpg
12K -rw-r–r-- 1 plex plex 11K Dec 3 03:27 ff/ffa4c2f442875185bd6cea5a4ebd7a144149adac.jpg
┌─(…exmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder)
└─(10:28:26)──> ls -lshptNr ff/* | head -10
20K -rw-r–r-- 1 plex plex 20K Nov 16 2021 ff/ff7230a5b9b1a277c3eefb3aa9abba4422c98659.jpg
56K -rw-r–r-- 1 plex plex 53K Nov 16 2021 ff/ff840dc700b9eefddfdd568ebb12c826a6d75e25.jpg
920K -rw-r–r-- 1 plex plex 917K Nov 16 2021 ff/ff3431e0e40863a77b072728ac53450b0152f21c.jpg
196K -rw-r–r-- 1 plex plex 195K Nov 17 2021 ff/ff61c1076d477002b6aa15a6352a6caeea6f7f7d.jpg
20K -rw-r–r-- 1 plex plex 20K Nov 17 2021 ff/ffd873da6b1ef0e0a69521a26ca86e6215a863b9.jpg
336K -rw-r–r-- 1 plex plex 333K Nov 17 2021 ff/ff055a5637a71a1fb0e48adcaaa098098f9f8e30.jpg
20K -rw-r–r-- 1 plex plex 20K Nov 17 2021 ff/fffc56cb34bc424395fceb0e37652a981a32fa2d.ppm
20K -rw-r–r-- 1 plex plex 17K Nov 17 2021 ff/ff5ac93493ebcb4fab1d51b9cb449596dd17f523.ppm
20K -rw-r–r-- 1 plex plex 17K Nov 17 2021 ff/ffcf354a2105987eb55b8ae7adf3947f4ea5dbea.ppm
332K -rw-r–r-- 1 plex plex 332K Nov 17 2021 ff/ff891f3d28cec9f0a9c1008b2a17ed1d4e488d68.jpg

`

[cjmanca]

Can’t help with the actual problem, but anything in the cache directory should be safe to delete. A cache is temporary storage by nature, meant to be regularly cleared. Anything that’s actually needed by the server will be automatically recreated/reaquired.

I redirected my plex cache to a ram disk, personally.

[rkv]

@cjmanca Thanks.

Here is what find and jdupes discover:

> pwd
/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder

>find . -type f -mtime +30 | wc -l
2449310

> jdupes -rm .
Scanning: 2748614 files, 257 items (in 1 specified)
2360955 duplicate files (in 133503 sets), occupying 575013 MB

I’m going ahead with deleting all the older entries and then I’m going to use jdupes to hard-link any remaining duplicates. I sure wish PLEX would rethink their management of this cache. Will report back on the final outcome.

[rkv]

@cjmanca, just curious how big is your ram disk? Mostly interested in how small it can be. Does Plex gracefully deal with the ram-disk volume being full?

[cjmanca]

For now, I’d either make a cron to clear it regularly or at least a script to clear it before plex starts (possibly as part of a weekly maintenance script). Or do what I did and set your transcode cache to a ram disk. I set plex to put it’s cache in the tmp directory, which is mapped to ram:

[rkv]

@cjmanca is ‘Plex Media Server/Cache/PhotoTranscoder’ where it is storing transcodes? I thought it was rescaled photo objects from posters and the like. I thought the setting you are showing is for transcoding of streamed videos which are not saved? That is deleted after they are streamed? Is my thinking with respect to the setting wrong?

[cjmanca]

I use docker, and have /tmp/ mapped to /dev/shm/plex/tmp (which is the native linux ramdisk, using a maximum of half your total system memory). I could have used the tmpfs option in docker though to specify a different amount. I have 32gb of system memory, so at most it would be using 16gb, and I haven’t had issues.

[cjmanca]

Hmm, perhaps that’s different then? I should look into moving that into a ramdisk too maybe. It’s still a cache though, so should be perfectly fine to clear.

[rkv]

As an update, find and jdupes helped me to reduce the Plex Media Server/Cache/PhotoTranscoder directory to 2.9G. However, Plex seems to add over 1G perday of which close to 90% are duplicates. The creation of so many duplicates would certainly seem to indicate a weakness in Plex’s management of this cache.

My find and jdupes command sequences are here:

> pwd
/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder

>find . -type f -mtime +30 | wc -l
2449310

> jdupes -rm .
Scanning: 2748614 files, 257 items (in 1 specified)
2360955 duplicate files (in 133503 sets), occupying 575013 MB

>find * -type f -mtime +30 | wc -l
2449310

> find . -type f -mtime +30 -delete

> du -sh .
64G	.

> sudo jdupes -r -L .

> du -sh .
2.9G	.

# A day later ...

> du -sh .
4.0G	.

> jdupes -rm .
Scanning: 305396 files, 257 items (in 1 specified)
3409 duplicate files (in 824 sets), occupying 939 MB

> sudo jdupes -r -L .

> du -sh .
3.4G	.

[dzm06]

Note: OP is using a Linux distro, so everything. discussed here is relevant to their configuration. This would likely work for macOS as well (a FreeBSD distro). I have no idea if any of this would apply to Windows, or what machinations would be needed to apply these to Dicker containers or whatever streamlined BusyBox Linux distro your NAS is using.

Not an actual fix for the problem (bad cache management and garbage collection), but perhaps useful to address the symptom (PhotoTranscoder growing without bounds).

Brute force solution:
This will prevent the directory from getting new files written to it. Plex appears to work fine in this configuration, but your mileage may vary.

• Remove all contents in the folder: rm -rf /path/to/plex/Cache/PhotoTranscoder/*
• Change the permissions such that Plex can no longer abuse this directory: chmod a-w /path/to/plex/Cache/PhotoTranscoder

Less brute force - delete cache files older than a certain age:
Since Plex seems to never delete files (or do so rarely), or only wants to delete files older than 30 days when working as designed, this will act as a more aggresive garbage collector:

• Add a crontab that will delete all files older than X days or X minutes: 34 * * * * find /path/to/plex/Cache/PhotoTranscoder/ -type f -mmin +1680 -exec rm {} \; 2>&1
  This cron entry will run at 34 minutes past the hour every hour of the day (e.g. 12:34, 1:34, 2:34, etc). The find command will find all files in the directory path specified that were monified 1680 minutes (28 hours) or longer ago and will delete them. I chose these values (34 minutes) to avoid other scheduled jobs that typically run at the top of the hour, and the age (28 hours) so that if the cache files ARE actually getting used then they’ll be available the next day after they were created.

Other options to be considered with unknown real impact:

• Create a local disk image remounted over the PhotoTranscoder directory (use dd to create the image, and use the mount -o option in fstab). This would allow you to effectively pre-allocated a certain amount of space (100m? 10G?) and sandbox Plex into that space.
• Create a tempfs (RAM disk) and mount it over the PhotoTranscoder directory (same value as the local disk, except that this lives in RAM and avoids all these writes hitting your SSD)
• If using a FS like btrfs, create a subvolume over the PhotoTranscoder directory with quotas enabled to restrict the growth

All three of these would allow you to limit how much space Plex could chew up with this cache directory. In my own experience Plex just fills up the available space and does no garbage collection to try to free space, so the value is ultimately limited.

On my system I allocated a 50m tempfs volume and mounted it over the PhotoTranscoder directory. I then use the cron job to be more aggressive in the garbage collection. Plex seems to work perfectly well in this configuration.

[ChuckPa]

@rkv

Will deleting PhotoTranscoder subdirectories break things? or entries older than 30 days?

If you stop PMS, delete the PhotoTranscoder cache directory, then restart PMS

1. You don’t hurt anything
2. The worst thing that happens is:
   – An extra line in your logfile as it says it’s recreating it
   – You will notice a slight delay as they are regenerated as needed.

I will occasionally go in there and smash down the JPEG files.

Don’t delete the codecs, especially the .device_id file because these are your codecs and transcoder license key. Deleting these causes a noticeable delay the first time you want to play media again (it all has to be downloaded again)

[ChuckPa]

If I added a menu item in my DBRepair tool to do this type of maintenance, would it be helpful ?

DBRepair-Development-1.02.99.tar (70 KB)

Usage: Stop - Remove - Start - Exit

[rkv]

@ChuckPa, yes that would be helpful.

Since I first came across this issue, the items continue to grow. When I checked today I had 97G of items which were reduced to 13G after running jdupes. It does not appear that the butler is making any effort to eliminate old items. As you will see from the find command output there are more than 28k items older than 30 days.

du, find, and jdupes output as of 13 Jan 2024

13 Jan 2024
[18:12:03] /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

du -sh .
97G .

---

[18:12:13] /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

find . -type f -mtime +30 -print | wc -l
27596

---

[18:12:19] /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

jdupes -rm .
Scanning: 486762 files, 257 items (in 1 specified)
404421 duplicate files (in 11309 sets), occupying 89494 MB

[18:45:05] plexadmin:/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

sudo jdupes -qrHL .

---

[18:57:52] /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

find . -type f -mtime +30 -print | wc -l
28094

---

[19:15:07] plexadmin:/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder (0)

du -sh .
13G .

I’ll look at the DBRepair tool or implement a daily cronjob as @dzm06 suggests.

Thank you for the responses.

[dzm06]

One other option I didn’t mention that would allow more of the cached files to exist before you flush them is to use jdupes (on the Debian fork I’m using jdupes wasn’t installed, but that was easily fixed with apt-get install jdupes) to soft (or hard) link all the duplicated files together and then provide a longer garbage collection window. For example:

# df output showing the cache mountpoint use this morning
% df -h
[...]
tmpfs            50M   45M  5.9M  89% /mnt/volatile_cache

# run jdupes to create soft-links between all the redundant files
% pwd
/mnt/volatile_cache/PhotoTranscoder
% jdupes -rL .
Scanning: 179 files, 257 items (in 1 specified)
[SRC] ./32/32ff1f42d7ae403b355622835c31b0e075229860.jpg
-@@-> ./db/db8b347fbb52b30dfba6c6c45be67e7bb2368b06.jpg
[...]

# df to see the savings/impact
% df -h
[...]
tmpfs            50M   12M   39M  24% /mnt/volatile_cache

So doing a deduplicates cleanup (not deleting anything) saved 33M of storage. In theory these could be combined

Note that you can use hard links (-L) or soft links (-l). I don’t think Plex will care much about the difference. But using Soft Links would run the risk of ending up with orphaned links to deleted originals, where hard links should preserve that original over time.

To combine this with the more aggressive caching in crontab change the line to something like:

34 * * * * jdupes -qrL /mnt/volatile_cache ; find /mnt/volatile_cache -type f -mmin +3360 -exec rm {} \; 2>&1

As before, this runs at 32 minutes after the hour every hour of the day. It runs jdupes with the options -r (recurse into subdirectories), -L (replace duplicated files with a hard link to the first of the found duplicates), and -q (suppress the progress output in this automated, headless task). It waits for jdupes to finish (the ; tells the shell processor “wait for the previous command to finish, then do what comes next”). When jdupes finishes the find command runs, though I’ve changed the last-modified value to be 3,360 minutes (56 hours).

In theory this will result in one unique copy of each re-rendered image file, and multiple “pointers” in the file system that refer to it. The more aged pointers will eventually get deleted by the find command, but the underlying bits-on-disk will persist so long as there is at least one hard-link to it that is younger than 56 hours. Once the youngest is older than 56 hours the single unique copy will also be removed, and Plex will create a new one if it finds itself wanting it again.

I want to stress that all of this is just working around a clearly broken cache mechanism and garbage-collection mechanism in Plex server. The Plex team really should fix this.

[ChuckPa]

ALL

Given the feedback I’ve had, I’m going to release this capability in a 1.03.00 update of my tool.

I’ve not decided whether to call is ‘clean’, ‘remove’ , ‘purge’, or something else.

I welcome suggestions for a clever command name.

“Clean” is ambiguous
“Delete” is too scary
“purge” seems most appropriate so far “purge all JPG files older than 30 days”
“remove” is also applicable.

Guess it depends on which continent you live ?

All are welcome to join the development discussion:

PlexDBRepair development

Difficult choice time… “purge” - “Purge old files from PhotoTranscode cache” “remove” - “Remove old files from PhotoTranscode cache” My only concern is future development . I’ve toyed with “Clear”, “Clean”, “Purge”, “Remove”, and “Delete”. Delete - Can imply way too much Clear - Not descriptive enough (clear what? the screen? - lol) Clean - “clean the keyboard?” – lol Purge & Remove seem best candidates. I will change the numbering, I’ll move this down into its ow…