Content not getting optimized

General Discussions
1

I’ve got 3 plex servers.
NAS
Linux VM
Raspberry Pi

I’ve setup the NAS and Linux VM to access the same content. (mount the NAS via NFS on the Linux VM)
I use the Linux VM to transcode content, I then copy this to the Pi for using in the car.

The problem is that the Optimized versions doesn’t pick up new content as I add it to the library, in fact it seems to delete content.
the status looks good

9wqjnk76k4cw.jpg1196×342 83.8 KB

But if I drill down I see a bunch of issues. Now I can refresh that content 1 file at a time.
Which is tedious and impractical.

hymkf0f6tg1x.jpg615×550 49.1 KB

Questions
How can I get the optimizer to recognize and transcode the new content automatically.
or
How can I kick of a transcode for all the missing content.

Thanks
Rich

2

Check two things:

  1. File permissions of the new files you add. If plex can’t read them there is part #1 of the problem.
  2. Go to Settings - Server - Library and check these boxes.
  • Update my library automatically
  • Run a partial scan when changes are detected

Then run another update and see how much progress you’ve made.

3

Hi Chuck

file permissions are all good and those boxes were ticked, but I’ve dug a little deeper.

The good news
Files are auto optimizing, it is watching the dir and creating new content.
Specifically
The NAS is running DVR and creating new library content.
The linux server is also seeing (via nfs mount) that as new content then optimizing it.

The bad news
Something deleted a bunch of the optimized files
I need to look deeper into this. Theories are

  1. DVR is deleting the optimized files
  2. I also transcode the .ts file manually then replace the .ts with an mkv thats also been clipped (advert removal etc)
    I’ll check my workflow as I have a bunch of delete script (99% sure they are not the culprit as they don’t access the optimized content dir)

I transcode and add the mkv to the libray
then delete the .ts
So maybe the optimize logic is seeing the new file, but not transcoding it as there is already a version.
Then I delete the .ts which triggers the optimzer to delete it’s copy.

When I get a chance I’ll work through this and note what happens to the files at each step.

There is another ask though. Whatever the reason a file got deleted, could we have a “refresh all” button.
that would be awesome :slight_smile:

4

Imo, you just hit the nail on the head:

I also transcode the .ts file manually then replace the .ts with an mkv thats also been clipped (advert removal etc)
Specifically:
I transcode and add the mkv to the libray then delete the .ts

The best way to do this:

  1. pull the media to be processed OUT of plex.
  2. Process the media
  3. Update the library , empty trash, clean bundles before placing the new file back in
  4. Put the new MKV back in position
5

I’m having the same issue. I tell it to optimize, it completes, says it’s done, and then shows deleted. Plex has full permissions to the folder and just for kicks I set it as 777 still nothing. Looking in the Plex Media Server.log on debug it doesn’t show a whole lot after the transcode. Any suggestions?
Ubuntu Linux 16.10

6

Kick off the task again, watching its progress.
When it completes, without doing anything else, Collect the logs immediately (Settings - Server - Help - Download Logs)

Let me see that zip. If it’s deleting it will be there. If there’s any other obvious error, it will be there too

7

Ok a bunch more content got deleted from the Optimized folder without me doing any manual moving of files.
I’ve focused in on 1 particular show that is being recorded by the DVR.

I have set the optimized content to be in it’s own folder
The library has
47 .mkv files - content I have added to the library with the correct s00e00 naming
28 *.ts files the DVR recorded, these have date based names e.g PJ Masks (2015) - 2016-04-20 00 00 00 - Owlette and the Giving Owl.ts

In the optimized versions directory I have
26 mp4 files
These are named (2016-04-20.mp4) as an example.
1st bug
I have some .ts files from the same day. There is only 1 optimized version even though there were two source files.
I assume this is because the precision is lost on the transcoded file name.
PJ Masks (2015) - 2016-02-25 00 00 00 - Gekko and the Pogo Dozer.ts
PJ Masks (2015) - 2016-02-25 00 00 00 - Owlette and the Flash Flip Trip.ts
PJ Masks (2015) - 2016-03-02 00 00 00 - Gekko and the Super Ninjalinos.ts
PJ Masks (2015) - 2016-03-02 00 00 00 - Owlette’s Terrible Pterodactyl Trouble.ts

2nd bug
Something has deleted the optimized version of the 47 mkv files above. (the orginal mkv’s are still in the library)
Those versions were in the folder previously. (On Apr 30th)

Logs have rolled over so I’ll have to re-run this to try to capture the events in the log.

8

Here are some logs from around the event.

Cronjob that does an ls on the dir.
Sat 20 May 11:51:01 BST 2017
55
Sat 20 May 11:52:01 BST 2017
55
Sat 20 May 11:53:01 BST 2017
1
Sat 20 May 11:54:01 BST 2017
1
Sat 20 May 11:55:01 BST 2017

Plex logs from that time period
Plex Media Server.log:May 20, 2017 11:48:33.873 [0x7f44beffd700] INFO - AutoUpdate: no updates available
Plex Media Server.log:May 20, 2017 11:48:47.584 [0x7f44a2fff700] WARN - [CompanionPlayer] We already have a handler, overwriting.
Plex Media Server.log:May 20, 2017 11:50:06.383 [0x7f44a2fff700] ERROR - Error issuing curl_easy_perform(handle): 52
Plex Media Server.log:May 20, 2017 11:50:06.383 [0x7f44a2fff700] WARN - HTTP error requesting GET http://192.168.0.27:8008/ssdp/device-desc.xml (0, No error) (Empty reply from server)
Plex Media Server.log:May 20, 2017 11:52:27.630 [0x7f44a93ff700] INFO - It’s been 17521423 seconds, so we’re starting scheduled library update for section 5 (Kids Movies)
Plex Media Server.log:May 20, 2017 11:52:27.630 [0x7f44a93ff700] INFO - It’s been 9490923 seconds, so we’re starting scheduled library update for section 4 (Kids TV Shows)
Plex Media Server.log:May 20, 2017 11:52:27.631 [0x7f44a93ff700] INFO - It’s been 17521519 seconds, so we’re starting scheduled library update for section 3 (Movies)
Plex Media Server.log:May 20, 2017 11:52:27.631 [0x7f44a93ff700] INFO - It’s been 2649450 seconds, so we’re starting scheduled library update for section 6 (Photos and Videos)
Plex Media Server.log:May 20, 2017 11:52:27.631 [0x7f44a93ff700] INFO - It’s been 17521547 seconds, so we’re starting scheduled library update for section 1 (TV Shows)
Plex Media Server.log:May 20, 2017 11:52:30.222 [0x7f44a73ff700] INFO - Library section 4 (Kids TV Shows) will be updated because of a change in /mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks
Plex Media Server.log:May 20, 2017 11:52:50.994 [0x7f44a73ff700] INFO - Library section 4 (Kids TV Shows) will be updated because of a change in /mnt/PlexOptimized/TV/Plex Versions/Raspberry320 79/PJ Masks
Plex Media Server.log:May 20, 2017 11:52:51.796 [0x7f44a73ff700] INFO - Library section 4 (Kids TV Shows) will be updated because of a change in /mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks
Plex Media Server.log:May 20, 2017 11:52:52.310 [0x7f44a73ff700] INFO - Library section 4 (Kids TV Shows) will be updated because of a change in /mnt/PlexOptimized/TV/Plex Versions/Raspberry320 79/Paw Patrol
Plex Media Server.log:May 20, 2017 11:53:03.710 [0x7f44ba3fe700] ERROR - Couldn’t get size of file /mnt/Multimedia/Photos/2015/2015 Bude: boost::filesystem::file_size: No such file or directory: “/mnt/Multimedia/Photos/2015/2015 Bude”
Plex Media Server.log:May 20, 2017 11:53:03.710 [0x7f44ba3fe700] ERROR - Couldn’t get size of file /mnt/Multimedia/Photos/2015/2015 Sep 27 Lotty: boost::filesystem::file_size: No such file or directory: “/mnt/Multimedia/Photos/2015/2015 Sep 27 Lotty”

The only content being affected is content I have scheduled DVR recording for. Now the wierd thing is the server with the DVR (QNAP) is different to the server doing the “Optimized Versions” (linux VM). They both access the same library content via nfs (content is on the QNAP and access by the linux VM via nfs)

The other thing that may be relevant is the directory the DVR records to is symlinked to the actual dir.
lrwxrwxrwx 1 admin administ 43 Mar 12 09:37 Octonauts (2010) → /share/TV_Programs/KidsTV/Octonauts (2010)//
lrwxrwxrwx 1 admin administ 44 Mar 12 10:55 PAW Patrol (2013) → /share/TV_Programs/KidsTV/PAW Patrol (2013)//
lrwxrwxrwx 1 admin administ 42 Mar 12 11:54 PJ Masks (2015) → /share/TV_Programs/KidsTV/PJ Masks (2015)//
drwxr-xr-x 9 admin administ 4096 Apr 17 07:41 Postman Pat (1981)/
drwxrwxrwx 3 admin administ 4096 May 22 07:03 Raa Raa the Noisy Lion (2011)/

Some other linux logs
I setup inotify. I see a lot of delete entires (there are lots more than shown below)
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E36.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E37.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E38.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E39.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E40.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E41.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E42.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E43.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E44.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E45.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E46.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E47.mp4
/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ DELETE S01E48.mp4

Finally a bunch of auditctl logs. You can see my cronjob ls, then Plex Media Scanner unlinking the file.
I think we have the smoking gun :slight_smile:

type=PATH msg=audit(20/05/17 11:52:01.167:313516) : item=0 name=/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/S01E03.mp4 inode=32250961 dev=00:27 mode=file,644 ouid=plex ogid=plex rdev=00:00 obj=system_u:object_r:nfs_t:s0 objtype=NORMAL
type=CWD msg=audit(20/05/17 11:52:01.167:313516) : cwd=/root
type=SYSCALL msg=audit(20/05/17 11:52:01.167:313516) : arch=x86_64 syscall=getxattr success=no exit=ENODATA(No data available) a0=0x7ffd288c15b0 a1=0x7efe684b5ddf a2=0x0 a3=0x0 items=1 ppid=28174 pid=28176 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=38510 comm=ls exe=/usr/bin/ls subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)

type=PATH msg=audit(20/05/17 11:52:01.167:313517) : item=0 name=/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/S01E03.mp4 inode=32250961 dev=00:27 mode=file,644 ouid=plex ogid=plex rdev=00:00 obj=system_u:object_r:nfs_t:s0 objtype=NORMAL
type=CWD msg=audit(20/05/17 11:52:01.167:313517) : cwd=/root
type=SYSCALL msg=audit(20/05/17 11:52:01.167:313517) : arch=x86_64 syscall=getxattr success=no exit=ENODATA(No data available) a0=0x7ffd288c15b0 a1=0x7efe684b5db0 a2=0x0 a3=0x0 items=1 ppid=28174 pid=28176 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=38510 comm=ls exe=/usr/bin/ls subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)

type=PATH msg=audit(20/05/17 11:52:30.551:313521) : item=1 name=/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/S01E03.mp4 inode=32250961 dev=00:27 mode=file,644 ouid=plex ogid=plex rdev=00:00 obj=system_u:object_r:nfs_t:s0 objtype=DELETE
type=PATH msg=audit(20/05/17 11:52:30.551:313521) : item=0 name=/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/ inode=32244304 dev=00:27 mode=dir,755 ouid=plex ogid=plex rdev=00:00 obj=system_u:object_r:nfs_t:s0 objtype=PARENT
type=CWD msg=audit(20/05/17 11:52:30.551:313521) : cwd=/
type=SYSCALL msg=audit(20/05/17 11:52:30.551:313521) : arch=x86_64 syscall=unlink success=yes exit=0 a0=0x7f27800b64d8 a1=0x7f2793da40d0 a2=0x7f278b8967ac a3=0xffffff80 items=2 ppid=12493 pid=28205 auid=unset uid=plex gid=plex euid=plex suid=plex fsuid=plex egid=plex sgid=plex fsgid=plex tty=(none) ses=unset comm=Plex Media Scan exe=/usr/lib/plexmediaserver/Plex Media Scanner subj=system_u:system_r:initrc_t:s0 key=(null)

type=CONFIG_CHANGE msg=audit(20/05/17 11:52:30.552:313520) : auid=unset ses=unset op=“updated_rules” path=/mnt/PlexOptimized/TV/Plex Versions/blueberry 106/PJ Masks/S01E03.mp4 key=(null) list=exit res=yes