The bug I found is in the code that purges .grab old directories and the issue would arise if same path is used for more than one PMS.
It is possible the scan action is triggering the periodic purge of old .grab directories but is not itself a factor
Would like to establish if we have more than one issue or if it is the bug / issue I discovered where a Plex Media Server believes it is the sole owner of any .Grab directory that may appear in a library.
So far I have seen evidence / logs of the problem from @rjwerth @cosmos359 @schusterplex2 @darcilicious @boartensen
and I know from the detailed ProcMon evidence provided by @rjwerth with the server logs, that it was not the Plex Media Server that failed to find the path that deleted the path. ProcMon showed no system calls to delete the path from the server - so deletion was external
And we have feedback that @GHouse @WesMaher do share the media paths on more than one server - so for their environment, the issue I found may well be the cause.
I have not seen debug logs of the issue from @kelf @dcone @rqc @clong2001 @fredric100 @NetTinkerer @jamesbigler @nagaraj7 @jonmcwilliams @marengo @theamzngq_gmail_com
@garyjacek @f1nchy @nfrerichs1 @david-plex-diskstation @lightvox @hutchinsw @natemac @coolcube
So I need to know from everyone if the media paths do appear in libraries on more than one server. If this is not the case, let me know and if you have not provided server logs - please do.
If you all do in the library paths point to the same network shares for the media paths on more than one server (and that would fit in with the fact that not everyone is having this issue and also the fact it is mostly NAS media (what normally gets shared), then I can get the development team to take action
Hello @sa2000.
If we have all 2 or more servers couldn’t it be the second server that is deleting the .grab folder?
I mean… I shared my logs from server1 (NAS) where the tv recording started. The other server (shield) started to index… and deleted the folder. That would mean, there are no entries in log of the server1 (NAS). Correct?
And I share my library (on the NAS) with 2 Plex servers, too. (<-edited)If you all do share the media paths on more than one server (and that would fit in with the fact that not everyone is having this issue and also the fact it is mostly NAS media (what normally gets shared), then I can get the development team to take action
That sounds great.
Note my mention above that I’m running two Plex servers that point to the same directories.
I just set up a fresh install on a freenas server and am having the same issues. I recently built a new server because the last version for freenas 11.2 broke my permissions and figured I would start all over. I was surprised to see my new install have these issues with plex right off the bat.
There are no entries in the failing server for deleting the path. It just disappeared. The theory is that the other server that has a library on the same media path, ran the routine that purges .grab directories that are unused - May be that gets triggered when indexing (I have not checked)
I am not referring to library sharing but network shares being used in libraries on more than one server (and each one believes it owns any .grab directory there)
Excellent - sorry missed it
I have reconfigured my system since reporting this issue. But I believe at the time, I was running 3 PMS on a LAN, all of which pointed to the same shared network directories used for plex media libraries.
I reconfigured to 1 PMS using local storage for the media, with plex clients running on 3 networked machines. That resolved the issue.
@sa2000
I have 3 (sometimes 4) servers that all use the same media paths on the NAS.
I disabled the indexing from both the NVIDIA Shield devices and over the past two days, I have had no failing recordings.
I haven’t had a chance to look at the logs from the NVIDIA Shield devices. I’ve only ever looked at the logs from the Synology, but if the other devices were doing the deleting that explains a lot of the behavior.
PMS on Synology starts to record. Shield starts to index, see’s .grab folder and deletes it. PMS on Synology still has an open file handle on the recording, so the file moved to the recycle bin continues to grow.
Would the logs from the other servers show they are deleting the .grab folder?
@sa2000 There are 3 PMS’s that point to my library location.
With debug logging on the other servers not disabled, I would expect DEBUG log entries with text like these:
DEBUG - DVR:Grabber: Deleting "E:\Media\TV\.grab\9675d587c6d902d4d76edb238e5e24e3640d1800".
DEBUG - DVR:Grabber: Cleaned up 1 decrepit directories in 0.2 sec.
Yes I have 2 servers running. One on my synology NAS to record, and one on my Windows 10 primary computer to actually do the serving since it has a much more powerful cpu for transcoding on the fly.
I’m just relieved we finally have a real idea of the cause and possible solutions. Thanks much to everyone who’s chipped in with ideas and did so much work investigating:)
I still get the error. I do have multiple servers pointing to the same media path on my NAS. On this particular machine, it is a mapped drive in windows with admin privs. QNAP NAS runs plex with it mapped locally, and one other windows machine also has a mapped drive to it for plex access.
Yes, I have 3 servers. 2 with read/write access to the recording location, 1 with read-only access
Thanks for your help, it sounds promising!
@sa2000, I enabled library scanning on one of my secondary servers on the NVIDIA Shield, and then I saw this:
Plex Media Server.log:Jan 12, 2020 21:28:38.738 [30514] DEBUG - DVR:Grabber: Cleaning up orphaned grabs.
Plex Media Server.log:Jan 12, 2020 21:28:38.794 [30514] DEBUG - DVR:Grabber: Deleting "/storage/LCMD-DATA/Media/TV Shows/.grab/326d885401051c152a92a2b830b4d741654cf399".
Plex Media Server.log:Jan 12, 2020 21:28:39.161 [30514] DEBUG - DVR:Grabber: Cleaned up 1 decrepit directories in 0.4 sec.
So, yeah, it looks like the secondary server is cleaning up the .grab folder of the other server. Perhaps there is a way to uniquify the .grab folder based on the server.
I would prefer that PMS never delete .grab, since if something went wrong, I may still want to have the data there to inspect it.
I have 2 servers and from the advice of others I have disabled delete on both servers. My Mac is the main server and the Linux Debian is my DVR as it uses my TVheadend server to pull SAT>IP channels properly. I use to link the DVR directory on both servers and from what others have said, looks like the possible cause of the problem. I have stopped cataloging on my Mac and it works now all recording show and no unwanted deleting. In the future it does make sense to keep the content in the .grab folder from being deleted just move them if recording worked and leave if failed like others have mentioned. You could turn the failed recording into a partial recording so you could decide to re-record if possible at a later date and have a warning that it failed.
Until there is a fix for this and re-design on how the .grab directory is managed / named, I suggest other servers accessing the recording libraries are setup with read-only permissions and only the server doing the recordings is given the full access to the media paths.
I have an alpha build of Plex Media Server 1.18.6 with the fix.
If anyone interested in having it, please let me know giving me platform / binary required
I have been out for awhile. But yes, I have 2 Win10s with PMS and 1 Synology Diskstation with PMS and all point to the same libraries. I will ping you @sa2000 for the bits for 1.18.6 for the two platforms.
I’m using docker on Debian
