Plex is saying the same file exists twice

Server Version#: 1.19.5.3035
Player Version#: 4.38.1

See attached XML and logs, it lists a reference to the same file twice. This shows up in the UI as a (2) as if there are multiple playback options.

Plex Logs.zip (8.5 MB)

Hey, I included the XML. Check it out, it’s literally the same file referenced twice.

Shouldn’t #3 be “Empty Trash”?

Is Empty Trash not needed as part of the Plex Dance in general, or just for @epheterson’s specific problem?

Can you explain more about what has changed? That’s going to be important for a lot of people.

If I don’t Empty Trash, the library items remain, and Clean Bundles doesn’t have any effect.

Is Optimize Database now doing other things automatically? I didn’t think its function was required for the Plex Dance.

/confused.

@trumpy81 


If Optimize Database IS doing things, like Empty Trash automatically, that’s not gonna be popular with people like me who do not want the trash emptied automatically. We set that option for valid reasons.

Combine that (possible change) with the Scheduled tasks, which include Optimize Database every week, and I could end up with a lot of deleted and newly added items I don’t want.

The tips and tricks page for the Plex Dance doesn’t show a change in the procedure.

2 Likes

Thank you for the clarification. I had the same alarm as @leelynds.

@trumpy81 I don’t think that workaround is sufficient (haven’t tried it) because this keeps happening on new entries. Not all, but many.

We need to find a solution that prevents it from occurring for new downloads.

Are you downloading straight to the Library directory, or moving the file after it has completed?

Interesting idea, I do have that library scanning the download directory, and the final directory (to handle cases where the move doesn’t happen correctly). This may have recently started leading to this bug, although it’s not showing paths from both directories it may have a bug updating and removing an entry if two are identical.

I’ll try removing the download folder from the library to see if this stops, but this is a recent regression in Plex server (my settings haven’t changed in at least a year).

It’s not a ‘regression’
It’s a miracle this hasn’t happened up to now.

Don’t call Plex.

Call:
Ripley
or
The Vatican

'Cause what we have right here is Believe or Not - or a prelude to The Second Coming.
Pick one.

My MCEBuddy monitors download areas 'cause it’s smart enough to sit quietly until they actually arrive - then it renames them, sucks the 608s out of 'em (if present), turns 'em into srt files - names them too - and puts them in a Valid Library Type in the Plexiverse.

Plex isn’t smart enough to do that.
Don’t let it try.

Hey @JuiceWSA, I respectfully disagree that Plex isn’t smart enough to handle this, it’s clearly a bug if Plex references the same file twice on the same item, and by definition this is a regression if it never occurred before.

I also use Sonarr to rename/move/clean downloads and have never had issues before.

To be clear it’s referencing identical file paths, not the download directory and the final directory, and the file isn’t in two places at once except for maybe at the instant of the move (if it does safe move via copy/delete).

@trumpy81 I think you’re confusing the issue, at the time the download is occurring the final extension isn’t appended until it’s complete. Plex has never accidentally imported an incomplete download.

I don’t see how any of these steps make it less of an issue that Plex has changed behavior to allow multiple instances of identical paths on library items.

I do appreciate the help with suggestions (we’ll have to see if this indexing change even works), but Plex should collapse identical file entries for an item into one instead of maintaining two. This only started days ago, it’s likely a change in the last server update or one before.

https://support.plex.tv/articles/201543057-why-is-some-of-my-content-not-found/

Copy In Progress

Another thing that can cause issues is if you’re copying content to the drive and directly into the content location. In such a situation, when the content copy first starts, the Plex Media Server scanner can note the change and then wait 60s. If nothing else has changed at that point, it will perform a library scan.

However, if your content is copying and takes longer than a minute, then it’s possible for a situation to occur where the scan occurs before the copy finishes (and thus the content is not ready/available for the scan). Subsequent scans would not actually look in the directory because it doesn’t appear that the directory changed since the previous scan.

The easy solution for that is to not copy the content directly to your content location on the drive. Instead, copy it to another, temporary or “staging” location/folder on the drive first and then when the copying is completed, move it into the standard content location.

Thems da rules - ignore them at your own peril.

I agree the behavior you’re seeing is weird. It shouldn’t show up in Plex with the same path twice, even if you’re doing something else weird.

From a filesystem semantics perspective, that’s actually less “safe”. A single-step “atomic” move is instantaneous and thus “safer”.

With a “move”, the file will be 100% there before Plex is notified to scan. That’s ideal because when Plex goes to scan all of the files will be there, won’t be locked, can be read and probed, etc.

With a “copy”, Plex might be notified before the full file is available, possibly while it’s still locked for the copy, etc. Plex might not be able to scan the file. It might get notified again when the copy completes.

(I’m not sure how Plex interacts with the filesystem change APIs. It might take steps to avoid this.)

Edit: HAH, I was just going to say 
 I think there’s a help doc 
 which also answers my question about how Plex mitigates potential issues.

It’s definitely not intuitive.

@trumpy81, @JuiceWSA and @Volts, I do appreciate the extra information, but I persist that this is a bug and regression.

The solution @JuiceWSA shared is for if I’m having issues with content not being found, which is not my problem. To solve the problem that article mentions, another library scan would fix it, but in my case subsequent scans don’t fix it.

It’s not a big deal if the recommendation resolves my issue (and honestly isn’t a big deal in any case), but this is an easy bug to fix and was only recently added.

I agree that it seems like a bug for Plex to create multiple media file entries with the same path.

In the example you gave they have 100% the same path. They’re both marked available.

Even if you were doing “bad” things with files, or kicking puppies, I can’t think of a reason that Plex should be doing that.

I can think of reasons that it MIGHT do that, but I agree they seem like bugs.

If you can make it reproducible, maybe the Plex folks would be interested.

Obviously if everything is working when following the supported processes, that’s great news.

Thanks for seeing my side, @Volts. It’s very reproducible, but I may never see it again now that I removed the other directory. I’d be happy to add it back if Plex devs want to get more info or logs.

I’ll try and remember to chime back in here if removing the second directory worked, thanks everyone!

Hey all, so unfortunately it’s still happening sometimes even with the downloads folder not being indexed.

I wonder if it’s worth trying the latest (1.20.0.3125) PMS, and either upgrading your library or creating a test library. That has big changes to Scanner & Agent, and it would probably be easiest to get attention for the “new” version too.

Are the download and storage locations on the same disk? When the download is complete, is the file “instantly moved” or does it take a while (more than a minute?) to copy between locations?

If current PMS doesn’t solve it, it’s probably worth catching Plex logs of this happening.

I updated to the latest automatically last night, so we’ll see. About the copy vs. move, from what I read if Sonarr sees that the download and final directories are in the same filesystem, it’ll do a move instead of a copy. I’ve verified Sonarr sees the whole filesystem, so potentially, but I’m not sure how to verify in the logs even with debug/trace enabled.