What do you think about Music deduplication by Plex?

So I recently noticed that within my music library there are enough dublicated music songs, wouldnt that be a cool feature for plex to deduplicate them but at the same time keep the linkage within albums and playlists so the user doesnt notice the difference rather than having more space on the hdd left?

What do you thik about it? Does it make sense to you?

0 sense. Why would you have duplicated songs?

E.g. if you buy 2 mixed albums and they contain some overlapping songs?

@jaSunny said:
E.g. if you buy 2 mixed albums and they contain some overlapping songs?

They could be still differently enough, so deletion of one of them is not desirable.

  • they could have different sound (different mastering, different source of material)
  • they could be different ‘mixes’ (maxi, vs. album vs. radio vs …) (the ‘mix’ is not always noted in the track title)
  • one of them could be ‘cut’ to fit better within the narrow time frame of a ‘Best Of’ compilation

These are all things which can’t be assessed just by looking at the metadata.

I agree though if there was a filter for ‘duplicated’ songs it’d be easier to find them for consideration.

The space savings however are going to be minimal. I have ~300 albums including 20 multi CD compilation albums all encoded in FLAC and the whole lot comes to ~130GB. The MP3 copies come to about 35GB.

In terms of de-duplication what would be better is if Plex could see two identically named files with identical metadata in the same directory but one an MP3 and the other a FLAC and not put two versions of the track in it’s database, in the same way that you can have HD and SD versions of a movie/TV show.

@Buzzme: I think thats also an interesting idea.

@OttoKerner: Thats true. However one can also work with CRC checksums (hashes) or other techniques (ref). Also, different version of the same song should be reflected in the file size especially when they are of different length / duration

Not trying to hijack the thread, but this falls way down the priority list for me. Baby steps first. How about gapless playback? Dedup is for when we’re jogging.

@mdnitoil: Thanks, I appreciate your input. Sure, gapless playback is a nice feature but since its already within the top-feature-request list I think its not necessary to question it anymore :slight_smile:
My intention of this thread is more to get a feeling if my idea & need of dedup is shared by the community or if its rather a single standing point of view.

In the process of combining folders and between my itunes folder and other folders, I have many duplicates - sometime the same song in different formats such .wma, wav, and mp3.

I’d be interested.

Alright, thats good to hear.

So, I created a feature request:

Typically dedup is done at the file system (or lower) level. If you have a Linux system, this would be available already, but would apply across all files, not just music. There is even dedup available for QNAP NAS (not officially on mine), but it’s still available and working. I’ve read, that dedup is not very good across media files in general, unless there are repetative sidecar files, true multiple copies (e.g. for editing purposes) etc.