I only recently discovered that some physical releases have incorrectly pitched audio tracks (sometimes all of them; sometimes only dubbed tracks) due to bad authoring.
It would be neat if Plex could use sonic analysis to somehow crowdsource a comparison tool that would let users know that the content they uploaded on their server is inaccurate. To be honest, this is the kind of feature that I expect from Plex in the future…
Define “incorrect”.
The French audio track on this Canadian release is shifted about -0,72 semitone from what it should be. This means that EVERYTHING sounds lower pitched than it should while still having the proper duration. (Same tempo, but wrong pitch)
If I understand correctly, this is probably because the North American authoring team took the work from an European team who authored the French track for their own “25 fps”/50Hz standard. The North American team probably slowed down the track so that it would fit our 23.976 (or 24) fps standard, but they forgot to correct the pitch while doing so.
This means that I need to find a way to increase the pitch of this specific track by a factor of 1.042708333 while making sure that audio and video stay synced for the 3+ hours of that movie.
How do you go about correcting this situation when you find one now?
Well… I’m not super knowledgeable. This is a true rabbit hole because depending on the version of ffmpeg that one is using, it seems that the handling of DTS-HD MA tracks is different. It truly seems to be a PITA.
Apparently, I could transcode the DTS-HD MA track using the rubberband filter and the aresample filter (“aresample=async=1”). However, I’m not quite sure how to deal with the channel mapping.
I tried the following command, but it desynced once I remixed the FLAC into an MKV, which is why I investigated more. The pitch was corrected thought… I mean… I’m not sure “desync” is the proper word. It’s more like a delay that exponentially gets worse. The lipsyncing is fine at first, than it’s awfully out of sync.
ffmpeg -i "/mnt/8To/Media/Movies/The Lord of the Rings-The Fellowship of the Ring (2001) {edition-Extended Edition} {tmdb-120}/The Lord of the Rings-The Fellowship of the Ring (2001) {edition-Extended Edition} {tmdb-120}.mkv" -map 0:a:2 -af:a:2 "rubberband=pitch=1.042708333:pitchq=quality:formant=preserved,channelmap=0|1|2|3|4|5|6:6.1" -c:a:2 flac "/home/+++/Vidéos/LOTR-1.flac"
I read about how the “channelmap=0|1|2|3|4|5|6:6.1” was probably either superfluous or had the wrong syntax only a few minutes ago. It seems that my final headache mostly relates to understand which feature is implemented on my version of FFMPEG.
Someone on blu-rayDOTcom’s forum started a thread (dead now) about this issue in movie collecting. It’s a known mistake that happens in the industry.
Alliance Atlantis Vivafilm definitely dropped the ball back then. This is insulting because this is only affecting “The Fellowship of the Ring” content. Both remaining films inside the boxset have been properly authored. (Which is a good thing…) Nobody thought about replacing the broken track before publishing the boxset. 
I managed to manually fix the track using the a similar command (there might be mistakes, but those were the options I used…):
ffmpeg -i input.mkv -c:v copy -map 0:v:0 -map 0:a:2 -af "rubberband=pitch=1.042708333:pitchq=quality:formant=preserved,aresample=async=1" -c:a flac -compression_level 12 output.mkv
but I had to make new copies of my original Blu-ray because I had merged both parts together and the timestamps wouldn’t work well with rubberband.
I installed ffmpeg 8 (latest) on my windows PC. The last hurdle was that MKVToolnix doesn’t know what to do with FLAC tracks*, so I also had to use ffmpeg for merging both parts once again.
It’s funny now because I realize I got used to voices being a bit lower in the French version, but they are now a bit higher.
I kinda hope that this kind of nerdy feature will be implemented someday: Some kind of warning being displayed to at least let users know that one of their audio track has an issue. Wouldn’t sonic analysis be good at that?! So many advanced features could be added for people with a Plex Pass…
*FFMPEG can only decode DTS-HD MA, it cannot encode back to it, so I chose FLAC as the fallback codec…
Without a reference of what is supposed to be “correct”, there is no way to detect a pitch that’s slightly off.
Yeah… I guess you are right, because it means that Plex would need to store data somewhere so that the analysis would have something to compare with.
Comparing local tracks between each other would be a good start. If tracks are properly identified with metadata, Plex could locally compare tracks without commentaries nor audiodescription to find big discrepancies or to identify common authoring mistakes like the one I described.
It could even use alternate (local) editions of the same movie to get more comparison points… If most of the primary tracks are very similar, but one or two primary tracks appear to be different yet recognizable, I hope Plex could somehow warn the user and eventually suggest a fix. Maybe it could even fix it on the fly, like VLC can do. (Which wouldn’t require re-encoding.)
Surely there’s a way for analysers to recognize that one track has an obvious shifted audio pitch when compared to the other local audio tracks.
One can dream…
