Into/credits auto-skip needs to happen!
Resurrecting an old thread here, but this really is an obvious feature that is completely doable, especially since Plex is designed, for the most part, to work with “main stream” media. I like the idea of using TVDB etc. to just store the times for intro/credits, but not all rips are the same and it can be done with local media easily enough. So here’s a way to make it happen using dejavu and MoviePy, and an alternate method inspired by Videogrep.
(mostly) automated identification of intro/credits offsets in episodes of a series
- Manually extract the audio for the intro/credits from an episode (This process could be automated by using the intro theme music provided by Plex already. I don’t suppose there’s a similar feature for credits, so that would require some input from the user, just a timestamp)
- For each episode, extract the audio of the beginning and end (the first and last 3-5 minutes of the episode).
- Make a dejavu database containing beginning audio clip and another containing the end audio clip.
- Search the first dejavu database for the intro and the second for the credits, recovering, among other things, the offset of the match, indicating the time that the intro/credits starts in the beginning/end audio files. This can be used to find the offset of the intro and credits in the episode itself.
- With the intro/credits offsets and lengths known, could have Plex add the necessary GUI elements to skip the intro or skip to next episode. If that’s not possible, can use this information to cut the episode apart and use segmented MKV files, all achievable with MoviePy, to achieve the desired behavior. Or simply cut out the intro and credits for all but the first episode of each season and store the intro-less versions alongside the originals.
Steps 2 and after can be entirely automated, and I think there are some clever ways to automate step 1
E.g. one could get the sample intro/credit audio by searching for an identical audio segment in multiple episodes. One way would be to start by breaking up the first and last few minutes of several episodes first into 5 second clips and see which match, then increase the length of the clip to 10 seconds, then 15, then 30, etc. until you’ve found the maximum length clip whose audio fingerprint appears in all the files. The clip length would be adjusted to get it just right. You could even use a SciPy minimizer and a cleverly written function based on the quality of the audio fingerprint match to achieve very accurate results, though that may be overkill since an extra half-second won’t kill anybody. I’m not exactly sure what the process would look like in terms of the creation and use of dejavu databases, but I’m sure it’s possible.
Both dejavu and MoviePy are python libraries, and the basics of what I describe here would be easy to achieve. I imagine some fine tuning and testing would be necessary in order to achieve a fully automated solution, but that’s also very doable.
Of course, dejavu may not be the ideal too for this, but I think audio fingerprinting is the way to go.
A subtitle-based method
Videogrep is a clever tool that makes super-cuts of videos based on their accompanying subtitles. This made me consider that, if you have subtitles for every episode in a series, and the subtitles have something along the lines of “Intro music playing…” that shows at the beginning of the intro, then you can use that to get the intro/credits offsets, thereby eliminating the need for steps 1-4 above, skipping straight to the Plex GUI implementation for skipping the intro/credits or cutting the files based on these offsets.
Moving forward
I figure someone (who ideally isn’t me) would write this up as a separate Python utility as a proof of concept, and hand that over to the Plex devs so they could implement it right. It’s not completely necessary for Plex to get the extra metadata and playback changes to make this happen, since you could just adjust the files to achieve automatic skipping of intro/credits as desired. But a solution that requires the complicated use of segmented MKV files or results in nearly doubling the space of all your series just to cut out 2 minutes from each episode seems like a poor solution. This should really be a convenience offered to users no matter what, and therefore a completely automated process that runs alongside the, already expansive, background tasks performed by Plex. If Plex were to add this as a proper feature, then the implementation could be, ideally, something like what’s described below.
Intro/credits auto-skip in current Apple TV 4 Netflix app
Here’s how Netflix does it, and it’s phenomenal!
- When a series watching session is started (i.e. the app is launched and an episode is selected and played from beginning) the current episode plays from the beginning, and as the intro begins (which may be after some time because of a cold start), a button appears, “Skip Intro”, so the user need only press the “OK” button to skip past the intro.
- Towards the end of the episode, as the credits start, two buttons appear, “Watch credits” and “Playing next episode in 5…4…3…” (the number counts down), where “Playing next episode” is the default choice and by either pressing “OK” or doing nothing and allowing the countdown to complete, the remainder of the current episode is skipped and the next episode begins.
- If the next episode starts with an intro, it is skipped automatically, so that by either selecting “Playing next episode” or letting it timeout, the user is skipped straight to the end of the intro of the next episode. If the next episode does NOT start with an intro (has a cold start), then the behavior is as in (1).
After using this implementation in the Apple TV 4 Netflix app for several weeks, I have to say it’s EXACTLY how Plex needs to do it. Hope there’s no bogus copyright or anything keeping the implementation from being too similar.
