For Intro detection, can we not get feedback from users? Everytime a user’s server detects a start time for an intro for a TV Series and and Episode, can’t we have a quick communication back to Plex website that says “I detected an intro for TV Show X - Episode XX - Start(s) and Stop(s)”.
Then, when Plex website gets above a certain level of crowd confidence, then future server requests can just get a start and end times for Intros for each episode, without requiring each server to detect intros individually.
Let’s crowd source this feature and get those immediate Intro detection magic going when new episodes are added to servers where people have already done the heavy lifting for the first time detection of episodes. This should not take up very much data to keep track of and would be an awesome feature improvement.
I would imagine we would detect the start of the playing, minus black screens etc… But it’s a crowd confidence. If we get a 98% ping that intro start time is X, we can use that instead of re-checking. With this improvement, we could maybe have an edit series option that tells it not to use detection from plex website. Or a server level setting that allows a server owner to not get this info from Plex website.
You are forgetting:
Series can come from various sources.
Releases in different countries can have different “jingles”/“title cards” with differing duration stuck in front of the episode.
Same applies when aired on different tv stations.
So the start and end of the intro can already differ.
Then there is the issue of different video frame Rates, depending on country and tv system (PAL vs. NTSC). So the playing time of the episode can differ as a whole, which also affects the placement of the “intro start” and “intro end” marks.
Some tv stations stick a recap in front of episodes, some don’t.
The length of the recap can vary as well between stations.
So further variation of the intro placement.
With all the above, all Plex users then shall get the intro marker placements dictated by the file version which the majority of Plex users has?
We could even use a sound marker for episodes. Get a very simple wave for the first 5 seconds of an episode to make it like a checksum of an episode. So user’s scans would upload a checksum of a first 5 seconds with highs and lows etc… to match an intro detection with. So, if 70% of users have a checksum for X for this episode, but 10% of a different checksum, we could at least get a more accurate crowd confidence that matches the checksum for the start of an episode.
Just shooting some ideas out, but I think the instant availability of Intro detection would far out weigh the possibility of missing or not being quite as accurate as detecting it yourself on a few episodes. But leaving it to a server/library/show setting would help in the cases of not being as accurate.
This is also one of those features that can be rolled out silently, to gather user server’s intro detection info, checksum, start(s) end(s) and do some data checking if these feature would be doable, without rolling it out. Just data collection to power the feature. You could even run the same data back and see What percentage of users that have calculated intros, if we told them to use our data, how accurate would we be? Could totally make this a data informed decision while testing this feature before it even gets released.
What exactly is wrong with the existing info detection? Seems to work just fine for me.
I can’t see this working at all, I think you’re under estimating just how different “the same episode” can be from one source to the next. I doubt you’ll get 70% agreement, and even if you did, it really screws it for the remaining 30%
Current method works great. Don’t get me wrong. I have fairly large libraries though, and my server is spending a lot of time detecting intros. Everytime I check, it’s detecting intros. I run a GTX 1080 on the server for this heavy lifting, but it’s always detecting intros. Feels like that is a lot of GPU waste when maybe, just maybe, we can take learnings from user server owners. It would all ride on if we can determine if episode source 1 is the same as an other’s episode 1 source. I mentioned checksuming the audio of the beginning of an episode. Could be doable and would account for all the formats and source variations.
Intro detection is already audio analysis based so it’s not doing any lifting with your GTX 1080.
You’re suggesting it do analysis to find the start of the file, then more analysis to find the start and duration of the Intro - then submit that timing to a DB somewhere. That’s more work that it does now.
Didn’t know intro detection was solely audio analysis as it is and didn’t rely on gpu if present. Doesn’t change the ask though “Isn’t there some way that we can quickly determine what an intro is, with less work and crowd sourced?”.
Even on a slow computer, Intro Detection should be pretty quick, and should only happen once per episode.
Perhaps something is going wrong! Consider sharing server logs!
Do you feel like something else is happening?
How long does Intro Detection take per episode?
There was a previous issue where if the Conversion Queue was paused, Intro Detection became very slow. Confusingly, you had to start an Optimization task, and then check the Conversion Queue, to see if this was affecting you. I think that’s been resolved.
The process isn’t slow. It’ll hang on an entire season for a few minutes and then move on to the next one. I was doing a lot of bulk processing, dumping large amounts of series into plex folders and was just noticing that the server was doing a lot of intro detection. I didn’t know if that would be impactful for users streaming and just made me think that maybe these steps could be crowd sourced with high level of accuracy and verification while in exploration. I don’t know all the complexities involved or if there would be that much time saving but, from a software engineering background myself, anything is doable. A moderator has already weighed in too about the varying complexities of sources etc, so the question I’d have is if this would save time and would it be pretty accurate. One could argue at a minimum, any time saved not doing more processing, is good for the planet and saving electricity
We can already upload and retrieve marker data for credits, so I don’t see why not add for intro as well.
Performing the analysis to detect intros can require a lot of processing power and take time to complete. This could really cut down on the power and time if someone has already uploaded the intro markers and can be greatly beneficial for helping if a server ever has to be recreated or reinstalled.
The most CPU intensive thing I’ve seen on my Plex server, besides transcoding, is the intro detection. It would be great to have an online intro marker source like we have for credit markers.
This is desperately needed. Seems like a huge waste of resources to force everyone to detect intros locally when crowd sourcing works so well with credits. Any plans from the Plex team to implement this? Since the technology is already there for credits it shouldn’t be too much work to implement the same feature on intros.
From what I understand, they hash the filename, and that’s what’s used as a signature of sorts to compare with other people’s files.
The best alternative way for this is to get and compare audio signatures. So, rather than doing the audio intro detection analysis, it generates an audio signature for the file and then compares that signature to a database.
The problem with hashing, whether using the filename or the file content, is that you’re comparing files that are exactly alike. But people do all sorts of things to their files; they convert them, change tracks, remove tracks, etc. The peaks and lows in an audio waveform stay consistent even after conversion. So you can create a signature where a certain amount of deviation is allowed.
From what I know, Plex’s subtitle sync records something similar, but it only records 1s and 0s when it detects voices.
The problem is, there’s just not enough desire/will to work on something like this because the current way works, albeit not optimally.
To add to this, MusicBrainz can generate audio signatures to thousands of files pretty quickly, so I would implement something similar to that. It’s not doing heavy analysis, it’s really just reading the audio track then comparing that signature to a database.
