2018-03-27, 17:45
Introducing an Intro, Outro Skip Feature for Video Playback on Kodi
Name: Shiqi Dong
Forum/E-mail: stev_end/[email protected]
Summary:
The project will focus on introducing a feature that allows users to skip the intro and outro of a video. Since different shows/videos will have different lengths for their intros and outros (or maybe not even have any intros/outros), a machine learning approach will be implemented. The model will focus on analyzing audio cues as well as video frames to determine intro and outro sections.
How will I achieve this:
Introductions can be determined via audio cues such as an intro theme music or perhaps a sound that only appears during the beginning of the show. For example, many sitcoms that do not have intro themes often have an intro sound that fades as the characters begin to interact, while crime dramas start with an intro theme. Similarly in TV shows, most intros are fixed length, and remains that way for the entirely of the show across different episodes. Thus, another solution is to detect the first few intro frames, fast forward that fixed length of time, and detect and verify that the last few frames of the intro is correct. Outros can be determined similarly via audio cues, but another method to determine the outro of a media playback is the video frames during the outro. Most traditional content include a credits screen that is very text heavy (especially in contrast to the content before). A machine learning model can be implemented to analyze the text content (how many words, how many of those words are names or professions). For example, the model can refer to a dictionary for media industry professions, or an IMDB list of actors present in the show. If those keywords appear multiple times within a span of 3-5 seconds, we can make an accurate prediction that the scene is an outro.
What will the project focus on:
The project will focus on detecting the intro and outro sequences of a video and allowing the users to skip it. The first focus will be on analyzing the video – breaking a video down to its basic info (video size, length, fps etc.), its audio track and the individual frames that compose the video – as well as finding an efficient way to minimize and store that info in text. The next focus will be on detecting the intro via audio cues. This can be either via the theme song, an intro sound, or the lack of an audio track for the beginning of a show. A large quantity of this intro data will be manually sifted through to determine the average length of an intro, how frequent an intro theme appears, and whether the video for the intro remains consistent for the same show across different episodes. The next focus will be applying the audio cue solution to the outros and determine its effectiveness. Should the audio method be sufficient, the next focus will be on refining that. If not, I aim to provide a solution that analyzes the content of the frame, specifically how much text it contains and the length that the screen is exclusively black. The final focus, if time permitted is to retrieve data from media content aggregators such as IMDB. This data can include a list of professions within the entertainment industry, and a list of top 1000 actor/actresses for example. Text can be extracted frame by frame, and if multiple succeeding frames contains a high number of those keywords (professions, actor names), then it can be predicted that such a screen is an outro screen.
Benefits:
This project will benefit users of Kodi who watch video content presented in a traditional manner, such as TV shows, movies, documentaries, etc. That is, videos that contain an intro and an outro.
Goals:
The main goal will be to implement an addon for Kodi will allow for the features mentioned above. As mentioned above, I hope to achieve this via a machine learning solution, such that it can be adapted to multiple types of media content automatically with some human input. If the machine learning approach proves to be far too difficult, or presents a non-satisfactory solution to the issue, I hope to implement a manual approach.
What does it touch in Kodi:
This software will be an addon for Kodi, so it will be using Kodi hooks to provide the functionality.
Requirements:
This project can be completed with Python or Go as the main language. It will most likely be implemented using Python as that is the primary language used for Kodi addons. A proprietary machine learning model will be implemented to allow better fine tuning. This will be implemented in Python
Possible mentors:
----------------------------
Additional
----------------------------
Name: Shiqi Dong
Forum/E-mail: stev_end/[email protected]
Summary:
The project will focus on introducing a feature that allows users to skip the intro and outro of a video. Since different shows/videos will have different lengths for their intros and outros (or maybe not even have any intros/outros), a machine learning approach will be implemented. The model will focus on analyzing audio cues as well as video frames to determine intro and outro sections.
How will I achieve this:
Introductions can be determined via audio cues such as an intro theme music or perhaps a sound that only appears during the beginning of the show. For example, many sitcoms that do not have intro themes often have an intro sound that fades as the characters begin to interact, while crime dramas start with an intro theme. Similarly in TV shows, most intros are fixed length, and remains that way for the entirely of the show across different episodes. Thus, another solution is to detect the first few intro frames, fast forward that fixed length of time, and detect and verify that the last few frames of the intro is correct. Outros can be determined similarly via audio cues, but another method to determine the outro of a media playback is the video frames during the outro. Most traditional content include a credits screen that is very text heavy (especially in contrast to the content before). A machine learning model can be implemented to analyze the text content (how many words, how many of those words are names or professions). For example, the model can refer to a dictionary for media industry professions, or an IMDB list of actors present in the show. If those keywords appear multiple times within a span of 3-5 seconds, we can make an accurate prediction that the scene is an outro.
What will the project focus on:
The project will focus on detecting the intro and outro sequences of a video and allowing the users to skip it. The first focus will be on analyzing the video – breaking a video down to its basic info (video size, length, fps etc.), its audio track and the individual frames that compose the video – as well as finding an efficient way to minimize and store that info in text. The next focus will be on detecting the intro via audio cues. This can be either via the theme song, an intro sound, or the lack of an audio track for the beginning of a show. A large quantity of this intro data will be manually sifted through to determine the average length of an intro, how frequent an intro theme appears, and whether the video for the intro remains consistent for the same show across different episodes. The next focus will be applying the audio cue solution to the outros and determine its effectiveness. Should the audio method be sufficient, the next focus will be on refining that. If not, I aim to provide a solution that analyzes the content of the frame, specifically how much text it contains and the length that the screen is exclusively black. The final focus, if time permitted is to retrieve data from media content aggregators such as IMDB. This data can include a list of professions within the entertainment industry, and a list of top 1000 actor/actresses for example. Text can be extracted frame by frame, and if multiple succeeding frames contains a high number of those keywords (professions, actor names), then it can be predicted that such a screen is an outro screen.
Benefits:
This project will benefit users of Kodi who watch video content presented in a traditional manner, such as TV shows, movies, documentaries, etc. That is, videos that contain an intro and an outro.
Goals:
The main goal will be to implement an addon for Kodi will allow for the features mentioned above. As mentioned above, I hope to achieve this via a machine learning solution, such that it can be adapted to multiple types of media content automatically with some human input. If the machine learning approach proves to be far too difficult, or presents a non-satisfactory solution to the issue, I hope to implement a manual approach.
What does it touch in Kodi:
This software will be an addon for Kodi, so it will be using Kodi hooks to provide the functionality.
Requirements:
This project can be completed with Python or Go as the main language. It will most likely be implemented using Python as that is the primary language used for Kodi addons. A proprietary machine learning model will be implemented to allow better fine tuning. This will be implemented in Python
Possible mentors:
----------------------------
Additional
----------------------------
- Tell us about the computer(s) and devices you have available for working on your SoC project?
- Desktop with Windows 10 / Ubuntu / Debian
- Raspberry Pi 3, Google Chromecast, Android phone for testing
- Desktop with Windows 10 / Ubuntu / Debian
- When did you first start programming?
- I first stated programming in Grade 10. The first language I used was Java, and then subsequently Python.
- I first stated programming in Grade 10. The first language I used was Java, and then subsequently Python.
- Are you a user of Kodi? When did you first start using it?
- I have been introduced to Kodi before by a friend but I have not used it
- I have been introduced to Kodi before by a friend but I have not used it
- What do you primarily use Kodi for?
- Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
- I have not contributed to open source projects before
- I have not contributed to open source projects before
- If you have not contributed to open source projects, do you have other code we can look at?
- What sorts of programming projects have you done on your own time?
- Some of the early projects I have worked on are sudoku solvers and generators, as well as a simple snake game.
- I have also begun learning to use Unity and C# to create a simple platformer.
- Some of the early projects I have worked on are sudoku solvers and generators, as well as a simple snake game.
- How much time do you have available, and how would you plan to use it?
- I will have at a minimum 60 hours per week available. Over the summer I will also be taking a few classes.
- I will have at a minimum 60 hours per week available. Over the summer I will also be taking a few classes.
- Do you plan to have a job or study during the summer in conjunction with Summer of Code?
- I will be taking two classes in the summer.
- I will be taking two classes in the summer.
