“HiStory” – Leveraging human perception for effective performance in video retrieval tasks on mobile devices

Abstract

The objective of this MSc project is to explore the use of mobile platforms, such as touch screen equipped smart phones and tablets, for video retrieval applications and produce and evaluate an innovative prototype.The MSc project builds upon the research undertaken for a preceding (experimentation) project. That work validated the viability of complex thumbnail based interfaces for mobile platforms. It provided a strong foundation for the technical parameters such as thumbnail size, quantity, and layout. In addition interesting findings in the domain of human perception abilities and how they pertain to the field of video retrieval were also documented. Naturally, this work also led to a number of new research questions which form the basis of the MSc thesis.

More specifically the MSc thesis treats two interrelated tasks. The first is to perform a more extensive study and analysis on the effects of physical user interaction, in this case touch gestures, on the performance of video retrieval related tasks; a question that was directly derived from the findings of the experimentation project. The second will be to utilise these findings, along with the previous findings in order to design, implement, test, and evaluate a thumbnail based video browser for mobile platforms. The aim of this browser will be to take full advantage of the interaction characteristics of the platform and what we have learnt of the abilities of human perception. The browser, which is called “HiStory” (Hierarchical Storyboard), will provide the users with a seamless and dynamic overview of the content of a video while giving them the option to control the granularity of the layout therefore allowing for effective Known-Item-Search task solving.

With regard to the evaluation of the browser, its effectiveness will be measured by conducting a user study, which will gather quantitative data on accuracy and speed, and qualitative data in the form of live feedback and questionnaires. Also taken into account are the evaluation methods stipulated by the international academic community involved in the field of video retrieval. The final implementation will be submitted to the Video Browser Showdown 2013 in order to gather expert feedback in addition to that gained by the user study.