Web-Powered Virtual Site Exploration Based on Augmented 360 Degree Video via Gesture-Based Interaction

Abstract

Physically attending an event or visiting a venue might not always be practically feasible (e.g., due to travel overhead). This article presents a system that enables users to remotely navigate in and interact with a real-world site using 360° video as primary content format. To showcase the system, a demonstrator has been built that affords virtual exploration of a Belgian museum. The system blends contributions from multiple research disciplines into a holistic solution. Constituting technological building blocks include 360° video, the Augmented Video Viewing (AVV) methodology that allows for Web-driven annotation of video content, a walk-up-and-use mid-air gesture tracking system to enable natural user interaction with the system, Non-Linear Video (NLV) constructs to unlock semi-free visitation of the physical site, and the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard for adaptive media delivery purposes. The system’s feature list will be enumerated and a high-level discussion of its technological foundations will be provided. The resulting solution is completely HTML5-compliant and therefore portable to a gamut of devices.