Real-time Video-Based View Interpolation of Soccer Events using Depth-Selective Plane Sweeping

Abstract

In this paper we present a novel technique to synthesize virtual camera viewpoints for soccer events. Our real-time video-based rendering technique does not require a precise estimation of the scene geometry. We initially segment the dynamic parts of the scene to consequently estimate a depth map of the filtered foreground regions using a plane sweep strategy. The depth map is indicatively segmented to depth information per player. A consecutive plane sweep is used, where the depth sweep is limited to the depth range of each player individually, effectively removing major ghosting artifacts, such as third legs or ghost players. The background and shadows are interpolated independently. For maximum performance our technique is implemented using a combination of NVIDIA's shaders language Cg and NVIDIA's general purpose computing framework CUDA. The rendered results of an actual soccer game demonstrate the usability and accuracy of our framework.