Video enhancement using reference photographs

Abstract

Digital video cameras are becoming commonplace in many households, but they still leave something to be desired in terms of image quality. Their poor light sensitivity make images noisy and blurry, and internal storage bandwidth limits the frame resolution. We present a technique for enhancing a low quality video sequence, using a set of high quality reference photographs, taken of the same scene. Our technique generates a high quality frame by copying information from the photographs in a patch-wise fashion. The copying is guided by a sparse set of reliable correspondences between the video frames and photographs. Our technique is purely image-based, and does not require depth estimation. A robust descriptor is employed for establishing valid matches between the video frames and the photographs. Then, the geometric transformation is estimated between every corresponding patch. With only a few reference photographs, we are able to reduce noise and motion blur, and more important, increase resolution by a factor of 6 (see Fig. 1).