Haptic Feedback in Virtual Environments: Towards a Multi-modal Interface

Abstract

The goal of virtual environments research is to give the user an experience that is indistinguishable from the real world. Nevertheless, research has mostly concentrated on visual output. Our research has shown that visual feedback limits the realism of the user's experience. In order to allow for an intuitive interaction with the virtual environment, the user interface must be multimodal. In virtual environments, touch is a useful modality, since people rely on touch in real world interactions. Furthermore, devices that can generate haptic feedback became recently available. The research described in this thesis incorporates haptic feedback in virtual environments, thus extending the possible interactions in these environments. First, the use of touch was assessed using passive touch in the domain of cut-out animations and active touch in the domain of curve drawing. Next, the modelling environment, we use in our research, was enhanced with haptic feedback and new interaction techniques were developed. Furthermore, haptic rendering techniques were developed in order to make 3D objects touchable. Since user interfaces cannot be analytically evaluated, two user experiments were conducted in order to evaluate our interaction techniques. The results prove our interaction techniques to be useful. In haptic environments, the user often needs both hands to manipulate devices. This means that a mouse and keyboard cannot be used to give commands. A natural solution is the use of speech to give commands. Indeed, we have shown that speech is a useful modality in a haptic environments. We believe that the use of haptic feedback is an important step towards the creation of an intuitive and flexible multi-modal user interface for virtual environments.