Squeeze Me and I’ll Change: An Exploration of Frustration-triggered Adaptation for Multimodal Interaction

Abstract

Complex 3D interaction in virtual environments may inhibit user interaction and cause frustration. Supporting adaptivity based on the detected user frustration can be considered as one promising solution to enhance user interaction. Our work proposes to provide adaptive assistance to users who are frustrated during their interaction with 3D user interfaces in virtual environments. The obtrusiveness of physiological measurements to detect frustration inspired us to investigate the pressure patterns exerted on a 3D input device for this purpose. The experiment presented in this paper has shown a great potential on utilizing the finger pressure measures as an alternative to physiological measures to indicate user frustration during interaction. Furthermore, the findings in this particular context showed that adaptation of haptic interaction was effective in increasing the user's performance and making users feel less frustrated in performing their tasks in the 3D environment.