Hasselt UIMS: a tool for describing multimodal interactions with composite events

Abstract

Implementing multimodal interactive systems with eventdriven tools requires splitting the code across multiple event handlers, which greatly complicate programmers’ work. To alleviate this complexity we propose to extend the capabilities of existing event-driven tools so that instead of detecting user events as if they were independent, they are able to detect sequences of semantically related events defined by programmers. Such extended capability can only be possible with notations for (1) defining event patterns, and for (2) binding these patterns with one or multiple event handlers, and (3) specifying human-machine dialogs. This paper presents Hasselt UIMS, a tool that supports the creation of multimodal interfaces with these notations.