Multimodal Interaction with Mobile Devices: Fusing a Broad Spectrum of Modality Combinations

Beschreibung

This dissertation contributes to the fields of intelligent user interface design, mobile and pervasive computing, and multimodal interaction. The work presents a multimodal architecture for use in mobile scenarios such as shopping and navigation. It also analyses a wide range of feasible modality input combinations for these contexts. For this purpose, two interlinked demonstrators have been designed for stand-alone use on mobile devices. Of particular importance is the design and implementation of a modality fusion module capable of combining input from a range of communication modes like speech, handwriting, and gesture. The described implementation is able to account for confidence value biases arising within and between modalities and also provides a method for resolving semantically overlapped input. Tangible interaction with real-world objects and symmetric multimodality are two further themes addressed in this work. Finally, the results from two usability field studies provide insight on user preference and modality intuition for different modality combinations, as well as user acceptance for anthropomorphized objects.

Autor*in

Rainer Wasinger

Themen in »Multimodal Interaction with Mobile Devices: Fusing a Broad Spectrum of Modality Combinations«

Mobile computing Modality fusion Multimodal interaction Tangible interfaces

Stimmen zu »Multimodal Interaction with Mobile Devices: Fusing a Broad Spectrum of Modality Combinations«

Details

Link teilen