ISSN:
1572-8412
Keywords:
natural language interfaces
;
multimodal HGI
;
referent specification
;
deixis
;
gestures
;
referent identification
Source:
Springer Online Journal Archives 1860-2000
Topics:
Computer Science
,
Media Resources and Communication Sciences, Journalism
Notes:
Abstract During face-to-face communication, the dialog partners can see and hear each other. Each speaker produces a variety of phenomena parallel to speech. Some of them, e.g. intonation, are coded vocally, others are coded by motor responses (facial expression, gestures, etc.). If human-computer-interaction (HCI) tries to mimic this situation, at least some non-verbal phenomena have to be integrated into natural language input and output. A multitude of new devices (mouse, joystick, touch-screens, etc.) have enabled this transition to multimodal HCI. Gestures which illustrate the content of the verbal message are especially suitable for integration into HCI. A relevant subset of them is pointing gestures, which specify elements of the visual context. They are performed frequently because their use shortens and simplifies the verbal output. As an illustration of these considerations, the NL dialog system XTRA (University of Saarbrücken) is presented. It allows reference to elements of a tax form by the combination of textual input and simulated pointing gestures. In order to explore the regularities of this “form deixis,” an experiment has been carried out within the framework of the XTRA-project. Furthermore, its results were taken for an evaluation of the currently used simulation technique. Michael Wille is a researcher in the AI laboratory at the University of Saarbrücken. He has studied computer science, economics and cognitive psychology. he has worked on expert systems for SIEMENS (hardware diagnosis). His master's thesis (1989) was called “Evaluation and Extension of a Module for the Simulation and Analysis of Pointing Gestures.” His main research interest is multimedia interaction.
Type of Medium:
Electronic Resource
URL:
http://dx.doi.org/10.1007/BF00124149
Permalink