tailieunhanh - Báo cáo khoa học: "Confirmation in Multimodal Systems"

Systems that attempt to understand natural human input make mistakes, even humans. However, humans avoid misunderstandings by confirming doubtful input. Multimodal systems--those that combine simultaneous input from more than one modality, for example speech and gesture--have historically been designed so that they either request confwmation of speech, their primary modality, or not at all. Instead, we experimented with delaying confirmation until after the speech and gesture were combined into a complete multimodal command. . | Confirmation in Multimodal Systems David R McGee Philip R Cohen and Sharon Oviatt Center for Human-Computer Communication Department of Computer Science and Engineering Oregon Graduate Institute . Box 91000 Portland Oregon 97291-1000 dmcgee pcohen oviatt @ ABSTRACT Systems that attempt to understand natural human input make mistakes even humans. However humans avoid misunderstandings by confirming doubtful input. Multimodal systems those that combine simultaneous input from more than one modality for example speech and gesture have historically been designed so that they either request confirmation of speech theừ primary modality or not at all. Instead we experimented with delaying confirmation until after the speech and gesture were combined into a complete multimodal command hl controlled experiments subjects achieved more commands per minute at a lower error rate when the system delayed confirmation than compared to when subjects confirmed only speech hl addition this style of late confirmation meets the user s expectation that confirmed commands should be executable. KEYWORDS multimodal confirmation uncertainty disambiguation Mistakes are inevitable in practice conversation breaks down almost instantly in the absence of a facility to recognize and repair errors ask clarification questions give confirmation and petform disambiguation. 1 INTRODUCTION We claim that multimodal systems 2 3 that issue commands based on speech and gesture input should not request confirmation of words or ink Rather these systems should when there is doubt request confirmation of theữ understanding of the combined meaning of each coordinated language act. The purpose of any confirmation act after all is to reach agreement on the overall meaning of each command. To test these claims we have extended our multimodal map system QuickSet 4 5 so that it can be tuned to request confirmation either before or after integration of modalities. Using QuickSet we have .