Time-Offset Interaction with a Holocaust Survivor

28/05/2015 - 12:00


Time-offset interaction is a new technology that enables conversational interaction with a person who is not present, using pre-recorded video statements: a large set of statements are prepared in advance, and users access these statements through natural conversation that mimics face-to-face interaction. The speaker's reactions to user questions are selected by a statistical classifier, using technology that is similar to interactive systems with synthetic characters; recordings of answers, listening and idle behaviors, and blending techniques are used to create a persistent visual image of the speaker throughout the interaction.

A preliminary working system of time-offset interaction has been created with statements recorded by Pinchas Gutter, a Holocaust survivor, talking about his personal experiences before, during and after the Holocaust. Statements were recorded in two rounds; user questions were collected between the rounds through a “Wizard of Oz” system, where live operators select an appropriate reaction to each user utterance in real time. The result of this process is that 95% of newly elicited user questions are addressed by the recorded statements. The automated conversational system is presently undergoing beta testing in a museum. Time-offset interaction will allow future generations to experience a face-to-face conversation with a Holocaust survivor.

This talk will present the concept of time-offset interaction, the underlying language processing technologies, and the process of content elicitation needed to ensure robust coverage. The talk will include a live demonstration of time-offset interaction.