Personalized Human-Robot Interaction Based on Multimodal Perceptual Cues

This thesis work addresses the generation of personalized robot behavior to establish long-term human-robot relationships. Socially interactive robots need the same behaviors and capabilities as humans to interact naturally with humans. One of these capabilities is the perception of multimodal -- verbal, visual, and para-linguistic -- cues, and the other is the display of behavior via speech and body language. Usually, people use these cues about some communicative functions and store relevant data over time. Guided by psychological concepts and ideas, the proposed system in this work personalizes robot behavior based on short-term and long-term memory. The selection of relevant information for personalized robot behavior is of paramount importance. Omitting irrelevant data in each step reduces the information space tremendously and leads to less processing time. This process is based on previous experiences and the current perception of the available information. A mental model for each interlocutor is generated based on the data extracted from each human interaction partner. Personalized robot behavior is generated with three behavioral components - speech, gestures/postures, and facial expressions. A dedicated speech generation system has been implemented, catering to short-term and long-term personalized interactions. Several facial expressions and gestures/postures have been implemented on the robots - Robin and Emah. Experiments with the two humanoid robots have shown promising results for various instances of interaction in an episodic manner. The experiments demonstrated the correct realization of the interaction partner over time. The system proposed has shown improved engagement and long-term fluid relationship between a returning human interaction partner and the robot.