Pose modality, referring to human poses and robot poses, is a powerful cue to reveal humans’ internal status and also a communication channel that naturally arises in motion planning for the robots. However, compared to other modalities, e.g., audition, vision, poses have been largely overlooked even though there is an increasing interaction demand, either from human poses to the robot’s perception, i.e., pose inference, or from robot poses to the human’s perception, i.e., pose generation.

The thesis considers typical HRI scenarios and proposes novel models to meet such demand. To infer humans’ cognitive/affective status, we propose a corpus-based state transition model to sense engagement dynamics, and a learning-based multi-modality fusion models to estimate emotion intensity...[ Read more ]

Pose modality, referring to human poses and robot poses, is a powerful cue to reveal humans’ internal status and also a communication channel that naturally arises in motion planning for the robots. However, compared to other modalities, e.g., audition, vision, poses have been largely overlooked even though there is an increasing interaction demand, either from human poses to the robot’s perception, i.e., pose inference, or from robot poses to the human’s perception, i.e., pose generation.

The thesis considers typical HRI scenarios and proposes novel models to meet such demand. To infer humans’ cognitive/affective status, we propose a corpus-based state transition model to sense engagement dynamics, and a learning-based multi-modality fusion models to estimate emotion intensity. Results show that the models enable robots to be significantly more intelligent in handling complex interactions with peripheral interference, as well as in perceiving human partners under incomplete observations. To generate robot poses, we propose mathematical models for robots to simulate humans’ behaviour and approximate human poses with physical constraints. Evaluation suggests that the generated poses significantly improve interaction transparency and affect humans’ perception towards the robot capability and the interaction outcomes. We further employ Learning from Demonstration (LfD) to scale up the pose generation, enabling robots to robustly and efficiently learn from demonstrated poses. Results show the potentials of LfD in learning from incomplete demonstrations and in generalizing demonstrated poses to new scenarios.

To sum up, this thesis presents the computational models for cognitive/affective inference from human body poses, and explores the generation of human-like poses in HRI. The thesis takes the first step to systematically investigate the pose modality as a communication channel in HRI.