Engagement has been shown to be a critical factor for effective rehabilitation, as it enhances cortical plasticity within the central nervous system. Engagement with nature can lead to psychological restoration by evoking involuntary attention, as suggested by Kaplan's Attention Restoration Theory (ART). In this thesis, we focus on studying human engagement using multimodal measurements taken when users interact with therapeutic environments. We focus on engagement modulation, quantification, and enhancement.

First, to encourage patient's engagement during rehabilitation, we propose a multimodal robotic system that integrates robotics, virtual reality (VR), eye tracking, and galvanic skin response (GSR) sensor. We first present a gaze-controlled robotic system that combines multiple con...[ Read more ]

Engagement has been shown to be a critical factor for effective rehabilitation, as it enhances cortical plasticity within the central nervous system. Engagement with nature can lead to psychological restoration by evoking involuntary attention, as suggested by Kaplan's Attention Restoration Theory (ART). In this thesis, we focus on studying human engagement using multimodal measurements taken when users interact with therapeutic environments. We focus on engagement modulation, quantification, and enhancement.

First, to encourage patient's engagement during rehabilitation, we propose a multimodal robotic system that integrates robotics, virtual reality (VR), eye tracking, and galvanic skin response (GSR) sensor. We first present a gaze-controlled robotic system that combines multiple control strategies, which can provide a more engaging environment for a wide range of rehabilitative therapies than currently available. Next, we describe a rehabilitation system that can simultaneously monitor and modulate engagement. To our knowledge, this is the first system that estimates engagement using behavioral signals, since most prior systems used physiological signals. Our results show that, consistent with flow channel theory, engagement follows an upside-down U shape with respect to task difficulty. Furthermore, we develop a closed-loop GSR-based dynamic difficulty adjustment (DDA) system, which has been shown to work effectively in enhancing engagement.

Second, we study the calming effect of viewing a Japanese garden on young adults. Past works have shown that exposure to Japanese garden benefits Alzheimer patients. We built a system that enables us to collect synchronized measurements of gaze and multiple physiological signals. Our results suggest that exposure to Japanese garden leads to a greater shift towards parasympathetic neural system (PNS) activity, relative to the control space. Moreover, we explore the effects of exposure to virtual gardens (via photographs and VR), and study the correlation between gaze behaviors and physiological responses. We find that viewing the green areas of the scene evokes relaxation related responses.