Joint development of perception and behavior based on efficient coding theory has been proposed and proved to be an effective approach in the study of eye movements. Disparity vergence, one type of eye movement, is believed to achieve robustness and accuracy by the cooperation between fovea and periphery. Complicated realistic environments, where objects in different depths project conflicting disparity information on different regions of the retina, cannot be handled well by previous joint development models of vergence control, which combine information from fovea and periphery in parallel.

Here, we propose a hierarchical approach to integrating foveal and peripheral vision for vergence control. The model consists of two levels. At the bottom level, three options receive input f...[ Read more ]

Joint development of perception and behavior based on efficient coding theory has been proposed and proved to be an effective approach in the study of eye movements. Disparity vergence, one type of eye movement, is believed to achieve robustness and accuracy by the cooperation between fovea and periphery. Complicated realistic environments, where objects in different depths project conflicting disparity information on different regions of the retina, cannot be handled well by previous joint development models of vergence control, which combine information from fovea and periphery in parallel.

Here, we propose a hierarchical approach to integrating foveal and peripheral vision for vergence control. The model consists of two levels. At the bottom level, three options receive input from the fovea, the inner periphery, the outer periphery of the retina. At the top level, a policy chooses one of the three options for vergence control.

Our experimental results show that the hierarchical model for integration of fovea and periphery provides more accurate and robust vergence control than a previous model where the sensory representation from different scales was combined in parallel. In addition, the hierarchical model shows good performance in dealing with a wide range of situations, such as small objects moving in depth, a lack of content in the fovea, and occlusion. Furthermore, a periphery to fovea process is observed when applying the hierarchical model in vergence control, which is consistent with psychological evidence found by other researchers.

Keywords- stereo disparity; vergence mechanism; reinforcement learning; multiple agents; fovea and periphery