The recent success of Convolutional Neural Networks has drawn extensive researches in both academic and industrial sectors. There are numerous theoretical and experimental researches on deep neural networks. However, the reasons behind their excellent generalization performance remain unknown. In this thesis, we propose a simple method to improve the generalization robustness of the neural network and it provides a better understanding of the neural network during training process. We find that deep neural network is a lazy learner if it is subject to a simple regression problem. We also find that there are two learning phases during training by layer analysis. Then, we study the properties of convolutional kernels through controllable datasets. Lastly, we study the effectivenes...[ Read more ]

The recent success of Convolutional Neural Networks has drawn extensive researches in both academic and industrial sectors. There are numerous theoretical and experimental researches on deep neural networks. However, the reasons behind their excellent generalization performance remain unknown. In this thesis, we propose a simple method to improve the generalization robustness of the neural network and it provides a better understanding of the neural network during training process. We find that deep neural network is a lazy learner if it is subject to a simple regression problem. We also find that there are two learning phases during training by layer analysis. Then, we study the properties of convolutional kernels through controllable datasets. Lastly, we study the effectiveness of weak-learning kernels and the avoidance of the overparameterization effect in deep neural networks by N - x analysis.