THESIS
2023
1 online resource (x, 39 pages) : illustrations (chiefly color)
Abstract
Pruning is a common technique used to reduce the size of convolutional neural network
(CNN) models by removing unimportant weights or channels. It can result in a smaller,
more compact model that is easier to deploy on low-power devices such as mobile phones
or embedded systems. In recent years, the size of the CNN models keeps growing and
leads to a large design space for pruning, making it challenging to find the optimal pruning
policy. Therefore, hand-crafting pruning methods become time-consuming and expensive.
To increase the practicality of pruning, it is important to automate the pruning
process. Bayesian optimization (BO) has recently emerged as a competitive algorithm for
auto-pruning due to its robust theoretical foundation and high sampling efficiency. However,
the increased...[
Read more ]
Pruning is a common technique used to reduce the size of convolutional neural network
(CNN) models by removing unimportant weights or channels. It can result in a smaller,
more compact model that is easier to deploy on low-power devices such as mobile phones
or embedded systems. In recent years, the size of the CNN models keeps growing and
leads to a large design space for pruning, making it challenging to find the optimal pruning
policy. Therefore, hand-crafting pruning methods become time-consuming and expensive.
To increase the practicality of pruning, it is important to automate the pruning
process. Bayesian optimization (BO) has recently emerged as a competitive algorithm for
auto-pruning due to its robust theoretical foundation and high sampling efficiency. However,
the increased size of CNN models leads to a higher dimension of the BO searching
space. As BO’s performance deteriorates significantly due to the curse of dimensionality,
it is not suitable for the auto-pruning tasks of modern CNN models.
To address this issue, we introduce a novel clustering algorithm that reduces the design
space’s dimensionality based on the statistics of the CNN models, thus accelerating
the search process with little loss in accuracy. Furthermore, we propose a rollback algorithm
that allows us to recover the high-dimensional design space to make up for the loss
caused by clustering and achieve higher pruning accuracy. We conduct experiments on
modern CNN models, including ResNet, MobileNetV1, MobileNetV2, and VGG. The results demonstrate that our method significantly improves BO’s convergence rate when
pruning deep CNNs without increasing the running time. Moreover, We have made our
codes open source, which can be accessed at https://github.com/fanhanwei/BOCR.
Post a Comment