Cell segmentation is a critical task in fully automatic computer cytology diagnosis. Overlapping cells pose a major challenge to cell segmentation because of blurred edges and inhomogeneous cytoplasm. There are several existing literatures on merging a variety of features into energy functional to improve the accuracy of overlapping cell segmentation. However, these solutions cannot solve the above problem very well, because most energy functionals are static therein and the relationship between different features is not exploited. In this thesis, we consider the following issues associated with the overlapping cell segmentation problem: 1) system framework of the overlapping cell segmentation, 2) shape prior analysis of cells, 3) dynamic energy functional construction, and 4)...[ Read more ]

Cell segmentation is a critical task in fully automatic computer cytology diagnosis. Overlapping cells pose a major challenge to cell segmentation because of blurred edges and inhomogeneous cytoplasm. There are several existing literatures on merging a variety of features into energy functional to improve the accuracy of overlapping cell segmentation. However, these solutions cannot solve the above problem very well, because most energy functionals are static therein and the relationship between different features is not exploited. In this thesis, we consider the following issues associated with the overlapping cell segmentation problem: 1) system framework of the overlapping cell segmentation, 2) shape prior analysis of cells, 3) dynamic energy functional construction, and 4) efficient algorithm for energy functional minimization. We divide our system framework into six steps: image smoothing, edge detection, mass extraction, nucleus localization, cytoplasm segmentation and cytoplasm refinement. In this thesis, we mainly focus on the cytoplasm segmentation which is the major challenge of overlapping cell segmentation. It is well-known that the shape prior can guide the segmentation process in face of misleading features. We obtain the shape prior of cells using shape alignment technique and propose to use the nuclear norm as a measurement of shape similarity between final cytoplasm and shape prior. To obtain an insightful solution, we propose a novel adaptive energy functional involving shape similarity by adaptively adjusting the weighting parameter of edge energy. We alsp give the insight on designing the adaptive weighting parameter. To show the advantage of adaptive weighting parameter, we compare the performance of our model with static and adaptive weighting parameter. In this thesis, we develop a monotone Accelerated Proximal Gradient algorithm for our non-convex non-smooth problem. To show the importance of shape prior, we compare the performance of our model with and without shape similarity. We evaluate our proposed method using the International Symposium on Biomedical Imaging (ISBI) 2014 and 2015 challenge datasets. Results demonstrate that our method produces competitive accuracy compared to state-of-the-art methods.