In this thesis, we propose a generative topic model for image labeling applications and demonstrate it specifically on the problem of simultaneous multi-class object recognition and segmentation. Our proposed model has been inspired by some recently proposed topic models, such as latent Dirichlet allocation (LDA) and correlated topic model (CTM). However, borrowing such language models directly for vision applications is inappropriate due to their “bags of words” assumption, which implies that each word is drawn independently given its latent topic. To relax this restrictive assumption, we propose an extended topic model called correlated topic random field (CTRF) by modeling the latent topics of the patches in an image as a Markov random field (MRF). Due to the difference in nature of...[ Read more ]

In this thesis, we propose a generative topic model for image labeling applications and demonstrate it specifically on the problem of simultaneous multi-class object recognition and segmentation. Our proposed model has been inspired by some recently proposed topic models, such as latent Dirichlet allocation (LDA) and correlated topic model (CTM). However, borrowing such language models directly for vision applications is inappropriate due to their “bags of words” assumption, which implies that each word is drawn independently given its latent topic. To relax this restrictive assumption, we propose an extended topic model called correlated topic random field (CTRF) by modeling the latent topics of the patches in an image as a Markov random field (MRF). Due to the difference in nature of text and images, we introduce a global appearance model which generalizes from a discrete vocabulary space for text to a continuous feature space for images. Furthermore, we introduce a local appearance model to adaptively represent the data-dependent features for accurate segmentation. Inference in the CTRF model is based on an integrated expectation maximization (EM) framework. Extensive experiments performed on benchmark data sets demonstrate the success of CTRF for simultaneous multi-class object recognition and segmentation.