Making choices among alternatives is the fundamental way for us to express our preferences. Effective analysis of such comparison data gives rise to the rating system, which assigns scores to items for evaluation and prediction. With the value captured by a rating system, we can make better and more informed selections. In terms of developing a satisfied rating system and efficiently analyzing comparisons, new opportunities and challenges come forth in parallel with the recent advances of data collection and storage. In this thesis, we discuss the development of a rating system and statistical analysis of comparison data.

As an intuitive way for humans to make decisions, the pairwise comparison is mainly studied. We start by summarizing available rating systems and statistical...[ Read more ]

Making choices among alternatives is the fundamental way for us to express our preferences. Effective analysis of such comparison data gives rise to the rating system, which assigns scores to items for evaluation and prediction. With the value captured by a rating system, we can make better and more informed selections. In terms of developing a satisfied rating system and efficiently analyzing comparisons, new opportunities and challenges come forth in parallel with the recent advances of data collection and storage. In this thesis, we discuss the development of a rating system and statistical analysis of comparison data.

As an intuitive way for humans to make decisions, the pairwise comparison is mainly studied. We start by summarizing available rating systems and statistical models for data consisting of comparisons. Then we study the asymptotic theory of maximum likelihood estimate (MLE) of the Bradley-Terry model given sparse observations. The theoretically justified consistency of MLE provides a guide and support to problems of large-scale network data. After that, we propose a data-driven rating system that aims to achieve higher prediction accuracy. The rating system takes advantage of the AdaBoost and Elo-type rating update to invest more resources on "hard" matches. The proposed method is evaluated to be more accurate than competing rating approaches on the ATP dataset.