Hong Kong is a two literature three language (Cantonese, Mandarin and English) city. The people speak in a mixed-code manner-many English terms are used in the Cantonese or Mandarin utterance. Therefore, a speech system used in Hong Kong should be able to handle these three languages and mixed-language manner. Because of this reason, the focus in the research undertaken for this thesis was to upgrade the speech system, SALSA, form English only to become a trilingual system which can support Cantonese, Mandarin and English, and also support links with Chinese and English words together....[ Read more ]

Hong Kong is a two literature three language (Cantonese, Mandarin and English) city. The people speak in a mixed-code manner-many English terms are used in the Cantonese or Mandarin utterance. Therefore, a speech system used in Hong Kong should be able to handle these three languages and mixed-language manner. Because of this reason, the focus in the research undertaken for this thesis was to upgrade the speech system, SALSA, form English only to become a trilingual system which can support Cantonese, Mandarin and English, and also support links with Chinese and English words together.

Since we did not have access to multilingual resources, the first idea of building a multilingual system was to use the monolingual resources of these three languages combined together. However, this approach did not work. Therefore, we applied cross-lingual recognition in the SALSA system; we used an English model to recognize all three languages. Through our research we showed that cross-lingual recognition can be applied on a small domain recognition tasks. In the trilingual SALSA system, by using this method, the recognition rate of English and Cantonese is over 90% and Mandarin is over 80%.

In addition we also did some work on cross-lingual adaptation. We used an English model as a base model, and used the MAP adaptation technique to adapt a small amount of Mandarin data. The use of this technique showed that without enough training data, using the cross-lingual adaptation method ensures a better performance. Using cross-lingual adaptation, a speech system with small amount of training data.