In this paper, we propose and evaluate approaches to categorizing Chinese
texts, which consist of term extraction, term selection, term clustering and text
classification. We propose a scalable approach which uses frequency counts to
identify left and right boundaries of possibly significant terms. We used the
combination of term selection and term clustering to reduce the dimension of the
vector space to a practical level. While the huge number of possible Chinese terms
makes most of the machine learning algorithms impractical, results obtained in an
experiment on a CAN news collection show that the dimension could be
dramatically reduced to 1200 while approximately the same level of classification
accuracy was maintained using our approach. We also studied and compared the
performance of three well known classifiers, the Rocchio linear classifier, naive
Bayes probabilistic classifier and k-nearest neighbors(kNN) classifier, when they
were applied to categorize Chinese texts. Overall, kNN achieved the best accuracy,
about 78.3%, but required large amounts of computation time and memory when
used to classify new texts. Rocchio was very time and memory efficient, and
achieved a high level of accuracy, about 75.4%. In practical implementation,
Rocchio may be a good choice.
Relation:
International Journal of Computational Linguistics and Chinese Language Processing (CLCLP) 5(2) : 43-58
