One of the most popular image representations for image classification is based on the bag-of-words (BoW) features. However, the number of keypoints that need to be detected from images to generate the BoW features is usually very large, which causes two problems. First, the computational cost during the vector quantization step is high. Second, some of the detected keypoints are not helpful for recognition. To resolve these limitations, we introduce a framework, called iterative keypoint selection (IKS), with which to select representative keypoints for accelerating the computational time to generate the BoW features, leading to more discriminative feature representation. Each iteration in IKS is comprised of two steps. In the first step some representative keypoint(s) are identified from each image. Then, the keypoints are filtered out if the distances between them and the identified representative keypoint(s) are less than a pre-defined distance. The iteration process continues until no unrepresentative keypoints can be found. Two specific approaches are proposed to perform the first step of IKS. IKS1 focuses on randomly selecting one representative keypoint and IKS2 is based on a clustering algorithm in which the representative keypoints are the closest points to their cluster centers. Experiments carried out based on the Caltech 101, Caltech 256, and PASCAL 2007 datasets demonstrate that performing keypoint selection using IKS1 and IKS2 to generate both the BoW and spatial-based BoW features allows the support vector machine (SVM) classifier to provide better classification accuracy than with the baseline features without keypoint selection. However, it is found that the computational cost of IKS1 is larger than the baseline methods. On the other hand, IKS2 is able to not only efficiently generate the BoW and spatial-based features that reduce the computational time for vector quantization over these datasets, but also provides better classification results than IKS1 over the PASCAL 2007 and Caltech 256 datasets.
