ClassificationModulesExamples/KNNExample/KNNExample.cpp

This class implements the K-Nearest Neighbor classification algorithm (http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm). KNN is a simple but powerful classifier, based on finding the closest K training examples in the feature space for the new input vector. The KNN algorithm is amongst the simplest of all machine learning algorithms: an object is classified by a majority vote of its neighbors, with the object being assigned to the class most common amongst its k nearest neighbors (k is a positive integer, typically small). If k = 1, then the object is simply assigned to the class of its nearest neighbor.This implementation of the algorithm will return the class label of the class that gains the majoriy vote of its neighbours. If the average distance of the closest K neighbors with the class label of the majority vote is greater than that of that classes rejection threshold, then the predicted class label will be set to 0, indicating that the majority class was rejected. This feature can be enabled or disabled by setting the enableNullRejection paramter to false.

Remarks

This implementation is based on Bishop, Christopher M. Pattern recognition and machine learning. Vol. 1. New York: springer, 2006.

/*
 GRT MIT License
 Copyright (c) <2012> <Nicholas Gillian, Media Lab, MIT>
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software 
 and associated documentation files (the "Software"), to deal in the Software without restriction, 
 including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, 
 and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, 
 subject to the following conditions:
 
 The above copyright notice and this permission notice shall be included in all copies or substantial 
 portions of the Software.
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT 
 LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
 WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 */

/*
 GRT KNN Example
 This examples demonstrates how to initialize, train, and use the KNN algorithm for classification. 
 
 The K-Nearest Neighbor (KNN) Classifier is a simple classifier that works well on basic recognition problems, however it can be slow for real-time prediction if there are a large number of training examples and is not robust to noisy data. 
 
 In this example we create an instance of a KNN algorithm and then train the algorithm using some pre-recorded training data.
 The trained KNN algorithm is then used to predict the class label of some test data.
 
 This example shows you how to:
 - Create an initialize the KNN algorithm and set the number of neighbors to use for clasification
 - Load some ClassificationData from a file and partition the training data into a training dataset and a test dataset
 - Train the KNN algorithm using the training dataset
 - Test the KNN algorithm using the test dataset
 - Manually compute the accuracy of the classifier

You should run this example with one argument pointing to the data you want to load. A good dataset to run this example is acc-orientation.grt, which can be found in the GRT data folder.
*/

//You might need to set the specific path of the GRT header relative to your project
#include <GRT/GRT.h>
using namespace GRT;
using namespace std;

int main (int argc, const char * argv[])
{
 //Parse the data filename from the argument list
 if( argc != 2 ){
 cout << "Error: failed to parse data filename from command line. You should run this example with one argument pointing to the data filename!\n";
 return EXIT_FAILURE;
 }
 const string filename = argv[1];

 //Create a new KNN classifier with a K value of 10
 KNN knn(10);
 knn.setNullRejectionCoeff( 10 );
 knn.enableScaling( true );
 knn.enableNullRejection( true );
 
 //Train the classifier with some training data
 ClassificationData trainingData;
 
 if( !trainingData.load( filename ) ){
 cout << "Failed to load training data: " << filename << endl;
 return EXIT_FAILURE;
 }
 
 //Use 20% of the training dataset to create a test dataset
 ClassificationData testData = trainingData.split( 80 );
 
 //Train the classifier
 if( !knn.train( trainingData ) ){
 cout << "Failed to train classifier!\n";
 return EXIT_FAILURE;
 }
 
 //Save the knn model to a file
 if( !knn.save("KNNModel.grt") ){
 cout << "Failed to save the classifier model!\n";
 return EXIT_FAILURE;
 }
 
 //Load the knn model from a file
 if( !knn.load("KNNModel.grt") ){
 cout << "Failed to load the classifier model!\n";
 return EXIT_FAILURE;
 }
 
 //Use the test dataset to test the KNN model
 double accuracy = 0;
 for(UINT i=0; i<testData.getNumSamples(); i++){
 //Get the i'th test sample
 UINT classLabel = testData[i].getClassLabel();
 VectorFloat inputVector = testData[i].getSample();
 
 //Perform a prediction using the classifier
 bool predictSuccess = knn.predict( inputVector );
 
 if( !predictSuccess ){
 cout << "Failed to perform prediction for test sampel: " << i <<"\n";
 return EXIT_FAILURE;
 }
 
 //Get the predicted class label
 UINT predictedClassLabel = knn.getPredictedClassLabel();
 VectorFloat classLikelihoods = knn.getClassLikelihoods();
 VectorFloat classDistances = knn.getClassDistances();
 
 //Update the accuracy
 if( classLabel == predictedClassLabel ) accuracy++;
 
 cout << "TestSample: " << i << " ClassLabel: " << classLabel << " PredictedClassLabel: " << predictedClassLabel << endl;
 }
 
 cout << "Test Accuracy: " << accuracy/double(testData.getNumSamples())*100.0 << "%" << endl;
 
 return EXIT_SUCCESS;

}