If you have any doubts in the below, contact us by dropping a mail to the Kung Fu Panda.
We will get back to you very soon.
Support Vector Machines (SVM)
- considered a combination of K Nearest Neighbours(KNN) and linear regression models
- mostly used for classification, but can also be used for numeric prediction.
- SVM tries to separate out the different categories of data by creating a flat boundary called 'hyperplane'.
- SVM tries to find out the Maximum Margin Hyperplane(MMH) that separates the two classes with maximum distance.
- Hyperplane is a flat surface in the N dimensional space.
- SVM requires all features to be numeric.
what is a support vector?
- support vectors are the points from each class that are closest to the Maximum Margin Hyperplane(MMH).
- each class should have one or more support vector.
- an important property of SVM is that just by the support vectors, it is possible to define the hyperplane MMH.
- in linearly separable data, get the shortest distance between the points of two classes, perpendicular line is the MMH.
How it works
- if data is not linearly separable, we allow some data to have wrong class while maximizing the distance after adjusting the penalty for wrong classification.
- another way to do it is using a kernel trick.
- kernel trick tries to transform the non linear data into linear data by viewing it in some other dimension.
- adds additional dimensions to the data, and can learn the relationship between dimensions that were originally not stated.
Pros of SVM with non linear kernel
- can be used for classification n regression.
- noise doesn't affect it much, doesn't overfit.
- easier than ANN
- used a lot in image processing/recognition.
Cons of SVM with non linear kernel
- need to test with a number of kernel and model params to find out the best model.
- slow to train.
- model is difficult to understand.
various packages for SVM in R
- e1071 package
- klaR package
- kernlab package (for beginners, written in R)
- data_train=data[1:1000, ]
- data_test=data[1001:1500, ]
- model=ksvm(target ~ predictors, data=data_train, kernel="vanilladot", C=1)
- kernel= "rbfdot" means radial bias function
- "polydot" means polynomial function
- "tanhdot" means hyperbolic tangent sigmoid
- "vanilladot" means linear
- C = penalty for violation by the MMH.
- predictions=predict(model, data_test)
- correct = predictions == data_test$dependentVar
//This way of calculating the accuracy is for classification problem, if using regression, we could use cor().