K-NN — For Beginners

KNN as Classification Algorithm:

KNN ( K Nearest Neighbor ) algorithm is one of the simple and yet powerful algorithm from the pool of supervised learning algorithms. It is simple to understand for a beginner in machine learning. Also, this is most often asked interview question for entry level machine learning positions, perhaps because it can be coded within 30 minutes and it can be coded either in Java or Python. It is powerful because it gives descent accuracy in most cases. This algorithm can be used for classification as well as regression. In this post, we will understand how it works for classification using iris data set. This data set consists of 3 different types of irises’ namely Setosa, Versicolour and Virginica. It has 4 features Sepal Length, Sepal Width, Petal Length and Petal Width. We can directly get this data set as below.

from sklearn.datasets import load_iris
iris_data = load_iris()
X = iris_data.data
y = iris_data.target

KNN is different from other supervised learning algorithms like logistic regression in the sense that this is a “model free” learning algorithm that means it does not compute any weights or hyper parameters during training step. In fact, there is no training step. Training and Prediction occurs at the same time. So, it is known as instance-based learning or lazy learning, where the function is only approximated locally and all computation is deferred until classification step.

The algorithm consists of following two steps: