K Nearest Neighbors (KNN) can also be used for regression tasks, where the goal is to predict a continuous output rather than a discrete class label. Here's how KNN works for regression:
-
Distance Calculation: Similar to classification, for a given test data point, the algorithm calculates the distance to all training data points.
-
Finding Neighbors: It identifies the 'K' nearest neighbors based on the calculated distances.
-
Averaging Values: Instead of voting for a class label, KNN for regression takes the average (or sometimes a weighted average) of the target values of the K nearest neighbors.
-
Prediction: The predicted value for the test data point is the computed average from the K neighbors.
Example Code for KNN Regression
Here’s a simple example using KNeighborsRegressor from scikit-learn:
from sklearn.neighbors import KNeighborsRegressor
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_regression
# Generate synthetic regression data
X, y = make_regression(n_samples=100, n_features=1, noise=0.1)
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create and train the KNN regressor
knn_regressor = KNeighborsRegressor(n_neighbors=5)
knn_regressor.fit(X_train, y_train)
# Predict on the test set
predictions = knn_regressor.predict(X_test)
# Print predictions
print("Predictions:", predictions)In this example, we generate synthetic regression data, split it into training and testing sets, and then use KNN to predict the target values for the test set.
