Feature Scaling (Standardization and Normalization) is one of the important steps while preparing the data. Whether to use feature scaling or not depends upon the algorithm you are using. Some algorithms require feature scaling and some don't. Lets see it in detail.
Algorithms which require Feature Scaling (Standardization and Normalization)
Any machine learning algorithm that computes the distance between the data points needs Feature Scaling (Standardization and Normalization). This includes all curve based algorithms.
Example:
1. KNN (K Nearest Neigbors)
2. SVM (Support Vector Machine)
3. Logistic Regression
4. K-Means Clustering
Algorithms that are used for matrix factorization, decomposition and dimensionality reduction also require feature scaling.
Example:
1. PCA (Principal Component Analysis)
2. SVD (Singular Value Decomposition)
Why is Feature Scaling required?
Consider a dataset which contains the age and salary of the employee. Now age is a 2 digit number while salary would be of 5 to 6 digit. If we don't scale both the features (age and salary), salary will adversely affect the accuracy of the algorithm (if the algorithm is distance based). So, if we don't scale the features, then large scale features will dominate the small scale features due to which algorithm will produce wrong predictions.
Algorithms which don't require Feature Scaling (Standardization and Normalization)
The algorithms which rely on rules like tree based algorithms don't require Feature Scaling (Standardization and Normalization).
Example:
1. CART (Classification and Regression Trees)
2. Random Forests
3. Gradient Boosted Decision Trees
Algorithms that rely on distributions of the variables also don't need feature scaling.
Example:
1. Naive Bayes
Related Article: Why to standardize and transform the features in the dataset before applying Machine Learning algorithms?
Algorithms which require Feature Scaling (Standardization and Normalization)
Any machine learning algorithm that computes the distance between the data points needs Feature Scaling (Standardization and Normalization). This includes all curve based algorithms.
Example:
1. KNN (K Nearest Neigbors)
2. SVM (Support Vector Machine)
3. Logistic Regression
4. K-Means Clustering
Algorithms that are used for matrix factorization, decomposition and dimensionality reduction also require feature scaling.
Example:
1. PCA (Principal Component Analysis)
2. SVD (Singular Value Decomposition)
Why is Feature Scaling required?
Consider a dataset which contains the age and salary of the employee. Now age is a 2 digit number while salary would be of 5 to 6 digit. If we don't scale both the features (age and salary), salary will adversely affect the accuracy of the algorithm (if the algorithm is distance based). So, if we don't scale the features, then large scale features will dominate the small scale features due to which algorithm will produce wrong predictions.
Algorithms which don't require Feature Scaling (Standardization and Normalization)
The algorithms which rely on rules like tree based algorithms don't require Feature Scaling (Standardization and Normalization).
Example:
1. CART (Classification and Regression Trees)
2. Random Forests
3. Gradient Boosted Decision Trees
Algorithms that rely on distributions of the variables also don't need feature scaling.
Example:
1. Naive Bayes
Related Article: Why to standardize and transform the features in the dataset before applying Machine Learning algorithms?
Beautiful.
ReplyDeleteIMHO Logistic Regression doesn't require standardization.
LR with Polynomial or interaction predictors do require.