What is Factor Analysis? What is the difference between Exploratory Factor Analysis and Confirmatory Factor Analysis?

Factor Analysis is a statistical techniques used for dimensionality reduction in machine learning. Factor Analysis is used to reduce a large number of variables into fewer numbers of factors (variables) based on the correlation among the variables. It tries to capture maximum variance in the data with minimum number of variables. 

You can find more details about dimensionality reduction in my following articles:

Why is Dimensionality Reduction required?
Feature Selection and Feature Extraction Techniques
Difference between Covariance and Correlation
What is Multicollinearity?

Types of Factor Analysis

There are mainly two types of Factor Analysis:

1. Exploratory Factor Analysis (EFA)
2. Confirmatory Factor Analysis (CFA)

1. Exploratory Factor Analysis: It assumes that any indicator or variable may be associated with any factor. This is the most common factor analysis used by researchers and it is not based on any prior theory. Best example of Exploratory Factor Analysis is PCA (Principal Component Analysis). 

Advantages and Disadvantages of PCA
PCA vs t-SNE

2. Confirmatory factor analysis (CFA): It is used to determine the factor and factor loading of measured variables, and to confirm what is expected on the basic or pre-established theory. CFA assumes that each factor is associated with a specified subset of measured variables.