Lets solve a regression problem using neural networks. We will build a sequential model in Keras to predict house prices based on some parameters. We will use KerasRegressor to build a regression model.
You can download housing_data.csv from here. You can also download my Jupyter notebook containing below code of Neural Network Regression implementation.
Step 1: Import required libraries like pandas, numpy, sklearn, keras and matplotlib
import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
import matplotlib.pyplot as plt
%matplotlib inline
Step 2: Load and examine the dataset
dataset = pd.read_csv('housing_data.csv')
dataset.head()
dataset.shape
dataset.describe(include='all')
Please note that "describe()" is used to display the statistical values of the data like mean and standard deviation.
Step 3: Mention X and Y axis
X=dataset.iloc[:,0:13]
y=dataset.iloc[:,13].values
X contains the list of attributes
Y contains the list of labels
Step 4: Split the dataset into training and testing dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, random_state=0)
Step 5: Scale the features
scaler = MinMaxScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
y = y.reshape(-1,1)
y = scaler.fit_transform(y)
This step is must for neural networks. Feature scaling is very important for neural networks to perform better and predict accurate results. We should scale both X and y data.
Step 6: Build a neural network
def build_regression_model():
model = Sequential()
model.add(Dense(50, input_dim=13, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(optimizer='adam', loss='mean_squared_error')
return model
We are creating a sequential model with fully connected layers. We are using four layers (one input layer, one output layer and two hidden layers). Input layer and hidden layers are using "relu" activation function while output layer is using "linear" activation function.
Input layer and hidden layers contain 50 neurons and output layer contains only one neuron as we need to output only one value (predicted house price). You can change the number of neurons in the input and hidden layers as per your data and model performance. Number of hidden layers and number of neurons in each layer are the hyperparameters which you need to tune as the the performance of the model.
We are using "adam" optimizer and mean square error as a loss function.
We can also use dropout in hidden layers for regularization. But, for this example, I am skipping this step for simplification.
Step 7: Train the neural network
regressor = KerasRegressor(build_fn=build_regression_model, batch_size=32, epochs=150)
training_history = regressor.fit(X_train,y_train)
We are using 150 epochs with batch size of 32. Number of epochs and batch size are also the hyperparameters which need to be tuned.
Step 8: Print a loss plot
plt.plot(training_history.history['loss'])
plt.show()
This plot shows that after around 140 epochs, the loss does not vary so much. That is why, I have taken number of epochs as 150 in step 7 while training the neural network.
Step 9: Predict from the neural network
y_pred= regressor.predict(X_test)
y_pred
The y_pred is a numpy array that contains all the predicted values for the input values in the X_test.
Lets see the difference between the actual and predicted values.
df=pd.DataFrame({'Actual':y_test, 'Predicted':y_pred})
df
Step 10: Check the accuracy
meanAbsoluteError = mean_absolute_error(y_test, y_pred)
meanSquaredError = mean_squared_error(y_test, y_pred)
rootMeanSquaredError = np.sqrt(meanSquaredError)
print('Mean Absolute Error:', meanAbsoluteError)
print('Mean Squared Error:', meanSquaredError)
print('Root Mean Squared Error:', rootMeanSquaredError)
Output:
Mean Absolute Error: 2.9524098807690193 Mean Squared Error: 19.836363961675836 Root Mean Squared Error: 4.453803314210882
We have got the root mean square error as 4.45. We can further decrease this error using cross validation and tuning our hyperparameters. I am leaving it for you to practice.
Step 11: Visualize the results using scatter plot
plt.scatter(range(len(y_test)), y_test, c='g')
plt.scatter(range(len(y_test)), y_pred, c='b')
plt.xlabel('Test data')
plt.ylabel('Predicted data')
plt.show()
We are displaying test labels and predicted values in different colors (green and blue). From the scatter plot, we can visualize that our neural network has done a great job.
Step 12: Visualize results using regression plot
To further visualize the predicted results, we can draw a regression plot.
fig, ax = plt.subplots()
ax.scatter(y_test, y_pred)
ax.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'k--', lw=4)
ax.set_xlabel('Test data')
ax.set_ylabel('Predicted data')
plt.show()
I hope, I was able to demonstrate this regression problem to a large extent. If you have further any doubt, please post a comment.
You can download housing_data.csv from here. You can also download my Jupyter notebook containing below code of Neural Network Regression implementation.
Step 1: Import required libraries like pandas, numpy, sklearn, keras and matplotlib
import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
import matplotlib.pyplot as plt
%matplotlib inline
Step 2: Load and examine the dataset
dataset = pd.read_csv('housing_data.csv')
dataset.head()
dataset.shape
dataset.describe(include='all')
Please note that "describe()" is used to display the statistical values of the data like mean and standard deviation.
Step 3: Mention X and Y axis
X=dataset.iloc[:,0:13]
y=dataset.iloc[:,13].values
X contains the list of attributes
Y contains the list of labels
Step 4: Split the dataset into training and testing dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, random_state=0)
Step 5: Scale the features
scaler = MinMaxScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
y = y.reshape(-1,1)
y = scaler.fit_transform(y)
This step is must for neural networks. Feature scaling is very important for neural networks to perform better and predict accurate results. We should scale both X and y data.
Step 6: Build a neural network
def build_regression_model():
model = Sequential()
model.add(Dense(50, input_dim=13, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(optimizer='adam', loss='mean_squared_error')
return model
We are creating a sequential model with fully connected layers. We are using four layers (one input layer, one output layer and two hidden layers). Input layer and hidden layers are using "relu" activation function while output layer is using "linear" activation function.
Input layer and hidden layers contain 50 neurons and output layer contains only one neuron as we need to output only one value (predicted house price). You can change the number of neurons in the input and hidden layers as per your data and model performance. Number of hidden layers and number of neurons in each layer are the hyperparameters which you need to tune as the the performance of the model.
We are using "adam" optimizer and mean square error as a loss function.
We can also use dropout in hidden layers for regularization. But, for this example, I am skipping this step for simplification.
Step 7: Train the neural network
regressor = KerasRegressor(build_fn=build_regression_model, batch_size=32, epochs=150)
training_history = regressor.fit(X_train,y_train)
We are using 150 epochs with batch size of 32. Number of epochs and batch size are also the hyperparameters which need to be tuned.
Step 8: Print a loss plot
plt.plot(training_history.history['loss'])
plt.show()
This plot shows that after around 140 epochs, the loss does not vary so much. That is why, I have taken number of epochs as 150 in step 7 while training the neural network.
Step 9: Predict from the neural network
y_pred= regressor.predict(X_test)
y_pred
The y_pred is a numpy array that contains all the predicted values for the input values in the X_test.
Lets see the difference between the actual and predicted values.
df=pd.DataFrame({'Actual':y_test, 'Predicted':y_pred})
df
Step 10: Check the accuracy
meanAbsoluteError = mean_absolute_error(y_test, y_pred)
meanSquaredError = mean_squared_error(y_test, y_pred)
rootMeanSquaredError = np.sqrt(meanSquaredError)
print('Mean Absolute Error:', meanAbsoluteError)
print('Mean Squared Error:', meanSquaredError)
print('Root Mean Squared Error:', rootMeanSquaredError)
Output:
Mean Absolute Error: 2.9524098807690193 Mean Squared Error: 19.836363961675836 Root Mean Squared Error: 4.453803314210882
We have got the root mean square error as 4.45. We can further decrease this error using cross validation and tuning our hyperparameters. I am leaving it for you to practice.
Step 11: Visualize the results using scatter plot
plt.scatter(range(len(y_test)), y_test, c='g')
plt.scatter(range(len(y_test)), y_pred, c='b')
plt.xlabel('Test data')
plt.ylabel('Predicted data')
plt.show()
We are displaying test labels and predicted values in different colors (green and blue). From the scatter plot, we can visualize that our neural network has done a great job.
Step 12: Visualize results using regression plot
To further visualize the predicted results, we can draw a regression plot.
fig, ax = plt.subplots()
ax.scatter(y_test, y_pred)
ax.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'k--', lw=4)
ax.set_xlabel('Test data')
ax.set_ylabel('Predicted data')
plt.show()
I hope, I was able to demonstrate this regression problem to a large extent. If you have further any doubt, please post a comment.
No comments:
Post a Comment