tag:blogger.com,1999:blog-44234964616961373692024-03-18T10:21:43.925-07:00The Professionals PointDelphi, C#, Python, Machine Learning, Deep Learning, TensorFlow, KerasNaresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.comBlogger939125tag:blogger.com,1999:blog-4423496461696137369.post-35972271371335005252023-06-19T08:18:00.005-07:002023-06-19T08:18:49.284-07:00600+ AWS Objective Type Questions (12 Quizzes with 50 Questions each)<p><span style="font-family: arial;">We have compiled 600 AWS objective questions across 12 test papers to help you test your understanding and expertise. </span></p><p><a href="https://mytechbasket.com/basket.php"><span style="font-family: georgia; font-size: x-large;">My Tech Basket</span></a></p><p><span style="font-family: arial;">Following topics are covered in these 12 exams.</span></p><p><span style="font-family: arial;"><b>AWS Basics:</b></span></p><p><span style="font-family: arial;">We start our journey with the fundamentals. This section introduces AWS, its core concepts, and the benefits it offers. Gain insights into the AWS Global Infrastructure, AWS Management Console, and key AWS services. Understand how AWS can help you build scalable and reliable applications in the cloud.</span></p><p><span style="font-family: arial;"><b>Compute Services in AWS:</b></span></p><p><span style="font-family: arial;">Discover the power of compute services in AWS. Explore Amazon Elastic Compute Cloud (EC2), Amazon Elastic Container Service (ECS), and AWS Lambda. Learn how to provision virtual servers, containers, and serverless functions to meet your application's computational needs.</span></p><p><span style="font-family: arial;"><b>Storage and Database Services in AWS:</b></span></p><p><span style="font-family: arial;">Data is the lifeblood of any application. Dive into AWS storage services such as Amazon Simple Storage Service (S3), Amazon Elastic Block Store (EBS), and Amazon Relational Database Service (RDS). Uncover the capabilities of each service and explore options for data storage, backup, and retrieval.</span></p><p><span style="font-family: arial;"><b>Networking and Content Delivery in AWS:</b></span></p><p><span style="font-family: arial;">Connectivity and content delivery are crucial aspects of any cloud infrastructure. Discover AWS networking services, including Amazon Virtual Private Cloud (VPC), Amazon Route 53, and Amazon CloudFront. Learn how to create secure and scalable network architectures to ensure optimal performance for your applications.</span></p><p><span style="font-family: arial;"><b>Security and Identity Services in AWS:</b></span></p><p><span style="font-family: arial;">Security is of paramount importance in the cloud. Explore AWS Identity and Access Management (IAM), AWS Key Management Service (KMS), and AWS Certificate Manager (ACM). Understand how these services can help you secure your AWS resources, manage access permissions, and encrypt sensitive data.</span></p><p><span style="font-family: arial;"><b>Management and Monitoring Services in AWS:</b></span></p><p><span style="font-family: arial;">Efficiently managing and monitoring your AWS environment is essential for smooth operations. Learn about AWS CloudFormation, AWS CloudTrail, and Amazon CloudWatch. Discover how these services enable you to automate resource provisioning, track API activity, and monitor system metrics.</span></p><p><span style="font-family: arial;"><b>Analytics and Big Data Services in AWS:</b></span></p><p><span style="font-family: arial;">Harness the power of data analytics and big data processing with AWS. Dive into services such as Amazon Redshift, Amazon Athena, and AWS Glue. Gain insights into data warehousing, data lakes, and analytics workflows in the AWS ecosystem.</span></p><p><span style="font-family: arial;"><b>AI and Machine Learning Services in AWS:</b></span></p><p><span style="font-family: arial;">Explore the cutting-edge world of artificial intelligence and machine learning on AWS. Discover services like Amazon Rekognition, Amazon SageMaker, and Amazon Comprehend. Learn how to leverage these services to build intelligent applications and extract valuable insights from data.</span></p><p><span style="font-family: arial;"><b>Serverless Computing in AWS:</b></span></p><p><span style="font-family: arial;">Serverless computing offers a paradigm shift in application development. Explore AWS Lambda, Amazon API Gateway, and AWS Step Functions. Understand the benefits of serverless architecture and learn how to build scalable and cost-effective serverless applications.</span></p><p><span style="font-family: arial;"><b>DevOps and CI/CD in AWS:</b></span></p><p><span style="font-family: arial;">Accelerate your software delivery pipeline with AWS DevOps services. Dive into AWS CodePipeline, AWS CodeCommit, and AWS CodeDeploy. Learn how to automate application deployment, implement continuous integration and continuous deployment (CI/CD), and foster collaboration within development teams.</span></p><p><span style="font-family: arial;"><b>AWS Scenario-Based Mixed Questions - Part 1:</b></span></p><p><span style="font-family: arial;">Test your knowledge with scenario-based questions that simulate real-world AWS scenarios. Explore various AWS services and their application in practical use cases. Challenge yourself and enhance your problem-solving skills.</span></p><p><span style="font-family: arial;"><b>AWS Scenario-Based Mixed Questions - Part 2:</b></span></p><p><span style="font-family: arial;">Continue the journey of scenario-based questions in this second part. Encounter new challenges and expand your understanding of AWS by solving complex scenarios. Strengthen your ability to architect solutions and make informed decisions.</span></p><p><span style="font-family: arial;">Remember, practice and hands-on experience are key to mastering AWS. Use the 600 objective questions provided to assess your knowledge and identify areas for further exploration. Start your AWS journey today and unlock the full potential of cloud computing!</span></p><p><span style="font-family: arial;">Visit </span><a href="https://mytechbasket.com/" style="font-family: arial;">My Tech Basket</a><span style="font-family: arial;"> for more AWS resources, tutorials, and additional study material.</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-65323196922218382352023-06-15T09:23:00.007-07:002023-06-15T09:23:52.360-07:0030 Must-Know TensorFlow Interview Questions and Answers<p><span style="font-family: arial;">Prepare for interviews by understanding the fundamental concepts, practicing coding, and exploring real-world use cases of TensorFlow. These TensorFlow interview questions and answers should serve as a starting point to help you in your preparation.</span></p><p><b style="font-family: arial;">1. What is TensorFlow?</b></p><p><span style="font-family: arial;">TensorFlow is an open-source machine learning framework developed by Google that is widely used for building and training machine learning models.</span></p><p><span style="font-family: arial;"><b>2. What are the key features of TensorFlow?</b></span></p><p><span style="font-family: arial;">Some key features of TensorFlow include its flexibility, scalability, support for distributed computing, automatic differentiation, and support for both CPU and GPU computations.</span></p><p><span style="font-family: arial;"><b>3. What is a TensorFlow graph?</b></span></p><p><span style="font-family: arial;">A TensorFlow graph is a computational graph that represents the flow of data and operations in a TensorFlow model. It consists of nodes (representing operations) and edges (representing data tensors).</span></p><p><span style="font-family: arial;"><b>4. What are tensors in TensorFlow?</b></span></p><p><span style="font-family: arial;">Tensors are multi-dimensional arrays used to represent data in TensorFlow. They can be scalars (0-dimensional), vectors (1-dimensional), matrices (2-dimensional), or higher-dimensional arrays.</span></p><p><span style="font-family: arial;"><b>5. What is the difference between TensorFlow 1.x and TensorFlow 2.x?</b></span></p><p><span style="font-family: arial;">TensorFlow 2.x introduced several improvements and simplifications compared to TensorFlow 1.x, including eager execution by default, a more intuitive API, and improved support for customization and deployment.</span></p><p><span style="font-family: arial;"><b>6. How can you define a model in TensorFlow?</b></span></p><p><span style="font-family: arial;">In TensorFlow, you can define a model by creating a computational graph using TensorFlow's high-level APIs like Keras or by building the graph manually using lower-level TensorFlow operations.</span></p><p><span style="font-family: arial;"><b>7. Explain the concept of eager execution in TensorFlow.</b></span></p><p><span style="font-family: arial;">Eager execution is a mode in TensorFlow 2.x that allows you to execute operations immediately as they are called, rather than building a computational graph first. It makes TensorFlow code more intuitive and easier to debug.</span></p><p><span style="font-family: arial;"><b>8. What is the purpose of placeholders in TensorFlow?</b></span></p><p><span style="font-family: arial;">Placeholders are used to feed data into a TensorFlow model during training or inference. They are typically used for inputs that may vary in size or value during different training or inference steps.</span></p><p><span style="font-family: arial;"><b>9. How can you save and restore TensorFlow models?</b></span></p><p><span style="font-family: arial;">TensorFlow provides the <b>tf.train.Saver</b> class that allows you to save and restore model variables. You can save the entire model or specific variables to disk and later restore them to continue training or perform inference.</span></p><p><span style="font-family: arial;"><b>10. Explain the concept of checkpoints in TensorFlow.</b></span></p><p><span style="font-family: arial;">Checkpoints are files that store the values of all variables in a TensorFlow model at a specific point in training. They can be used to save and restore model states, track training progress, and resume training from a specific checkpoint.</span></p><p><span style="font-family: arial;"><b>11. What is TensorFlow Lite?</b></span></p><p><span style="font-family: arial;">TensorFlow Lite is a lightweight version of TensorFlow designed for mobile and embedded devices. It enables the deployment of TensorFlow models on resource-constrained platforms.</span></p><p><span style="font-family: arial;"><b>12. How can you optimize TensorFlow models for better performance?</b></span></p><p><span style="font-family: arial;">TensorFlow provides various techniques for optimizing models, including quantization, pruning, model compression, and hardware-specific optimizations like using GPU or TPU accelerators.</span></p><p><span style="font-family: arial;"><b>13. What is transfer learning in TensorFlow?</b></span></p><p><span style="font-family: arial;">Transfer learning is a technique in which pre-trained models are used as a starting point for training a new model on a different but related task. It allows leveraging knowledge learned from large datasets and models.</span></p><p><span style="font-family: arial;"><b>14. How can you deploy TensorFlow models in production?</b></span></p><p><span style="font-family: arial;">TensorFlow models can be deployed in production using various methods, such as serving the model through TensorFlow Serving, converting the model to a TensorFlow.js format for web deployment, or deploying on cloud platforms like TensorFlow Extended (TFX) or TensorFlow on AWS.</span></p><p><span style="font-family: arial;"><b>15. What are some common activation functions in TensorFlow?</b></span></p><p><span style="font-family: arial;">Some common activation functions in TensorFlow include sigmoid, tanh, ReLU (Rectified Linear Unit), softmax, and Leaky ReLU.</span></p><p><span style="font-family: arial;"><b>16. What is the purpose of optimizers in TensorFlow?</b></span></p><p><span style="font-family: arial;">Optimizers in TensorFlow are used to minimize the loss function and update the model's parameters during training. They apply various optimization algorithms like Stochastic Gradient Descent (SGD), Adam, RMSProp, etc.</span></p><p><span style="font-family: arial;"><b>17. How can you visualize TensorFlow graphs?</b></span></p><p><span style="font-family: arial;">TensorFlow provides tools like TensorBoard for visualizing TensorFlow graphs. You can add summary operations to your graph and use TensorBoard to visualize metrics, graph structures, and other useful information.</span></p><p><span style="font-family: arial;"><b>18. Explain the concept of data pipelines in TensorFlow.</b></span></p><p><span style="font-family: arial;">Data pipelines in TensorFlow are used to efficiently load and preprocess large datasets for training or inference. TensorFlow provides APIs like <b>tf.data</b> to build efficient data input pipelines.</span></p><p><span style="font-family: arial;"><b>19. What is distributed TensorFlow?</b></span></p><p><span style="font-family: arial;">Distributed TensorFlow enables the training and inference of TensorFlow models on multiple devices or machines. It allows parallel processing, scaling, and efficient utilization of resources.</span></p><p><span style="font-family: arial;"><b>20. What are some popular TensorFlow-based projects or libraries?</b></span></p><p><span style="font-family: arial;">Some popular TensorFlow-based projects or libraries include TensorFlow Hub, TensorFlow Extended (TFX), TensorFlow.js, TensorFlow Serving, and TensorFlow Lite.</span></p><p><span style="font-family: arial;"><b>21. What is eager execution in TensorFlow 2.x?</b></span></p><p><span style="font-family: arial;">Eager execution is a mode in TensorFlow 2.x that enables immediate execution of operations. It eliminates the need for explicit session management and allows for dynamic control flow and easy debugging.</span></p><p><span style="font-family: arial;"><b>22. How can you handle overfitting in TensorFlow?</b></span></p><p><span style="font-family: arial;">To handle overfitting in TensorFlow, you can use techniques like regularization (e.g., L1 or L2 regularization), dropout, early stopping, and data augmentation. These techniques help prevent the model from memorizing the training data and improve generalization.</span></p><p><span style="font-family: arial;"><b>23. What are TensorFlow Estimators?</b></span></p><p><span style="font-family: arial;">TensorFlow Estimators are a high-level API that simplifies the process of model development, training, and evaluation. They provide pre-built models and encapsulate the training loop, making it easier to create production-ready models.</span></p><p><span style="font-family: arial;"><b>24. What is the purpose of TensorBoard in TensorFlow?</b></span></p><p><span style="font-family: arial;">TensorBoard is a web-based visualization tool provided by TensorFlow. It allows you to track and visualize various aspects of your model's performance, such as loss, accuracy, and computation graphs, making it easier to analyze and debug your models.</span></p><p><span style="font-family: arial;"><b>25. How can you save and load only the model weights in TensorFlow?</b></span></p><p><span style="font-family: arial;">You can save and load only the model weights in TensorFlow using the <b>tf.keras.Model.save_weights()</b> and <b>tf.keras.Model.load_weights()</b> methods. This is useful when you want to reuse the model architecture but load different weights.</span></p><p><span style="font-family: arial;"><b>26. What is the difference between TensorFlow and PyTorch?</b></span></p><p><span style="font-family: arial;">TensorFlow and PyTorch are both popular deep learning frameworks. While TensorFlow has a stronger focus on production deployment, distributed computing, and mobile deployment, PyTorch is known for its dynamic computation graph, simplicity, and strong research community.</span></p><p><span style="font-family: arial;"><b>27. How can you handle imbalanced datasets in TensorFlow?</b></span></p><p><span style="font-family: arial;">To handle imbalanced datasets in TensorFlow, you can use techniques like oversampling the minority class, undersampling the majority class, or using advanced algorithms like SMOTE (Synthetic Minority Over-sampling Technique) to generate synthetic samples.</span></p><p><span style="font-family: arial;"><b>28. What is the purpose of the TensorFlow Extended (TFX) library?</b></span></p><p><span style="font-family: arial;">TensorFlow Extended (TFX) is an end-to-end platform for deploying production machine learning pipelines. It provides tools and components for data validation, preprocessing, model training, model analysis, and serving.</span></p><p><span style="font-family: arial;"><b>29. How can you use TensorFlow for natural language processing (NLP) tasks?</b></span></p><p><span style="font-family: arial;">TensorFlow provides various tools and APIs for NLP tasks, such as the TensorFlow Text library, which offers a collection of text-related operations and models. Additionally, pre-trained models like BERT and GPT-2 can be fine-tuned for specific NLP tasks using TensorFlow.</span></p><p><span style="font-family: arial;"><b>30. What are TensorFlow's eager execution advantages over graph execution?</b></span></p><p><span style="font-family: arial;">Eager execution in TensorFlow offers advantages like improved flexibility, easier debugging, more intuitive code, support for dynamic control flow, and the ability to use Python's debugging tools seamlessly.</span></p><p><span style="font-family: arial;">These are sample questions, and the actual questions you may encounter in an interview can vary. It's important to have a solid understanding of TensorFlow concepts, programming, and practical implementation to perform well in a TensorFlow interview.</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-31350150668350440282023-06-02T23:44:00.002-07:002023-06-02T23:44:21.663-07:00 How to send emails to your users from your website using Brevo (SendInBlue) API in PHP?<p><span style="font-family: arial;">Below PHP code demonstrates how to send emails from your website using Brevo (SendInBlue) API in PHP. You should have API key handy before using this code. Please put entire code in try catch block.</span></p><p><span style="font-family: arial;"><b>//Set endpoint and api key</b></span></p><p><span style="font-family: arial;">$endpoint = 'https://api.brevo.com/v3/smtp/email';</span></p><p><span style="font-family: arial;">$api_key = 'YOUR_API_KEY';</span></p><p><span style="font-family: arial;"><b>//Request payload</b></span></p><p><span style="font-family: arial;">$data = array(</span></p><p><span style="font-family: arial;"> 'sender' => array(</span></p><p><span style="font-family: arial;"> 'name' => 'Sender Alex',</span></p><p><span style="font-family: arial;"> 'email' => 'senderalex@example.com'</span></p><p><span style="font-family: arial;"> ),</span></p><p><span style="font-family: arial;"> 'to' => array(</span></p><p><span style="font-family: arial;"> array(</span></p><p><span style="font-family: arial;"> 'email' => 'testmail@example.com',</span></p><p><span style="font-family: arial;"> 'name' => 'John Doe'</span></p><p><span style="font-family: arial;"> )</span></p><p><span style="font-family: arial;"> ),</span></p><p><span style="font-family: arial;"> 'subject' => 'Hello world',</span></p><p><span style="font-family: arial;"> 'htmlContent' => '<html><head></head><body><p>Hello,</p><p>This is my first transactional email sent from Brevo.</p></body></html>'</span></p><p><span style="font-family: arial;">);</span></p><p><span style="font-family: arial;"><b>//Set cURL options</b></span></p><p><span style="font-family: arial;">$options = array(</span></p><p><span style="font-family: arial;"> CURLOPT_URL => $endpoint,</span></p><p><span style="font-family: arial;"> CURLOPT_POST => true,</span></p><p><span style="font-family: arial;"> CURLOPT_POSTFIELDS => json_encode($data),</span></p><p><span style="font-family: arial;"> CURLOPT_RETURNTRANSFER => true,</span></p><p><span style="font-family: arial;"> CURLOPT_HTTPHEADER => array(</span></p><p><span style="font-family: arial;"> 'accept: application/json',</span></p><p><span style="font-family: arial;"> 'api-key: ' . $api_key,</span></p><p><span style="font-family: arial;"> 'content-type: application/json'</span></p><p><span style="font-family: arial;"> )</span></p><p><span style="font-family: arial;">);</span></p><p><span style="font-family: arial;"><b>//Initialize cURL session</b></span></p><p><span style="font-family: arial;">$curl = curl_init();</span></p><p><span style="font-family: arial;"><b>//Set cURL options</b></span></p><p><span style="font-family: arial;">curl_setopt_array($curl, $options);</span></p><p><span style="font-family: arial;"><b>//Execute the request</b></span></p><p><span style="font-family: arial;">$response = curl_exec($curl);</span></p><p><span style="font-family: arial;"><b>//Check for errors</b></span></p><p><span style="font-family: arial;">if ($response === false) {</span></p><p><span style="font-family: arial;"> echo 'Error: ' . curl_error($curl);</span></p><p><span style="font-family: arial;">} else {</span></p><p><span style="font-family: arial;"> <b> //Process the response</b></span></p><p><span style="font-family: arial;"> $response_data = json_decode($response, true);</span></p><p><span style="font-family: arial;"> if (isset($response_data['message'])) {</span></p><p><span style="font-family: arial;"> echo 'Email sent successfully!';</span></p><p><span style="font-family: arial;"> } else {</span></p><p><span style="font-family: arial;"> echo 'Email sending failed. Error: ' . $response_data['error'];</span></p><p><span style="font-family: arial;"> }</span></p><p><span style="font-family: arial;">}</span></p><p><b style="font-family: arial;">//Close cURL session</b></p><p><span style="font-family: arial;">curl_close($curl);</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-165534229925710022023-06-02T22:56:00.000-07:002023-06-02T22:56:02.727-07:00PayPal vs. Stripe: Choosing the Best Payment Platform for International Payments<p><span style="font-family: arial;">PayPal and Stripe are two leading payment platforms that cater to businesses seeking to accept payments internationally through their websites. While both platforms offer similar core functionality, there are some notable differences worth considering.</span></p><p><span style="font-family: arial;"><b>PayPal:</b></span></p><p><span style="font-family: arial;"><b>Trusted and Recognized:</b> PayPal is one of the most widely recognized and trusted payment platforms globally. It has been around for a long time, and many users are familiar with the PayPal brand.</span></p><p><span style="font-family: arial;"><b>User-Friendly Setup:</b> PayPal offers a user-friendly setup process, allowing businesses to quickly create an account, link their bank account or credit card, and start accepting payments.</span></p><p><span style="font-family: arial;"><b>Multiple Payment Options:</b> PayPal supports various payment methods, including credit cards, debit cards, PayPal accounts, and digital wallets. This flexibility allows customers to choose their preferred payment method during checkout.</span></p><p><span style="font-family: arial;"><b>International Transactions</b>: PayPal supports transactions in multiple currencies, making it suitable for businesses operating globally. It also handles currency conversion automatically, simplifying cross-border transactions.</span></p><p><span style="font-family: arial;"><b>Buyer and Seller Protection: </b>PayPal provides built-in buyer and seller protection programs. This helps protect both parties in case of disputes, chargebacks, or fraudulent transactions, providing an additional layer of security.</span></p><p><span style="font-family: arial;"><b>Stripe:</b></span></p><p><span style="font-family: arial;"><b>Developer-Friendly Integration:</b> Stripe is known for its developer-friendly APIs and extensive documentation, making it easier for businesses to integrate and customize payment solutions according to their specific needs.</span></p><p><span style="font-family: arial;"><b>Seamless Checkout Experience:</b> Stripe offers a highly customizable and optimized checkout experience. It allows businesses to design and control the entire payment flow on their website, creating a seamless and branded user experience.</span></p><p><span style="font-family: arial;"><b>Advanced Payment Feature</b>s: Stripe provides a comprehensive set of payment features beyond standard payment processing. It supports subscriptions, recurring billing, complex payment flows, and offers more advanced features for businesses with specific requirements.</span></p><p><span style="font-family: arial;"><b>Global Payment Support:</b> Stripe supports payments in over 135 currencies and provides localized payment methods, such as Alipay and WeChat Pay, making it suitable for businesses targeting international customers.</span></p><p><span style="font-family: arial;"><b>Advanced Fraud Prevention:</b> Stripe incorporates advanced fraud detection mechanisms, machine learning algorithms, and provides tools to help businesses mitigate fraudulent transactions. It offers customizable fraud rules and real-time risk evaluation.</span></p><p><span style="font-family: arial;">When choosing between PayPal and Stripe, it is important to consider your specific business requirements. Evaluate factors such as ease of integration, customization options, target audience, transaction volume, and desired payment features. Additionally, compare transaction fees, pricing structures, and available customer support to make an informed decision that aligns with your business goals and preferences. </span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-19516133321379510422023-06-02T10:23:00.003-07:002023-06-02T10:23:51.411-07:0020 Commonly Asked Data Science Interview Questions and Answers<p><span style="font-family: arial;">Here are 20 commonly asked data science interview questions and answers.</span></p><p><span style="font-family: arial;"><b>1. What is the role of a data scientist in a business setting?</b></span></p><p><span style="font-family: arial;">A data scientist helps businesses make data-driven decisions by analyzing large volumes of data, building predictive models, identifying patterns and trends, and providing insights to solve complex problems.</span></p><p><span style="font-family: arial;"><b>2. How do you handle missing data in a dataset?</b></span></p><p><span style="font-family: arial;">Missing data can be handled by various methods such as removing rows with missing values, imputing missing values using statistical measures like mean or median, or using advanced techniques like multiple imputation or predictive models.</span></p><p><span style="font-family: arial;"><b>3. What is the difference between univariate, bivariate, and multivariate analysis?</b></span></p><p><span style="font-family: arial;">Univariate analysis involves analyzing a single variable, bivariate analysis involves analyzing the relationship between two variables, and multivariate analysis involves analyzing the relationship between three or more variables.</span></p><p><span style="font-family: arial;"><b>4. How do you assess the quality of a data visualization?</b></span></p><p><span style="font-family: arial;">The quality of a data visualization can be assessed based on factors such as clarity, accuracy, relevance to the audience, effective use of visual elements, and the ability to convey insights or patterns in the data.</span></p><p><span style="font-family: arial;"><b>5. What are some common techniques for feature selection in data science?</b></span></p><p><span style="font-family: arial;">Common techniques for feature selection include filter methods (such as correlation and information gain), wrapper methods (such as forward/backward selection and recursive feature elimination), and embedded methods (such as LASSO and Ridge regression).</span></p><p><span style="font-family: arial;"><b>6. Explain the concept of outlier detection and its importance in data analysis.</b></span></p><p><span style="font-family: arial;">Outlier detection involves identifying observations that significantly deviate from the normal behavior of the data. Outliers can impact the statistical analysis and model performance, so detecting and handling them appropriately is crucial for accurate insights.</span></p><p><span style="font-family: arial;"><b>7. How do you handle imbalanced datasets in classification problems?</b></span></p><p><span style="font-family: arial;">Imbalanced datasets, where one class is significantly more prevalent than others, can be addressed by techniques such as oversampling the minority class, undersampling the majority class, or using advanced algorithms like SMOTE (Synthetic Minority Over-sampling Technique).</span></p><p><span style="font-family: arial;"><b>8. What are some common techniques for dimensionality reduction in data science?</b></span></p><p><span style="font-family: arial;">Common techniques for dimensionality reduction include Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), t-SNE (t-Distributed Stochastic Neighbor Embedding), and autoencoders.</span></p><p><span style="font-family: arial;"><b>9. Explain the concept of time series analysis and its applications.</b></span></p><p><span style="font-family: arial;">Time series analysis involves studying and modeling data collected over time to uncover patterns, trends, and seasonality. It finds applications in forecasting, anomaly detection, economic analysis, stock market analysis, and many other fields.</span></p><p><span style="font-family: arial;"><b>10. How do you handle multicollinearity in regression analysis?</b></span></p><p><span style="font-family: arial;">Multicollinearity occurs when two or more predictor variables in a regression model are highly correlated. It can be handled by techniques such as removing one of the correlated variables, performing dimensionality reduction, or using regularization techniques like Ridge regression.</span></p><p><span style="font-family: arial;"><b>11. What is the role of hypothesis testing in data science?</b></span></p><p><span style="font-family: arial;">Hypothesis testing is used to make inferences about a population based on a sample of data. It helps data scientists determine if there is enough evidence to support or reject a specific hypothesis or claim about the data.</span></p><p><b><span style="font-family: arial;">12. </span><span style="font-family: arial;">Explain the concept of feature extraction in data science.</span></b></p><p><span style="font-family: arial;">Feature extraction involves transforming raw data into a reduced set of meaningful and informative features. It aims to capture the most relevant aspects of the data, reduce dimensionality, and improve the performance of machine learning models.</span></p><p><span style="font-family: arial;"><b>13. How would you approach a data science project from start to finish?</b></span></p><p><span style="font-family: arial;">The approach to a data science project typically involves understanding the problem, gathering and exploring the data, preprocessing and cleaning the data, performing exploratory data analysis, building and evaluating models, and communicating the findings or insights.</span></p><p><b><span style="font-family: arial;">14. </span><span style="font-family: arial;">What are some common data preprocessing techniques in data science?</span></b></p><p><span style="font-family: arial;">Common data preprocessing techniques include handling missing values, dealing with outliers, scaling or normalizing features, encoding categorical variables, and splitting the data into training and testing sets.</span></p><p><span style="font-family: arial;"><b>15. What is the purpose of feature scaling in data science?</b></span></p><p><span style="font-family: arial;">Feature scaling is used to standardize or normalize the range of features in a dataset. It ensures that features with different scales or units have a similar impact on the models and prevents one feature from dominating others during the learning process.</span></p><p><span style="font-family: arial;"><b>16. Explain the concept of cross-validation in data science.</b></span></p><p><span style="font-family: arial;">Cross-validation is a technique used to assess the performance and generalization of a model. It involves splitting the data into multiple subsets, training the model on one subset, and evaluating it on the remaining subsets. This helps estimate the model's performance on unseen data.</span></p><p><span style="font-family: arial;"><b>17. How do you handle outliers in data analysis?</b></span></p><p><span style="font-family: arial;">Outliers can be handled by removing them if they are due to data entry errors or by applying statistical methods such as Winsorization or trimming to replace extreme values with more reasonable values. Outliers can also be analyzed separately or treated as a separate group in certain cases.</span></p><p><span style="font-family: arial;"><b>18. What is the purpose of dimensionality reduction in data science?</b></span></p><p><span style="font-family: arial;">Dimensionality reduction techniques aim to reduce the number of features or variables in a dataset while preserving the most important information. It helps overcome the curse of dimensionality, simplifies data analysis, improves model performance, and reduces computational complexity.</span></p><p><span style="font-family: arial;"><b>19. How do you evaluate the performance of a clustering algorithm in data science?</b></span></p><p><span style="font-family: arial;">The performance of clustering algorithms can be evaluated using metrics such as silhouette score, cohesion, separation, or visual inspection of cluster quality. Additionally, domain-specific knowledge and interpretability of the clustering results are important considerations.</span></p><p><span style="font-family: arial;"><b>20. What is the role of data visualization in data science?</b></span></p><p><span style="font-family: arial;">Data visualization is a critical aspect of data science as it helps in understanding the patterns, trends, and relationships present in the data. It allows for effective communication of insights, supports decision-making, and aids in identifying anomalies or outliers.</span></p><p><span style="font-family: arial;">I have given very short answers. Please study and understand these concepts thoroughly to effectively answer data science interview questions. Good luck!</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-80768886384479463992023-05-31T01:20:00.002-07:002023-05-31T01:20:22.875-07:00How is Machine Learning transforming and revolutionizing the automobile sector?<p><span style="font-family: arial;">Machine learning is revolutionizing the automobile sector in various ways. Machine learning is driving innovation and transformation in the automobile industry, paving the way for safer, more efficient, and sustainable transportation. As technology continues to advance, the integration of machine learning in vehicles will further accelerate, shaping the future of mobility.</span></p><p><span style="font-family: arial;">Here are some ways in which machine learning is transforming the automobile industry:</span></p><p><span style="font-family: arial;"><b>1. Autonomous Driving:</b> Machine learning algorithms are a key component of self-driving cars. These algorithms analyze sensor data from cameras, LiDAR, radar, and other sources to perceive the surrounding environment, detect objects, and make real-time decisions for safe navigation. Machine learning enables autonomous vehicles to learn from data and improve their driving capabilities over time.</span></p><p><span style="font-family: arial;"><b>2. Advanced Driver Assistance Systems (ADAS):</b> Machine learning is used in ADAS to enhance driver safety and convenience. Algorithms can detect lane departures, recognize traffic signs and signals, predict potential collisions, and assist with features like adaptive cruise control, automatic emergency braking, and blind-spot detection.</span></p><p><span style="font-family: arial;"><b>3. Predictive Maintenance:</b> Machine learning algorithms analyze vehicle sensor data to predict maintenance needs and prevent breakdowns. By monitoring various parameters such as engine performance, tire wear, and battery health, machine learning models can identify patterns and anomalies, enabling proactive maintenance scheduling and reducing vehicle downtime.</span></p><p><span style="font-family: arial;"><b>4. Vehicle Diagnostics and Prognostics:</b> Machine learning techniques are employed to diagnose and predict vehicle faults and failures. Algorithms can analyze data from onboard sensors and historical maintenance records to identify potential issues, recommend repairs, and optimize vehicle performance.</span></p><p><span style="font-family: arial;"><b>5. Customer Experience and Personalization: </b>Machine learning enables automakers to provide personalized experiences to customers. By analyzing customer data, including preferences, driving behavior, and usage patterns, machine learning models can customize in-car features, infotainment systems, and assist with navigation, making the driving experience more enjoyable and tailored to individual needs.</span></p><p><span style="font-family: arial;"><b>6. Traffic Prediction and Optimization: </b>Machine learning algorithms can analyze historical traffic data, real-time information, and weather conditions to predict traffic patterns and congestion. This information can be used to optimize routes, improve navigation systems, and enable smarter traffic management for efficient and eco-friendly transportation.</span></p><p><span style="font-family: arial;"><b>7. Vehicle Insurance and Risk Assessment:</b> Machine learning models are used by insurance companies to assess risk factors and determine insurance premiums. By analyzing driver behavior data, including speed, acceleration, and braking patterns, machine learning algorithms can provide accurate risk assessments and enable usage-based insurance policies.</span></p><p><span style="font-family: arial;"><b>8. Supply Chain and Logistics</b>: Machine learning is applied in supply chain management and logistics to optimize inventory management, route planning, and demand forecasting. Algorithms can analyze historical data, market trends, and customer behavior to optimize logistics operations, reduce costs, and improve delivery efficiency.</span></p><p><span style="font-family: arial;"><b>9. Manufacturing and Quality Control:</b> Machine learning is used in automotive manufacturing for quality control and defect detection. Algorithms can analyze sensor data, images, and production parameters to identify anomalies, detect defects, and improve manufacturing processes, ensuring high-quality vehicles.</span></p><p><span style="font-family: arial;"><b>10. Energy Efficiency and Sustainability: </b>Machine learning algorithms can optimize vehicle energy consumption by analyzing driving patterns, road conditions, and battery performance. This helps in improving fuel efficiency, range optimization for electric vehicles, and reducing environmental impact.</span></p><p><span style="font-family: arial;"><b>11. Voice Recognition and Natural Language Processing:</b> Machine learning algorithms enable voice recognition systems in cars, allowing drivers to interact with their vehicles through voice commands. Natural language processing techniques understand and respond to spoken instructions, enabling hands-free operation and enhancing the overall user experience.</span></p><p><span style="font-family: arial;"><b>12. Smart Predictive Navigation:</b> Machine learning algorithms can learn from historical data and user preferences to provide smart predictive navigation. By analyzing factors such as time of day, traffic conditions, and personal habits, the navigation system can offer real-time route suggestions, optimize travel times, and provide personalized recommendations for stops along the way.</span></p><p><span style="font-family: arial;"><b>13. Gesture Recognition:</b> Machine learning enables gesture recognition systems in vehicles, allowing drivers to control various functions with hand gestures. This hands-free interaction enhances safety and convenience, as drivers can adjust settings, answer calls, or control infotainment systems without taking their hands off the steering wheel.</span></p><p><span style="font-family: arial;"><b>14. Fleet Management and Optimization:</b> Machine learning algorithms assist in fleet management by optimizing routes, managing vehicle assignments, and predicting maintenance needs. Fleet operators can utilize machine learning models to improve operational efficiency, reduce fuel consumption, and enhance overall fleet performance.</span></p><p><span style="font-family: arial;"><b>15. Smart Parking Solutions:</b> Machine learning is applied to develop smart parking systems that use sensor data and historical patterns to predict parking availability. Drivers can access real-time parking information, locate available spaces, and minimize the time spent searching for parking, reducing traffic congestion and emissions.</span></p><p><span style="font-family: arial;"><b>16. Cybersecurity and Threat Detection:</b> Machine learning techniques are used to enhance automotive cybersecurity. Algorithms can detect and mitigate potential cyber threats, such as unauthorized access or malware attacks, safeguarding vehicle systems and protecting sensitive data.</span></p><p><span style="font-family: arial;"><b>17. Vehicle-to-Vehicle Communication:</b> Machine learning facilitates vehicle-to-vehicle (V2V) communication, enabling cars to share real-time data with each other. This data exchange allows vehicles to coordinate movements, share traffic information, and improve safety by detecting potential collision risks.</span></p><p><span style="font-family: arial;"><b>18. Energy Harvesting and Efficiency:</b> Machine learning algorithms can optimize energy harvesting in electric vehicles by analyzing factors such as driving patterns, terrain, and energy consumption. This enables efficient energy regeneration, improved battery management, and extended range for electric vehicles.</span></p><p><span style="font-family: arial;"><b>19. Emotion Recognition and Driver Monitoring: </b>Machine learning models can analyze driver behavior and facial expressions to recognize emotions and monitor driver attention and fatigue levels. This technology can help detect drowsiness, distraction, or aggressive driving, promoting safer driving practices and reducing accidents.</span></p><p><span style="font-family: arial;"><b>20. Connected Car Analytics:</b> Machine learning is utilized to analyze the vast amount of data generated by connected cars. These insights can be leveraged for personalized marketing, vehicle performance monitoring, predictive maintenance, and the development of new services and features tailored to customer needs.</span></p><p><span style="font-family: arial;">Machine learning continues to push the boundaries of innovation in the automobile industry, transforming the way vehicles are designed, manufactured, operated, and experienced. With ongoing advancements, we can expect further improvements in safety, efficiency, connectivity, and sustainability in the automotive sector.</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-36008364391065598742023-05-30T09:19:00.002-07:002023-05-30T09:20:14.300-07:00How is Machine Learning transforming and revolutionizing the healthcare sector?<p><span style="font-family: arial;">Machine learning is having a transformative impact on the healthcare sector. </span><span style="font-family: arial;">Machine learning is revolutionizing healthcare by enhancing diagnosis accuracy, personalizing treatments, improving patient outcomes, and optimizing healthcare delivery. As the field continues to advance, the potential for machine learning in healthcare is vast, paving the way for more efficient, precise, and patient-centered care.</span></p><p><span style="font-family: arial;">Here are some ways in which machine learning is changing healthcare.</span></p><p><span style="font-family: arial;"><b>1. Disease Diagnosis:</b> Machine learning algorithms can analyze medical images such as X-rays, CT scans, and MRIs to assist in disease diagnosis. They can detect patterns and anomalies that may not be easily identifiable by human experts, leading to more accurate and timely diagnoses.</span></p><p><span style="font-family: arial;"><b>2. Personalized Treatment: </b>Machine learning models can analyze patient data, including medical records, genetic information, and lifestyle factors, to personalize treatment plans. This enables healthcare providers to deliver targeted interventions and optimize treatment outcomes for individual patients.</span></p><p><span style="font-family: arial;"><b>3. Predictive Analytics: </b>Machine learning algorithms can analyze large datasets and identify patterns to predict disease progression, readmission rates, and adverse events. This helps healthcare providers intervene early, prevent complications, and improve patient outcomes.</span></p><p><span style="font-family: arial;"><b>4. Drug Discovery and Development:</b> Machine learning is being used to accelerate the drug discovery and development process. By analyzing vast amounts of biological and chemical data, machine learning models can identify potential drug candidates, predict their efficacy, and optimize drug design.</span></p><p><span style="font-family: arial;"><b>5. Precision Medicine:</b> Machine learning enables the identification of patient subgroups with similar characteristics, leading to more precise and targeted treatments. It can also help predict how individual patients will respond to specific therapies, allowing for personalized medicine approaches.</span></p><p><span style="font-family: arial;"><b>6. Remote Monitoring and Telemedicine:</b> Machine learning algorithms can analyze patient data collected from wearables, sensors, and mobile devices to monitor health remotely. This enables the early detection of abnormalities, facilitates remote consultations, and improves access to healthcare in remote areas.</span></p><p><span style="font-family: arial;"><b>7. Electronic Health Records (EHR) Management:</b> Machine learning can automate the processing and analysis of electronic health records. It helps healthcare providers extract valuable insights, improve documentation accuracy, and streamline administrative tasks, leading to more efficient healthcare workflows.</span></p><p><span style="font-family: arial;"><b>8. Fraud Detection and Healthcare Billing: </b>Machine learning algorithms can analyze healthcare data to detect fraudulent activities, such as insurance fraud or improper billing practices. This helps reduce financial losses and ensures accurate billing and reimbursement processes.</span></p><p><span style="font-family: arial;"><b>9. Virtual Assistants and Chatbots:</b> Machine learning powers virtual assistants and chatbots that can interact with patients, answer their queries, and provide basic medical guidance. They enhance patient engagement, provide on-demand support, and assist in triaging healthcare needs.</span></p><p><span style="font-family: arial;"><b>10. Health Monitoring and Early Warning Systems:</b> Machine learning algorithms can analyze real-time patient data to detect early warning signs of deteriorating health. This helps healthcare providers intervene promptly, prevent emergencies, and improve patient safety.</span></p><p><span style="font-family: arial;"><b>11. Medical Imaging Analysis:</b> Machine learning algorithms are increasingly being used to analyze medical images such as mammograms, ultrasounds, and pathology slides. These algorithms can assist radiologists in detecting abnormalities, identifying specific types of cancer, and reducing false positives and false negatives.</span></p><p><span style="font-family: arial;"><b>12. Clinical Decision Support Systems:</b> Machine learning models can provide decision support to healthcare professionals by analyzing patient data, medical literature, and treatment guidelines. These systems can suggest optimal treatment options, predict patient outcomes, and provide evidence-based recommendations to assist clinicians in making informed decisions.</span></p><p><span style="font-family: arial;"><b>13. Predictive Risk Modeling:</b> Machine learning algorithms can identify patients at high risk of developing specific conditions or complications. By analyzing various data sources, such as electronic health records, genetic information, and lifestyle factors, these models can predict the likelihood of diseases such as diabetes, cardiovascular diseases, or sepsis. This enables proactive interventions and preventive measures.</span></p><p><span style="font-family: arial;"><b>14. Health Monitoring Wearables: </b>Machine learning plays a crucial role in analyzing data from wearable devices, such as fitness trackers and smartwatches. These algorithms can track vital signs, activity levels, sleep patterns, and other health-related metrics. By monitoring and analyzing this data, healthcare professionals can gain insights into a patient's overall health status, detect anomalies, and provide personalized recommendations.</span></p><p><span style="font-family: arial;"><b>15. Medical Research and Clinical Trials:</b> Machine learning is facilitating medical research and clinical trials by enabling the analysis of large datasets and identifying potential correlations or treatment responses. It can assist in patient recruitment for clinical trials, optimize trial design, and contribute to the discovery of new biomarkers or treatment targets.</span></p><p><span style="font-family: arial;"><b>16. Health Data Management and Privacy:</b> Machine learning techniques can help manage and secure vast amounts of health data. They can assist in data anonymization, patient de-identification, and data access controls to ensure privacy and comply with data protection regulations such as HIPAA (Health Insurance Portability and Accountability Act).</span></p><p><span style="font-family: arial;"><b>17. Disease Outbreak Prediction and Monitoring:</b> Machine learning models can analyze various data sources, including social media, internet search trends, and sensor data, to detect early signs of disease outbreaks. By monitoring patterns and identifying geographical hotspots, public health authorities can take timely preventive measures and allocate resources effectively.</span></p><p><span style="font-family: arial;"><b>18. Chronic Disease Management: </b>Machine learning algorithms can assist in the management of chronic diseases such as diabetes, asthma, and hypertension. By analyzing patient data, these algorithms can provide personalized recommendations, predict disease exacerbations, and support self-management efforts, leading to better patient outcomes.</span></p><p><span style="font-family: arial;"><b>19. Patient Risk Stratification:</b> Machine learning models can assess patient risk scores by considering various factors such as medical history, demographics, and genetic data. These risk stratification algorithms can help healthcare providers prioritize interventions, allocate resources efficiently, and improve population health management.</span></p><p><span style="font-family: arial;"><b>20. Healthcare Operations Optimization:</b> Machine learning can be applied to optimize healthcare operations, including resource allocation, bed management, and staff scheduling. By analyzing historical data, patient flow, and demand patterns, machine learning models can help hospitals and healthcare facilities streamline processes, reduce wait times, and improve efficiency.</span></p><p><span style="font-family: arial;">These are just a few examples of how machine learning is transforming the healthcare sector. With the increasing availability of healthcare data and advancements in machine learning techniques, the potential for improving patient care, enhancing diagnostics, and revolutionizing healthcare delivery is immense. </span></p><span style="font-family: arial;"></span><p></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-54784318523260923702021-04-29T05:36:00.007-07:002021-05-04T07:03:54.326-07:00100 basic terms and comparisons from PMBOK which a PMP certified project manager must know<p style="text-align: justify;"><span style="font-family: arial;">A PMP Certified Project Manager should be knowing following comparisons. These comparisons can be asked in a project management interview for PMP certified project managers. I have divided all these terms and comparisons based upon the knowledge areas.</span></p><p><span style="font-family: arial;"><b>General Introduction</b></span></p><p><span style="font-family: arial;"><b>1)</b> Project vs Operation</span></p><p><span style="font-family: arial;"><b>2)</b> Product Life Cycle vs Project Life Cycle</span></p><p><span style="font-family: arial;"><b>3)</b> Project Phase vs Phase Gate</span></p><p><span style="font-family: arial;"><b>4)</b> Development Approach: Predictive vs Adaptive vs Hybrid vs Iterative vs Incremental</span></p><p><span style="font-family: arial;"><b>5)</b> Portfolio Management vs Program Management vs Project Management</span></p><p><span style="font-family: arial;"><b>6)</b> Organizational Structure Type: Strong Matrix vs Weak Matrix vs Balanced Matrix</span></p><p><span style="font-family: arial;"><b>7)</b> Types of PMO: Supportive vs Controlling vs Directive</span></p><p><span style="font-family: arial;"><b>8)</b> Powers of Project Manager: Referent vs Positional vs Situational vs Coercive vs Ingratiating vs Persuasive</span></p><p><span style="font-family: arial;"><b>9)</b> Leadership Styles: Laissez-faire vs Servant Leader vs Transactional vs Transformational vs Charismatic vs Interactional </span></p><p><span style="font-family: arial;"><b>10)</b> Project Manager vs Project Coordinator vs Project Expeditor</span></p><p><span style="font-family: arial;"><b>11)</b> PMI Triangle: Technical Project Management vs Strategic and Business Management vs Leadership</span></p><p><span style="font-family: arial;"><b>12)</b> Code of Ethics and Professional Conduct: Aspirational Standards and Mandatory Standards</span></p><p><span style="font-family: arial;"><b>Integration Management</b></span></p><p><span style="font-family: arial;"><b>13)</b> OPA (Organizational Process Assets) vs EEF (Enterprise Environmental Factors)</span></p><p><span style="font-family: arial;"><b>14)</b> Business Case vs Benefits Management Plan</span></p><p><span style="font-family: arial;"><b>15)</b> Configuration Management Plan vs Change Management Plan</span></p><p><span style="font-family: arial;"><b>16)</b> Project Charter vs Team Charter</span></p><p><span style="font-family: arial;"><b>17)</b> Assumption log: Assumptions vs Constraints</span></p><p><span style="font-family: arial;"><b>18)</b> Change Requests: Preventive actions vs Corrective actions vs Defect repair</span></p><p><span style="font-family: arial;"><b>19)</b> Knowledge Management vs Information Management</span></p><p><span style="font-family: arial;"><b>20)</b> Explicit Knowledge vs Tacit Knowledge</span></p><p><span style="font-family: arial;"><b>21)</b> Work/Job Shadowing vs Reverse Shadowing</span></p><p><span style="font-family: arial;"><b>22)</b> Work Performance Data vs Work Performance Information vs Work Performance Report</span></p><p><span style="font-family: arial;"><b>23)</b> Work Performance Reports: Status report vs Progress report</span></p><p><span style="font-family: arial;"><b>24)</b> Payback Period vs Return on Investment (ROI) vs Internal Rate of Return (IRR) vs Discounted Cash Flow vs Net Present Value (NPV) vs Benefit Cost Ratio (BCR)</span></p><p><span style="font-family: arial;"><b>Scope Management</b></span></p><p><span style="font-family: arial;"><b>25)</b> Project Charter vs Project Scope Statement</span></p><p><span style="font-family: arial;"><b>26)</b> WBS: Work Package vs Planning Package</span></p><p><span style="font-family: arial;"><b>27)</b> WBS: Work Package vs Activities</span></p><p><span style="font-family: arial;"><b>28)</b> Scope Baseline vs Schedule Baseline vs Cost Baseline vs Performance Measurement Baseline</span></p><p><span style="font-family: arial;"><b>29)</b> Validate Scope Process vs Control Quality Process</span></p><p><span style="font-family: arial;"><b>30)</b> Scope Creep and Gold Plating</span></p><p><span style="font-family: arial;"><b>31)</b> Product Scope vs Project Scope</span></p><p><span style="font-family: arial;"><b>32)</b> Inspection vs Audit</span></p><p><span style="font-family: arial;"><b>33)</b> Verification vs Validation</span></p><p><span style="font-family: arial;"><b>34)</b> Work Breakdown Structure vs Resource Breakdown Structure vs Risk Breakdown Structure vs Organizational Breakdown Structure</span></p><p><span style="font-family: arial;"><b>35)</b> Brainstorming vs Brain-writing vs Nominal Group Technique</span></p><p><span style="font-family: arial;"><b>36)</b> Decision Making: Autocratic vs Multi-criteria vs Voting (Unanimity vs Majority vs Plurality)</span></p><p><span style="font-family: arial;"><b>Schedule Management</b></span></p><p><span style="font-family: arial;"><b>37)</b> Iterative Scheduling vs On-demand Scheduling</span></p><p><span style="font-family: arial;"><b>38)</b> Precedence Diagramming Method vs Arrow Diagramming Method</span></p><p><span style="font-family: arial;"><b>39)</b> Finish to Start (FS) vs Finish to Finish (FF) vs Start to Start (SS) vs Start to Finish (SF)</span></p><p><span style="font-family: arial;"><b>40)</b> Dependencies: Mandatory vs Discretionary vs External vs Internal</span></p><p><span style="font-family: arial;"><b>41)</b> Schedule Network Diagram: Path Convergence vs Path Divergence</span></p><p><span style="font-family: arial;"><b>42) </b>Critical Path Method vs Critical Chain Method</span></p><p><span style="font-family: arial;"><b>43)</b> Early Start vs Early Finish vs Late Start vs Late Finish</span></p><p><span style="font-family: arial;"><b>44)</b> Forward Pass vs Backward Pass</span></p><p><span style="font-family: arial;"><b>45)</b> Leads vs Lags</span></p><p><span style="font-family: arial;"><b>46) </b>Total Float vs Free Float</span></p><p><span style="font-family: arial;"><b>47)</b> Project Schedule: Gantt charts vs Bar charts vs Milestone charts vs Summary charts</span></p><p><span style="font-family: arial;"><b>48)</b> Fast-tracking vs Crashing</span></p><p><span style="font-family: arial;"><b>49) </b>Resource Optimization vs Resource Levelling</span></p><p><span style="font-family: arial;"><b>50)</b> Project Calendar vs Resource Calendar</span></p><p><span style="font-family: arial;"><b>51)</b> Contingency Reserve (Known Unknown) vs Management Reserve (Unknown Unknown)</span></p><p><span style="font-family: arial;"><b>52)</b> Triangular Distribution vs PERT/Beta Distribution</span></p><p><span style="font-family: arial;"><b>53)</b> Analogous Estimating vs Parametric Estimating vs Three-point Estimating vs Bottom-up Estimating</span></p><p><span style="font-family: arial;"><b>54)</b> Accuracy and Precision</span></p><p><span style="font-family: arial;"><b>55)</b> Activity List vs Milestone List</span></p><p><span style="font-family: arial;"><b>56)</b> Student Syndrome vs Parkinson's Law</span></p><p><span style="font-family: arial;"><b>Cost Management</b></span></p><p><span style="font-family: arial;"><b>57) </b>Rough Order of Magnitude Estimate (-25% to +75%) vs Budgeted Estimate (-10% to +25%) vs Definitive Estimate (-5% to +10%)</span></p><p><span style="font-family: arial;"><b>58) </b>Planned Value (PV) vs Earned Value (EV) vs Actual Cost (AC)</span></p><p><span style="font-family: arial;"><b>59)</b> Cost variance (CV = EV - AC) vs Schedule variance (SV = EV - PV)</span></p><p><span style="font-family: arial;"><b>60)</b> Cost Performance Index (CPI = EV/AC) vs Schedule Performance Index (SPI = EV/PV)</span></p><p><span style="font-family: arial;"><b>61)</b> Budget at Completion (BAC) vs Estimate at Completion (EAC) vs Variance at Completion (VAC = BAC - EAC)</span></p><p><span style="font-family: arial;"><b>62)</b> EAC Calculation: EAC forecast for ETC work performed at the budgeted rate (EAC = AC + (BAC – EV)) vs EAC forecast for ETC work performed at the present CPI (EAC = BAC / CPI) vs EAC forecast for ETC work considering both SPI and CPI factors (EAC = AC + [(BAC – EV) / (CPI × SPI)]) vs To-complete performance index (TCPI = Work Remaining / Fund Remaining)</span></p><p><span style="font-family: arial;"><b>Quality Management</b></span></p><p><span style="font-family: arial;"><b>63) </b>Quality vs Grade</span></p><p><span style="font-family: arial;"><b>64) </b>Attribute Sampling vs Variable Sampling</span></p><p><span style="font-family: arial;"><b>65) </b>Continual Improvement: Plan-do-check-act (PDCA) vs Total Quality Management(TQM) vs Six Sigma vs Lean Six Sigma</span></p><p><span style="font-family: arial;"><b>66)</b> Cost of Conformance vs Cost of Non-conformance</span></p><p><span style="font-family: arial;"><b>67)</b> Cost of Conformance: Prevention cost vs Appraisal cost</span></p><p><span style="font-family: arial;"><b>68)</b> Cost of Non-Conformance: Internal Failure Cost vs External Failure Cost</span></p><p><span style="font-family: arial;"><b>69)</b> Control Charts vs Flowcharts vs Histograms vs Cause-and-effect Diagrams vs Matrix Diagrams vs Scatter Diagrams vs Affinity Diagrams vs Mind Mapping</span></p><p><span style="font-family: arial;"><b>Resource Management</b></span></p><p><span style="font-family: arial;"><b>70)</b> Matrix: Responsibility Assignment Matrix (RAM / RACI) vs Probability and Impact Matrix vs Stakeholder Engagement Assessment Matrix</span></p><p><span style="font-family: arial;"><b>71)</b> Tuckman Ladder: Forming vs Storming vs Norming vs Performing vs Adjourning</span></p><p><span style="font-family: arial;"><b>72)</b> Co-located Teams vs Virtual Teams</span></p><p><span style="font-family: arial;"><b>73) </b>Conflict Management: Withdraw/Avoid vs Smooth/Accommodate vs Compromise/Reconcile (Lose-Lose) vs Force/Direct (Win-Lose) vs Collaborate/Problem Solve (Win-Win)</span></p><p><span style="font-family: arial;"><b>74)</b> Develop Team vs Manage Team</span></p><p><span style="font-family: arial;"><b>75)</b> Coaching vs Mentoring</span></p><p><span style="font-family: arial;"><b>Communication Management</b></span></p><p><span style="font-family: arial;"><b>76)</b> Push vs Pull communication</span></p><p><span style="font-family: arial;"><b>77)</b> Verbal vs Non-verbal communication (Paralinguistic, Vocal inflection, Pitch, Tone)</span></p><p><span style="font-family: arial;"><b>Risk Management</b></span></p><p><span style="font-family: arial;"><b>78) </b>Risks vs Issues</span></p><p><span style="font-family: arial;"><b>79)</b> Event vs Non-event Risks</span></p><p><span style="font-family: arial;"><b>80)</b> Variability Risk vs Ambiguity Risk vs Emergent Risks (Unknowable Unknowns)</span></p><p><span style="font-family: arial;"><b>81)</b> Secondary risk vs Residual risk</span></p><p><span style="font-family: arial;"><b>82)</b> Risk Register vs Risk Report</span></p><p><span style="font-family: arial;"><b>83)</b> Opportunities (Positive Risks) vs Threats (Negative Risks)</span></p><p><span style="font-family: arial;"><b>84) </b>Strategies for Opportunities: Exploit vs Enhance vs Accept vs Share</span></p><p><span style="font-family: arial;"><b>85)</b> Strategies for Threats: Avoid vs Mitigate vs Accept vs Transfer</span></p><p><span style="font-family: arial;"><b>86)</b> Active Risk Acceptance vs Passive Risk Acceptance</span></p><p><span style="font-family: arial;"><b>87)</b> Qualitative vs Quantitative Risk Analysis</span></p><p><span style="font-family: arial;"><b>88) </b>Quantitative Risk Analysis: Simulations (Monte Carlo Analysis) vs Sensitivity Analysis (Tornado Diagrams) vs Decision Tree Analysis vs Influence Diagrams</span></p><p><span style="font-family: arial;"><b>89)</b> Prompt Lists: PESTLE (political, economic, social, technological, legal, environmental) vs TECOP (technical, environmental, commercial, operational, political) vs VUCA (volatility, uncertainty, complexity, ambiguity)</span></p><p><span style="font-family: arial;"><b>90)</b> Risk Assessment Parameters: Urgency vs Proximity vs Dormancy vs Propinquity vs Detectability</span></p><p><span style="font-family: arial;"><b>Procurement Management</b></span></p><p><span style="font-family: arial;"><b>91)</b> Contract Types: Fixed Price vs Cost Reimbursable vs Time and Material</span></p><p><span style="font-family: arial;"><b>92)</b> Fixed Price Contracts: Firm Fixed Price (FFP) vs Fixed Price Incentive Fee (FPIF) vs Fixed Price with Economic Price Adjustments (FPEPA)</span></p><p><span style="font-family: arial;"><b>93)</b> Cost-reimbursable Contracts: Cost plus fixed fee (CPFF) vs Cost plus incentive fee (CPIF) vs Cost plus award fee (CPAF)</span></p><p><span style="font-family: arial;"><b>94)</b> Bid Documents: RFI (Request for Information) vs RFQ (Request for Quotation) vs RFP (Request for Proposal)</span></p><p><span style="font-family: arial;"><b>95) </b>Scope Statement vs Statement of Work (SOW) vs Terms of Reference (TOR)</span></p><p><span style="font-family: arial;"><b>96)</b> Make vs Buy Analysis</span></p><p><span style="font-family: arial;"><b>97)</b> Selecting a Seller vs Awarding a Contract</span></p><p><span style="font-family: arial;"><b>98)</b> Claims Administration: Claims vs Disputes vs Appeals</span></p><p><span style="font-family: arial;"><b>Stakeholder Management</b></span></p><p><span style="font-family: arial;"><b>99) </b>Stakeholder Analysis: Power/Interest vs Power/Influence vs Impact/Influence</span></p><p><span style="font-family: arial;"><b>100)</b> Engagement level of stakeholders: Unaware vs Resistant vs Neutral vs Supportive vs Leading</span></p>Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-55231197729684721462019-09-11T20:21:00.000-07:002019-09-11T20:55:09.482-07:00Preprocessing of raw data in NLP: Tokenization, Stemming, Lemmatization and Vectorization<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">NLP (Natural Language Processing) is a very interesting branch of Artificial Intelligence. Natural language is a language which we human use to communicate and interact with each other. In NLP, we are teaching computers to understand, interpret and manipulate human languages. In this article, we will focus on some of the preprocessing tasks which we perform on the raw data like Tokenization, Stemming, Lemmatization and Vectorization.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">While processing a natural language which we human speak, we need to take care of following things:</span><br />
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. Syntax:</b> The sentence should be grammatically correct. The arrangement of words in a sentence should follow all the grammar rules defined by a language. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Semantics:</b> It deals with the meaning of words and their interpretation within sentences.</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<br /><b>3. Pragmatics:</b> Same as semantics but it also consider context in which the word is used.<br />
<br />
<b>Applications of NLP</b><br />
<b></b><br />
Applications of NLP (Natural Language Processing) are unlimited. I have listed few of those:<br />
<br /><b>1.</b> Machine translation (like Google Translate)<br /><b>2</b>. Sentiment analysis (reviews and comments on e-commerce and social-networking sites)<br /><b>3</b>. Text classification, generation and automatic summarization</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4.</b> Automated question answering and conversational interfaces (like chatbots)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>5.</b> Personal assistants (like Alexa, Siri, Google Assistant, Cortana etc.)</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>6.</b> Auto-correct grammatical mistakes (MS Word and Grammarly use NLP to check grammatical errors)</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<b>7.</b> Spam filtering, auto-complete, auto-correct, auto-tagging, topic modelling, sentence segmentation, speech recognition, part of speech tagging, named entity recognition, duplicates detection and a lot more...<br />
<br />
<b>NLP Toolkit (library) in Python</b></span><br />
<b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">There are a lot of libraries in Python for NLP but the most commonly used library is NLTK( Natural Language Toolkit). It provides very efficient modules for preprocessing and cleaning of raw data like removing punctuation, tokenizing, removing stopwords, stemming, lemmatization, vectorization, tagging, parsing, and more.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
<b>Pre-processing of raw data in NLP</b></span><br />
<b></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
Following are the basic steps which we need to perform while cleaning the raw data in NLP:<br /><br />
<b>1. Remove Punctuation<br />2. Tokenization<br />3. Remove Stopwords</b></span><br />
<div>
<b><span style="font-family: "arial" , "helvetica" , sans-serif;">4. Stemming / Lemmatization<br />5. Vectorization</span></b></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<i></i><u></u><sub></sub><sup></sup><strike></strike><b><br />1. Remove Punctuation:</b> First of all, we should remove all the punctuation marks (like comma, semicolon, colon, apostrophe, quotation marks, dash, hyphen, brackets, braces, parentheses, ellipsis etc.) from the text as these carry negligible weight.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Tokenization:</b> Now create a list of words used in the text. Each word is called a token. We can use regular expression to find out tokens from the sentences otherwise NLTK has efficient modules for this task.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Remove Stopwords:</b> Now we need to remove all the stopwords from the token list. Stopwords are the words which occur frequently in a sentence but carry little weight (like the, for, is, and, or, been, to, this, that, but, if, in, a, as etc.).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4.1 Stemming:</b> It is used to reduce the number of tokens just like removing stopwords. In this process, we reduce inflected words to their word stem or root. We keep only the semantic meaning of similar words.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Examples:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1)</b> Tokens like stemming and stemmed are converted to a token <b>stem</b>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2) </b>Tokens like working, worked, works and work are converted to a token <b>work</b>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Points 1 and 2 clearly illustrate that how can we reduce the number of tokens in a token list using stemming. But wait! There is a problem. Consider following examples of stemming:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<div>
<b>3)</b> Tokens like meanness and meaning are converted to a token <b>mean</b>. Now this is wrong. Both tokens have different meanings, even then its treating both as same.<br />
<br />
<b>4)</b> Tokens like goose and geese are converted to the tokens <b>goos</b> and <b>gees</b> respectively (it will just remove "e" suffix from both the tokens). Now this is again wrong. "geese" is just a plural of "goose", even then its treating both tokens as different.<br />
<br />
Points 3 and 4 can be resolved using Lemmatization.<br />
<br />
NLTK library has 4 stemmers:<br />
<br />
<b>1) Porter Stemmer<br />2) Snowball Stemmer<br />3) Lancaster Stemmer<br />4) Regex-based Stemmer</b><br />
<b></b><br />
I mainly use Porter stemmer for stemming the tokens in my NLP code.<br />
<br />
<b>4.2: Lemmatization:</b> We saw the limitation of stemming in above examples (3 and 4). We can overcome these limitations using Lemmatization. It is more powerful and sophisticated as compared to stemming and returns more accurate and meaningful words / tokens by considering the context in which the word is used in a sentence.<br />
<br />
But the tradeoff is that, it is slower and complex as compared to the stemming.<br />
<br />
<b>Examples: </b><br />
<b><br /></b>
<b>1)</b> Tokens like meanness and meaning are retained as it is instead of reducing it to <b>mean</b> (unlike stemming).<br />
<br />
<b>2)</b> Tokens like goose and geese are converted to a token <b>goose</b> which is right. We should get rid of the token "geese" as it is just a plural of "goose".<br />
<br />
I mainly use WordNet Lemmatizer present in NLTK library.<br />
<br />
<b>5. Vectorization:</b> Machine Learning algorithms don't understand text. These need numeric data for matrix multiplications. Till now, we have just cleaned our tokens. So, in this process, we encode our final tokens into numbers to create feature vectors so that algorithms can understand. In other words, we will fit and transform vectorization methods to our preprocessed and cleaned data which we created till lemmatization.<br />
<b><br /></b>
<b>Document-term matrix:</b> Let's first understand this term before proceeding further. We use document term matrix to represent the words in the text in the form of matrix of numbers. The rows of the matrix represent the text responses to be analyzed, and the columns of the matrix represent the words / tokens from the text that are to be used in the analysis. </div>
<div>
<br /></div>
<div>
<b>Types of Vectorization</b><br />
<b><br /></b>
There are mainly 3 types vectorization:<br />
<br />
<b>1) Count vectorization<br />2) N-grams vectorization<br />3) Term Frequency - Inverse Document Frequency (TF-IDF)</b><br />
<b></b><br />
<b>1) Count vectorization:</b> It creates a document-term matrix which contains the count of each unique word / token in the text response.<br />
<b></b><br />
<b>2) N-grams vectorization:</b> It creates a document-term matrix which also considers context of the word depending upon the value of N.<br />
<br />
If N = 2, it is called bi-gram,<br />
If N = 3, it is called tri-gram,<br />
If N = 4, it is called four-gram and so on...<br />
<br />
We need to be careful about value of N and choose it wisely.<br />
<br />
Example: Consider a sentence "<i>NLP is awesome</i>". Count vectorization will create a column corresponding to each word in document-term matrix while N-gram will create columns like following in case of bi-gram:<br />
<br />
"NLP is", "is awesome"<br />
<br />
<b>3) Term Frequency - Inverse Document Frequency (TF-IDF)</b> - It is just like count vectorization but instead of count, it stores weightage of each word by using following formula:<br />
<br />
<b></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik9Z3DmtVMPAAJHAkjJIFrXdiAPYRlDG5FdybOOxCDq2eSCDSYpBiy6mVaq4RqKP0J6hRrcoj5xYtY_3tfLbfApanPM9ZQgyMkYEXIuA26WpPqD8J9f8kvPe6i6DNvlPfbN7jzdJB-xhuF/s1600/inverse_document_frequency.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="91" data-original-width="325" height="88" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik9Z3DmtVMPAAJHAkjJIFrXdiAPYRlDG5FdybOOxCDq2eSCDSYpBiy6mVaq4RqKP0J6hRrcoj5xYtY_3tfLbfApanPM9ZQgyMkYEXIuA26WpPqD8J9f8kvPe6i6DNvlPfbN7jzdJB-xhuF/s320/inverse_document_frequency.PNG" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b><br /></b>
<b>w(i, j)</b> = weightage of a particular word "i" in a document "j"</div>
<div>
<br /></div>
</span><div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>tf(i, j)</b> = term frequency of a word "i" in document "j" i.e. number of times the word "i" occurs in a document "j" divided by total number of words in document "j"</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>N</b> = number of total documents</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>df(i)</b> = number of documents containing the word "i"</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So, in this way, TF-IDF considers two facts while calculating the weightage of a word or token: </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1)</b> how frequent the word occurs in a particular document </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2)</b> and how frequent that word occurs in other documents</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Example</b>: Consider that we have 10 text messages and one of the text messages is "<i>NLP is awesome</i>". No other message contains the word "NLP". Now lets calculate weightage of the word NLP.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>tf(i, j)</b> = number of times the word NLP occurs in the text message divided by the total number of words in the text message. It comes out to be <b>(1/3) </b>as there are three words and NLP occurs only one time.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>N</b> = <b>10</b> as there are 10 text messages. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>df(i)</b> = number of text messages containing the word NLP which in our case is <b>1</b>.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">So, the final equation becomes:</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Weightage of NLP = (1/3) * log(10/1)</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this way, we fill all the rows and column of document-term matrix in TF-IDF.</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-92164776825368783292019-09-05T21:37:00.001-07:002019-09-05T21:47:37.997-07:00Image Recognition: Text Detection (Optical Character Recognition) using Google Cloud Vision API<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Google Cloud Vision API helps in label detection, face detection, logo detection, landmark detection and text detection (OCR: Optical Character Recognition). In this article, we will see how can we use Google Cloud Vision API to extract the text from the image? This is a step by step guide for text detection (OCR) using Google Cloud Vision API. Let's follow it.</span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">I will directly start from step 5. First 4 steps are same as mentioned in <a href="http://theprofessionalspoint.blogspot.com/2019/09/image-recognition-label-detection-using.html">my previous post</a> on label detection using Google Cloud Vision API.</span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">You can download my Jupyter notebook containing below code from <a href="https://drive.google.com/file/d/13atJK8EaFWPbsMIKhGdCDey7sZQ58TLb/view?usp=sharing">here</a>.</span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Import required libraries</b></span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<br /></div>
<div style="color: black; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; margin: 0px; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;">from googleapiclient.discovery import build<br />from oauth2client.client import GoogleCredentials<br />from base64 import b64encode</span></div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You may get import error "no module name..." if you have not already installed Google API Python client. Use following command to install it.</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>pip install --upgrade google-api-python-client</b></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If you also get import error for oauth2client, you must install it using following command:</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>pip3 install --upgrade oauth2client </b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Load credentials file</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Load the credentials file (which we created in step 3 of <a href="http://theprofessionalspoint.blogspot.com/2019/09/image-recognition-label-detection-using.html">my previous article</a>) and create a service object using it.</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="background-color: white;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">CREDENTIAL_FILE = 'credentials.json'<br />credentials = GoogleCredentials.from_stream(CREDENTIAL_FILE)<br />service = build('vision', 'v1', credentials=credentials)</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 7: Load image file (from which we need to extract the text)</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">I will load an image of cover page of my deep learning book and encode it so that it becomes compatible with the cloud vision API.</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
</div>
<div class="separator" style="clear: both; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieusUl-W4jhPQC-cJulS9S_VW_iXbhKkS6kIav1fegJPTKDFnP8f2oWnI_BmNbyw4VdEZpxBpcbGKm0E33f-aZPZCzrTfOISljb-EttmvNfbDbNrIuvqkkbmWgzGpxcv7f6M1CW5uBHIO7/s1600/deep_learning_ebook_cover_page.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1067" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieusUl-W4jhPQC-cJulS9S_VW_iXbhKkS6kIav1fegJPTKDFnP8f2oWnI_BmNbyw4VdEZpxBpcbGKm0E33f-aZPZCzrTfOISljb-EttmvNfbDbNrIuvqkkbmWgzGpxcv7f6M1CW5uBHIO7/s400/deep_learning_ebook_cover_page.png" width="265" /></a></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">IMAGE_FILE = book_cover_page.jpg'</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">with open(IMAGE_FILE, 'rb') as file:<br /> image_data = file.read()<br /> encoded_image_data = b64encode(image_data).decode('UTF-8')</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 8: Create a batch request</b></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a batch request which we will send to the cloud vision API. In the batch request, we will include the above encoded image and the instruction as </span><span style="background-color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>TEXT</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><b>_DETECTION</b>.</span></span></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><span style="background-color: white;"></span><span style="background-color: white;"></span><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">batch_request = [{<br /> 'image':{'content':encoded_image_data},<br /> 'features':[{'type':'<b>TEXT_DETECTION</b>'}],<br />}]</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 9: Create a request</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">request = service.images().annotate(body={'requests':batch_request})</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 10: Execute the request</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">response = request.execute()</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">This step will throw an error if you have not enabled billing (as mentioned in step 4 of <span style="background-color: white; color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><a href="http://theprofessionalspoint.blogspot.com/2019/09/image-recognition-label-detection-using.html">my previous article</a></span>). So, you must enable the billing in order to use Google Cloud Vision API. The charges are very reasonable. So, don't think too much and provide credit card details. For me, Google charged INR 1 and then refunded it back.</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 11: Process the response</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">For error handling, include this code:</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">if 'error' in response:<br /> raise RuntimeError(response['error'])</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are interested in text annotations here. So, fetch it from the response and display the results.</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">labels = response['responses'][0][<b>'textAnnotations'</b>]</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">extracted_text = extracted_texts[0]<br />print(extracted_text['description'], extracted_text['boundingPoly'])</span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Output:</b></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;">Objective Type Questions and Answers in Deep Learning<br />Deep<br />Learning<br />ARTIFICIAL<br />INTELLIGENCE<br />MACHINE<br />LEARNING<br />DEEP<br />LEARNING<br />NARESH KUMAR<br /> {'vertices': [{'x': 42, 'y': 77}, {'x': 2365, 'y': 77}, {'x': 2365, 'y': 3523}, {'x': 42, 'y': 3523}]}</span></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><span style="background-color: yellow;"></span><br /></span></div>
</div>
<div style="color: black; font-family: "times new roman"; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can test the above code using different images and check the accuracy of the API.</span></div>
</div>
</div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-19060562995086626552019-09-04T18:38:00.000-07:002019-09-05T21:48:53.830-07:00Image Recognition: Label Detection using Google Cloud Vision API<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Google Cloud Vision API helps in label detection, face detection, logo detection, landmark detection and text detection. In this article, we will see how can we use Google Cloud Vision API to identify labels in the image? This is a step by step guide for label detection using Google Cloud Vision API. Let's follow it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Setup a Google Cloud Account</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A) Go to: </span><span style="font-family: "arial" , "helvetica" , sans-serif;">https://console.cloud.google.com/</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">B) Login with your google credentials<br />C) You will see a dashboard. Create a Project if not already created.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Enable Cloud Vision API</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A) Go to console<br />B) Click on Navigation Menu<br />C) Click on API & Services >> Library<br />D) Search "cloud vision" and you will get the "Cloud Vision API". Enable this API if not already enabled.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Download credentials file</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br />A) Go to console<br />B) Click on Navigation Menu<br />C) Click on API & Services >> Credentials<br />D) Click on Create Credentials dropdown >> Service account key >> New service account<br />E) Enter Service account name<br />F) Select any role. I had selected Project >> Viewer<br />G) Save the file as JSON on your hard drive. Rename it to 'credentials.json'.<br />
<br />
<b>Step 4: Add billing information</b><br />
<b><br /></b>
A) Go to console<br />B) Click on Navigation Menu<br />C) Click on Billing<br />
<br />Now open the Jupyter notebook and try using this API. You can download my Jupyter notebook containing below code from <a href="https://drive.google.com/file/d/1DiK5FofhBcF3qEnjvsGW18xTU0tlb6mz/view?usp=sharing">here</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Import required libraries</b></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;">from googleapiclient.discovery import build<br />from oauth2client.client import GoogleCredentials<br />from base64 import b64encode</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You may get import error "no module name..." if you have not already installed Google API Python client. Use following command to install it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>pip install --upgrade google-api-python-client</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If you also get import error for oauth2client, you must install it using following command:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>pip3 install --upgrade oauth2client </b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Load credentials file</b></span></div>
<div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Load the credentials file (which we created in step 3) and create a service object using it.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="background-color: white;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">CREDENTIAL_FILE = 'credentials.json'<br />credentials = GoogleCredentials.from_stream(CREDENTIAL_FILE)<br />service = build('vision', 'v1', credentials=credentials)</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 7: Load image file (which needs to be tested)</b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will load an image of a cat and encode it so that it becomes compatible with the cloud vision API.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0xyYhThVgfYT45VFYv-TvZ6RzokynD5nWJwFOxOTjovQ5Zun1tSGnWWiumoUYB7s9oIDc901__2kjhtK_YbGK32byHeRt_dL3LVo9M48g5uicJUlmzolfbCDtLImX16menxQYvUgnbsvK/s1600/cat.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="101" data-original-width="135" height="149" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0xyYhThVgfYT45VFYv-TvZ6RzokynD5nWJwFOxOTjovQ5Zun1tSGnWWiumoUYB7s9oIDc901__2kjhtK_YbGK32byHeRt_dL3LVo9M48g5uicJUlmzolfbCDtLImX16menxQYvUgnbsvK/s200/cat.jpg" width="200" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">IMAGE_FILE = 'cat.jpg'</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">with open(IMAGE_FILE, 'rb') as file:<br /> image_data = file.read()<br /> encoded_image_data = b64encode(image_data).decode('UTF-8')</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 8: Create a batch request</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a batch request which we will send to the cloud vision API. In the batch request, we will include the above encoded image and the instruction as <span style="background-color: white; color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><b>LABEL_DETECTION</b>.</span></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">batch_request = [{<br /> 'image':{'content':encoded_image_data},<br /> 'features':[{'type':'<b>LABEL_DETECTION</b>'}],<br />}]</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 9: Create a request</b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">request = service.images().annotate(body={'requests':batch_request})</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 10: Execute the request</b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">response = request.execute()</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This step will throw an error if you have not enabled billing (as mentioned in step 4). So, you must enable the billing in order to use Google Cloud Vision API. The charges are very reasonable. So, don't think too much and provide credit card details. For me, Google charged INR 1 and then refunded it back.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 11: Process the response</b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For error handling, include this code:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">if 'error' in response:<br /> raise RuntimeError(response['error'])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are interested in label annotations here. So, fetch it from the response and display the results.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">labels = response['responses'][0][<b>'labelAnnotations'</b>]</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">for label in labels:<br /> print(label['description'], label['score'])</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Output</b>:</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;">Cat 0.99598557<br />Mammal 0.9890478<br />Vertebrate 0.9851104<br />Small to medium-sized cats 0.978553<br />Felidae 0.96784574<br />European shorthair 0.960582<br />Tabby cat 0.9573447<br />Whiskers 0.9441685<br />Dragon li 0.93990624<br />Carnivore 0.9342105</span></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can test the above code using different images and check the accuracy of the API.</span></div>
</div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-63224769262577974332019-08-30T18:58:00.001-07:002019-08-30T20:36:45.456-07:00Image Recognition using Pre-trained VGG16 model in Keras<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets use a pre-trained VGG16 model to predict an image from ImageNet database. We will load an image, convert that image to numpy array, preprocess that array and let the pre-trained VGG16 model predict the image.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">VGG16 is a CNN model. To know more about CNN, you can visit </span><a href="http://theprofessionalspoint.blogspot.com/2019/05/basic-introduction-of-various-layers-in.html" style="font-family: Arial, Helvetica, sans-serif;">my this post</a><span style="font-family: "arial" , "helvetica" , sans-serif;">. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">We are not fine-tuning the VGG16 model here. We are using it as it is. To fine-tune the existing VGG16 model, you can visit <a href="http://theprofessionalspoint.blogspot.com/2019/07/fine-tune-vgg16-model-for-image.html">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Y<span style="font-family: "arial" , "helvetica" , sans-serif;">ou can download my Jupyter notebook containing following code from <a href="https://drive.google.com/file/d/1nCnsAAodmT9OTpRDLXybm_0zlDDj23Og/view?usp=sharing">here</a>.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://www.blogger.com/"></a><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Import required libraries</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">import numpy as np<br />from keras.applications import vgg16<br />from keras.preprocessing import image</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Load pre-trained weights from VGG16 model for ImageNet dataset</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">model = vgg16.VGG16(weights='imagenet')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Load image to predict</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">img = image<b>.load_img</b>('cat.jpg', target_size=(224, 224))<br />img</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP3SzK9mVn-JwteEOO2M8A4WxfZbjyB7PMa-eWiwzQrd-Tdi4Om05SJuQzObhp-NejhfpCkTDyWv4SYdGXe6i1P_ittyRO8_Zr9W-1snQgmt4_Mz57R2k9P7XlGskDDjDyeVA3S78YLMg4/s1600/tiger_cat.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="101" data-original-width="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgP3SzK9mVn-JwteEOO2M8A4WxfZbjyB7PMa-eWiwzQrd-Tdi4Om05SJuQzObhp-NejhfpCkTDyWv4SYdGXe6i1P_ittyRO8_Zr9W-1snQgmt4_Mz57R2k9P7XlGskDDjDyeVA3S78YLMg4/s1600/tiger_cat.jpg" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please note that w<span style="color: black; display: inline; float: none; font-variant-east-asian: normal; font-variant-numeric: normal;">e need to reshape the image to 224X224 as it is a requirement for VGG16 model. You can download this image from <a href="http://www.image-net.org/search?q=cat">ImageNet official website</a>.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i></i><span style="font-family: "times new roman";"></span><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 4: Convert the image into numpy array</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">arr = image.<b>img_to_array</b>(img)<br />arr.shape</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;">(224, 224, 3)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Expand the array dimension</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">arr = np.<b>expand_dims</b>(arr, axis=0)<br />arr.shape</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">(1, 224, 224, 3)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Preprocess the array</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">arr = vgg16.preprocess_input(arr)<br />arr</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span>
<b style="font-family: Arial, Helvetica, sans-serif;">Step 7: Predict from the model</b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />predictions = model.<b>predict</b>(arr)</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">predictions</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We get an array as an output which is hard to understand. So, lets simplify it and see top 5 predictions made by the VGG16 model.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">vgg16.<b>decode_predictions</b>(predictions, top=5)</span></div>
<div>
<br /></div>
<div>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">[[('n02123045', 'tabby', 0.7138179),<br /> ('n02123159', 'tiger_cat', 0.21695374),<br /> ('n02124075', 'Egyptian_cat', 0.043560617),<br /> ('n04040759', 'radiator', 0.0053847637),<br /> ('n04553703', 'washbasin', 0.0024860944)]]</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="background-color: yellow;"></span><br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So, as per VGG16 model prediction, the given image may be a tabby (71%) or a tiger cat (21%). You can try the same with different images from ImageNet database and check your results.</span></div>
</div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-17714905759650424562019-08-24T00:43:00.002-07:002019-09-17T07:44:01.582-07:00eBook - Deep Learning Objective Type Questions and Answers<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">This book contains 205 objective type questions and answers covering various basic concepts of deep learning. It contains 19 chapters. Each chapter contains a short description of a concept and objective type questions from that concept. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can download this book from <a href="http://onlinemlquiz.com/deep_learning_objective_questions_ebook.php">here</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5w0aayBWRvqLxVx4ZVgk2RuP-oIM9vjgX4BP8JlIGTvJ3O2MiC2kW8gSJyoszHUBvXzNQHzx5LlE8pU1KYsNPGmytzooDNGFPpni8NX2l2mzcZV7qGPD_J1eaBK-eUFVo_vQ7yV31MdOl/s1600/deep_learning_ebook_cover_page.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1067" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5w0aayBWRvqLxVx4ZVgk2RuP-oIM9vjgX4BP8JlIGTvJ3O2MiC2kW8gSJyoszHUBvXzNQHzx5LlE8pU1KYsNPGmytzooDNGFPpni8NX2l2mzcZV7qGPD_J1eaBK-eUFVo_vQ7yV31MdOl/s320/deep_learning_ebook_cover_page.png" width="213" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please download this book, study and distribute among your friends and colleagues. I would be more than happy if this book can increase your deep learning knowledge to some extent.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Assumption</b>: This should not be your first book on deep learning as I have not covered deep learning concepts in detail, just given a short description to revise your concepts. So, I assume, you have some basic understanding of deep learning concepts before reading this book.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I am <b>continuously upgrading</b> this book and adding more and more objective type deep learning questions in this book as well as in deep learning quiz. So, stay tuned! You can visit this page once in a week and download the updated copy of this book.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Your opinion about this book matters a lot to me. Please post your comments and suggestions regarding this eBook on this blog post.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b style="font-size: x-large;">Table of Contents</b></span><br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheTBtNDfAPutOpMCHXU9FRb9oJ25dP7LthSIaZd-hIgCPDK2JOdRq2tOr0nTrlWAyrMxyjgoS4HzcuPUq74uqvgKUvt0tCGCHemGC4pVyP4m0geDHTUb4qkGCgL8Of-P7W9Ftt9Vnz7Ph9/s1600/deep_learning_ebook_table_of_contents.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="814" data-original-width="944" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheTBtNDfAPutOpMCHXU9FRb9oJ25dP7LthSIaZd-hIgCPDK2JOdRq2tOr0nTrlWAyrMxyjgoS4HzcuPUq74uqvgKUvt0tCGCHemGC4pVyP4m0geDHTUb4qkGCgL8Of-P7W9Ftt9Vnz7Ph9/s320/deep_learning_ebook_table_of_contents.PNG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Deep Learning Concepts</b></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This book contains objective questions on following Deep Learning concepts:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. Perceptrons:</b> Working of a Perceptron, multi-layer Perceptron, advantages and limitations of Perceptrons, implementing logic gates like AND, OR and XOR with Perceptrons etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Neural Networks:</b> Layers in a neural network, types of neural networks, deep and shallow neural networks, forward and backward propagation in a neural network etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Weights and Bias:</b> Importance of weights and biases, things to keep in mind while initializing weights and biases, Xavier Weight Initialization technique etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4. Activation Functions:</b> Importance of activation functions, Squashing functions, Step (Threshold), Logistic (Sigmoid), Hyperbolic Tangent (Tanh), ReLU (Rectified Linear Unit), Dying and Leaky ReLU, Softmax etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>5. Batches:</b> Epochs, Batches and Iterations, Batch Normalization etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>6. Gradient Descent:</b> Batch, Stochastic and Mini Batch Gradient Descent, SGD variants like Momentum, Nesterov Momentum, AdaGrad, AdaDelta, RMSprop and Adam, Local and Global Minima, Vanishing and Exploding Gradients, Learning Rate etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>7. Loss Functions:</b> categorical_crossentropy, sparse_categorical_crossentropy etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>8. CNN:</b> Convolutional Neural Network, Filters (Kernels), Stride, Padding, Zero Padding and Valid Padding, Pooling, Max Pooling, Min Pooling, Average Pooling and Sum Pooling, Hyperparameters in CNN, Capsule Neural Network (CapsNets), ConvNets vs CapsNets, Computer vision etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>9. RNN:</b> Recurrent Neural Network, Feedback loop, Types of RNN like One to One, One to Many, Many to One and Many to Many, Bidirectional RNN, Advantages and disadvantages of RNN, Applications of RNN, Differences between CNN and RNN etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>10. LSTM:</b> Long Short Term Memory, Gated cells like Forget gate, Input gate and Output gate, Applications of LSTM etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>11. Regularization</b>: Overfitting and underfitting in a neural network, L1 and L2 Regularization, Dropout, Data Augmentation, Early Stopping etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>12. Fine-tuning</b>: Transfer Learning, Fine-tuning a model, Steps to fine-tune a model, Advantages of fine-tuning etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>13. Autoencoders:</b> Components of an autoencoder like encoder, decoder and bottleneck, Latent space representation and reconstruction loss, Types of Autoencoders like Undercomplete autoencoder, Sparse autoencoder, Denoising autoencoder, Convolutional autoencoder, Contractive autoencoders and Deep autoencoders, Hyperparameters in an autoencoder, Applications of an Autoencoder, Autoencoders vs PCA, RBM (Restricted Boltzman Machine) etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>14. NLP (Natural Language Processing):</b> Tokenization, Stemming, Lemmatization and Vectorization (Count vectorization, N-grams vectorization, Term Frequency - Inverse Document Frequency (TF-IDF)), Document-term matrix, NLTK( Natural Language Toolkit) etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>15. Frameworks</b>: TesnorFlow, Keras, PyTorch, Theano, CNTK, Caffe, MXNet, DL4J etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>A note to readers</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This book is just a short summary of my online contents on:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> <a href="http://theprofessionalspoint.blogspot.com/">The Professionals Point</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2.</b> <a href="http://onlinemlquiz.com/">Online ML Quiz</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Contents of this book are available on my this blog (The Professionals Point) and objective type questions are available in the form of quiz on my website (Online ML Quiz). </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: x-small;"><b>Disclaimer</b>: Contents of this book are the sole property of www.onlinemlquiz.com. Questions should not be reproduced in any form without prior permission and attribution.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com1tag:blogger.com,1999:blog-4423496461696137369.post-42560533714014495162019-08-10T22:07:00.005-07:002019-08-10T22:09:44.176-07:00Solving a regression problem using a Sequential Neural Network Model in Keras<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets solve a regression problem using neural networks. We will build a sequential model in Keras to predict house prices based on some parameters. We will use </span><b style="font-family: arial, helvetica, sans-serif;">KerasRegressor </b><span style="font-family: "arial" , "helvetica" , sans-serif;">to build a regression model</span><span style="font-family: "arial" , "helvetica" , sans-serif;">.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can download <b>housing_data</b></span><b style="font-family: arial, helvetica, sans-serif;">.csv</b><span style="font-family: "arial" , "helvetica" , sans-serif;"> from <a href="https://drive.google.com/file/d/10roTTpr-A5im5yBYOo1Zw1PbpA7rUmjU/view?usp=sharing">here</a></span><span style="font-family: "arial" , "helvetica" , sans-serif;">. You can also download my <a href="https://drive.google.com/file/d/1qPEIsyzZlgIttQwfNgSIn4c967dxrVbG/view?usp=sharing">Jupyter notebook</a> </span><span style="font-family: "arial" , "helvetica" , sans-serif;">containing below code of Neural Network Regression implementation.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Import required libraries like pandas, numpy, sklearn, keras and matplotlib</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import numpy as np</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">import pandas as pd</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.preprocessing import MinMaxScaler</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.model_selection import train_test_split</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.metrics import mean_absolute_error, mean_squared_error</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.models import Sequential</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.layers import Dense</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.wrappers.scikit_learn import <b>KerasRegressor</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import matplotlib.pyplot as plt</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">%matplotlib inline</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Load and examine the dataset</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">dataset = pd.read_csv('housing_data.csv')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">dataset.head()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">dataset.shape</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">dataset.describe(include='all')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Please note that "describe()" is used to display the statistical values of the data like mean and standard deviation.</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Mention X and Y axis</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">X=dataset.iloc[:,0:13]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y=dataset.iloc[:,13].values</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>X contains the list of attributes</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Y contains the list of labels</i></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 4: Split the dataset into training and testing dataset</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, random_state=0) </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Scale the features</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">scaler = MinMaxScaler()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train = scaler.fit_transform(X_train)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_test = scaler.transform(X_test)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y = y.reshape(-1,1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y = scaler.fit_transform(y)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>This step is must for neural networks. Feature scaling is very important for neural networks to perform better and predict accurate results. We should scale both X and y data.</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Build a neural network</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">def build_regression_model():</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model = Sequential()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model.add(Dense(50, input_dim=13, activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model.add(Dense(50, activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model.add(Dense(50, activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model.add(Dense(1, activation='linear'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> model.compile(optimizer='adam', loss='mean_squared_error')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><span style="font-family: "arial" , "helvetica" , sans-serif;">return model</span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are creating a sequential model with fully connected layers. We are using four layers (one input layer, one output layer and two hidden layers). Input layer and hidden layers are using "relu" activation function while output layer is using "linear" activation function. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Input layer and hidden layers contain 50 neurons and output layer contains only one neuron as we need to output only one value (predicted house price). You can change the number of neurons in the input and hidden layers as per your data and model performance. Number of hidden layers and number of neurons in each layer are the hyperparameters which you need to tune as the the performance of the model. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are using "adam" optimizer and mean square error as a loss function.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can also use dropout in hidden layers for regularization. But, for this example, I am skipping this step for simplification.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 7: Train the neural network</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">regressor = KerasRegressor(build_fn=build_regression_model, batch_size=32, epochs=150) </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">training_history = regressor.fit(X_train,y_train)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are using 150 epochs with batch size of 32. Number of epochs and batch size are also the hyperparameters which need to be tuned. </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 8: Print a loss plot</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.plot(training_history.history['loss'])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.show()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpl6iiweygzvNoXJ6-H1e17Kq-LPq9xQatE_GITobXRnxCayVwA2tyqx2tyBV6KrykDr2wyIV-mAr_icLPJatsesV9SKaTg6YFGB72sBgHhMfcI4XqxxXlkxL5FTcjY7IsFHkntg4On8FF/s1600/loss_plot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="370" data-original-width="561" height="263" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpl6iiweygzvNoXJ6-H1e17Kq-LPq9xQatE_GITobXRnxCayVwA2tyqx2tyBV6KrykDr2wyIV-mAr_icLPJatsesV9SKaTg6YFGB72sBgHhMfcI4XqxxXlkxL5FTcjY7IsFHkntg4On8FF/s400/loss_plot.png" width="400" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This plot shows that after around 140 epochs, the loss does not vary so much. That is why, I have taken number of epochs as 150 in step 7 while training the neural network.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 9: Predict from the neural network</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_pred= regressor.predict(X_test)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_pred</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>The y_pred is a numpy array that contains all the predicted values for the input values in the X_test.</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Lets see the difference between the actual and predicted values.</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">df=pd.DataFrame({'Actual':y_test, 'Predicted':y_pred}) </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">df </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 10: Check the accuracy</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">meanAbsoluteError = <b>mean_absolute_error</b>(y_test, y_pred)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">meanSquaredError = <b>mean_squared_error</b>(y_test, y_pred)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">rootMeanSquaredError = np.sqrt(meanSquaredError)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print('Mean Absolute Error:', meanAbsoluteError) </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print('Mean Squared Error:', meanSquaredError) </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print('Root Mean Squared Error:', rootMeanSquaredError)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Output:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-size: 14px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.94px; overflow-wrap: break-word; white-space: pre-wrap; word-break: break-all;">Mean Absolute Error: 2.9524098807690193
Mean Squared Error: 19.836363961675836
Root Mean Squared Error: 4.453803314210882</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We have got the root mean square error as 4.45. We can further decrease this error using cross validation and tuning our hyperparameters. I am leaving it for you to practice.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 11: Visualize the results using scatter plot </b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.scatter(range(len(y_test)), y_test, c='g')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.scatter(range(len(y_test)), y_pred, c='b')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.xlabel('Test data')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.ylabel('Predicted data')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.show()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg45ue155I92vLS8bm5-VKWezxnevliX80ZtvF9L4Drs35FLDsAxzBcstJcWxFtR6gv4obRCrlolG7Tn8iG9smYs74xD0sNdxobuOPrShv0ATj0btDlon9JyIn-M9AQ55g1GlSJhTNretW4/s1600/keras_scatter_plot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="392" data-original-width="572" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg45ue155I92vLS8bm5-VKWezxnevliX80ZtvF9L4Drs35FLDsAxzBcstJcWxFtR6gv4obRCrlolG7Tn8iG9smYs74xD0sNdxobuOPrShv0ATj0btDlon9JyIn-M9AQ55g1GlSJhTNretW4/s400/keras_scatter_plot.png" width="400" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are displaying test labels and predicted values in different colors (green and blue). From the scatter plot, we can visualize that our neural network has done a great job.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 12: Visualize results using regression plot</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">To further visualize the predicted results, we can draw a regression plot.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">fig, ax = plt.subplots()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">ax.scatter(y_test, y_pred)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">ax.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'k--', lw=4)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">ax.set_xlabel('Test data')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">ax.set_ylabel('Predicted data')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.show()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAui1R7vNGS8zzNFu1gvxRnFaRbu-q-eaFj1DYL9pwhthQNgCPFUk2D42xAdUO9DrBmR6dRtPeX_-bvRVo6ozXhP85GYBLZzzrh9v-QVAFUq8vMUhqLyINbsHQ4Vfsetc4elRC8WKVTAGC/s1600/keras_regression_plot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="396" data-original-width="564" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAui1R7vNGS8zzNFu1gvxRnFaRbu-q-eaFj1DYL9pwhthQNgCPFUk2D42xAdUO9DrBmR6dRtPeX_-bvRVo6ozXhP85GYBLZzzrh9v-QVAFUq8vMUhqLyINbsHQ4Vfsetc4elRC8WKVTAGC/s400/keras_regression_plot.png" width="400" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I hope, I was able to demonstrate this regression problem to a large extent. If you have further any doubt, please post a comment.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-87783520778916954512019-07-06T22:05:00.003-07:002019-07-07T22:16:44.982-07:00Fine-tune VGG16 model for image classification in Keras<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Keras framework provides us a lot of pre-trained general purpose deep learning models which we can fine-tune as per our requirements. We don't need to build a complex model from scratch. In <a href="http://theprofessionalspoint.blogspot.com/2019/07/building-cnn-model-in-keras-using-mnist.html">my last article</a>, we built a CNN model from scratch for image classification. Instead of that, we can just fine-tune an existing, well-trained, well-proven, widely accepted CNN model which will save our a lot of effort, time and money.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">VGG16 is a proven proficient algorithm for image classification (1000 classes of images). Keras framework already contain this model. We will import this model and fine-tune it to classify the images of dogs and cats (only 2 classes instead of 1000 classes).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can download my Jupyter notebook containing below code from <a href="https://drive.google.com/file/d/1eXUfUTaEi4P-loQ50ih-mKCdnygjEECr/view?usp=sharing">here</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Import the required libraries</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import numpy as np</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">import matplotlib.pyplot as plt</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">%matplotlib inline</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import keras</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.preprocessing.image import ImageDataGenerator</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.models import <b>Sequential</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.layers import <b>Dense</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.optimizers import <b>Adam</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.metrics import confusion_matrix, accuracy_score, classification_report</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Create directory structure to contain images</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a directory structure which will contain the images of dogs and cats.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvWcOHXCtJUHX5K8kE6xfV9IHsFMj5y-WMWLsoARE2JgWwoTyNRJ7SgIAVbnwWT0h13rAWC40-h7y0dqjziTCjSXurSDOaMmrH3A-CUWSQ8WdpqT56Kt81C4AKyp9OT60Frq4QtZ6BL3SO/s1600/VGG16_directory_structure.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="156" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvWcOHXCtJUHX5K8kE6xfV9IHsFMj5y-WMWLsoARE2JgWwoTyNRJ7SgIAVbnwWT0h13rAWC40-h7y0dqjziTCjSXurSDOaMmrH3A-CUWSQ8WdpqT56Kt81C4AKyp9OT60Frq4QtZ6BL3SO/s320/VGG16_directory_structure.PNG" width="138" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have created a directory "cats_and_dogs". Under this directory, I have created 3 other directories "test", "train" and "valid". All these 3 directories contain "cat" and "dog" directories. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1</b>. "cat" and "dog" directories under "test" directory contain 5 images of cats and dogs respectively. Total 10 images for testing.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2</b>. "cat" and "dog" directories under "train" directory contain 20 images of cats and dogs respectively. Total 40 images for training.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3</b>. "cat" and "dog" directories under "valid" directory contain 8 images of cats and dogs respectively. Total 16 images for validation.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Data Preparation</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_path = 'C:/cats_and_dogs/train'</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">valid_path = 'C:/cats_and_dogs/valid'</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_path = 'C:/cats_and_dogs/test'</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_batches = <b>ImageDataGenerator</b>().<b>flow_from_directory</b>(train_path, target_size=(224,224), classes=['dog','cat'], batch_size=10)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">valid_batches = ImageDataGenerator().flow_from_directory(valid_path, target_size=(224,224), classes=['dog','cat'], batch_size=4)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=['dog','cat'], batch_size=10)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">Output:</span><br />
<span style="background-color: yellow; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.94px; overflow-wrap: break-word; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Found 40 images belonging to 2 classes.
Found 16 images belonging to 2 classes.
Found 10 images belonging to 2 classes.</span></span><br />
<span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.94px; overflow-wrap: break-word; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">In the above code, we are generating the images of 224x224 pixels and categorizing these images into cat and dog classes. It is clear from the output that we have 40 images for training, 16 images for validation and 10 images for testing as mentioned in step 2. </span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><b>Step 4: Print the images</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Lets output some of the images which we have prepared in step 3. Following is the standard code to print the images (copied from Keras documentation)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;">def plots(ims, figsize=(12,6), rows=1, interp=False, titles=None):</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> if type(ims[0]) is np.ndarray:</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> ims = np.array(ims).astype(np.uint8)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> if (ims.shape[-1] != 3):</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> ims = ims.transpose((0,2,3,1))</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> f = plt.figure(figsize=figsize)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> cols = len(ims)//rows if len(ims) % 2 == 0 else len(ims)//rows + 1</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> for i in range(len(ims)):</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> sp = f.add_subplot(rows, cols, i+1)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> sp.axis('Off')</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> if titles is not None:</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #f3f3f3; white-space: pre-wrap;"> sp.set_title(titles[i], fontsize=16)</span></span><br />
<span style="background-color: #f3f3f3; font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"> plt.imshow(ims[i], interpolation=None if interp else 'none')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Now, lets print the first batch of training images:</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">imgs, labels = <b>next</b>(train_batches)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">plots(imgs, titles=labels)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Output:</span></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4Fyf9TPuwTTCMTNbfdDIqwO_nC9Lp1gyogHyeblJ7Sme63J2UVdZoMyfphZIyeKwu_dCd5yG14JPdHEPNRqCfQ17cOtg13XHgv9MltgMy2pKXAxFAJLVEoDTwWLSuKFy7jE3M94PRgU7Q/s1600/Keras_cat_and_dog_images.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="130" data-original-width="1029" height="40" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4Fyf9TPuwTTCMTNbfdDIqwO_nC9Lp1gyogHyeblJ7Sme63J2UVdZoMyfphZIyeKwu_dCd5yG14JPdHEPNRqCfQ17cOtg13XHgv9MltgMy2pKXAxFAJLVEoDTwWLSuKFy7jE3M94PRgU7Q/s320/Keras_cat_and_dog_images.PNG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">We can see the scaled images of 10 cats and dogs. If you run again the above code, it will fetch next 10 images from training dataset as we are using batch size of 10 for training images.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>Step 5: Load and analyze VGG16 model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">vgg16_model = keras.applications.vgg16.<b>VGG16</b>()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">vgg16_model.<b>summary</b>()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>type</b>(vgg16_model)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">In the above code, first line will load the VGG16 model. It may take some time. By executing second line, we can see summary of the existing model. It has a lot of convolutional, pooling and dense layers. Executing third line, we can see this model is of type "Model". In next step, we will create a model of type "Sequential".</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>Step 6: Fine-tune VGG16 model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">Following are the steps involved in fine-tuning a model:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>1.</b> Copy all the hidden layers in a new model</span><br />
<b style="font-family: arial, helvetica, sans-serif; white-space: pre-wrap;">2.</b><span style="font-family: arial, helvetica, sans-serif; white-space: pre-wrap;"> Remove output layer</span><span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>3</b>. Freeze the hidden layers</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>4.</b> Add custom output layer</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">For more details on fine-tuning a model, please visit <a href="http://theprofessionalspoint.blogspot.com/2019/06/transfer-learning-and-fine-tuning-model.html">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">Lets perform all the above steps.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">model = Sequential()
for layer in vgg16_model.layers[:-1]:
model.add(layer)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">In the above code, we have created a new sequential model and copied all the layers of VGG16 model except the last layer which is an output layer. We have done this because we want our custom output layer which will have only two nodes as our image classification problem has only two classes (cats and dogs).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">Now, if we execute following statement, we will get replica of existing VGG16 model, except output layer.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">model.summary()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">Now, lets <b>freeze </b>the hidden layers as we don't want to change any weight and bias associated with these layers. We want to use these layers as it is as all these layers are already well trained on image classification problem.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">for layer in model.layers:
layer.<b>trainable </b>= False</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Now, add a custom output layer with only two nodes and softmax as activation function.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">model.add(Dense(2, activation='softmax'))</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">model.summary()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">Now, our new fine-tuned model is ready. Lets train it with new data and then predict from it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><b>Step 7: Compile the model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">model.<b>compile</b>(Adam(lr=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Using Adam as an optimizer and categorical cross entropy as loss function.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><b>Step 8: Train the model</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">model.<b>fit_generator</b>(train_batches, steps_per_epoch=4, validation_data=valid_batches, validation_steps=4, epochs=5, verbose=2)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Executing this step will take some time as we are using 5 epochs.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><b>Step 9: Predict from the model</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Lets print first batch of the test images.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">test_imgs, test_labels = <b>next</b>(test_batches)
plots(test_imgs, titles=test_labels)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">From the output, we can see that it shows the final results in form of [0. 1.], </span></span><span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;">[1. 0.] etc. Lets format this output so that we can get it in form of 0, 1 etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; white-space: pre-wrap;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">test_labels = test_labels[:,0]
test_labels</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Now, finally make prediction.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">predictions = model.<b>predict_generator</b>(test_batches, steps=1, verbose=0)</span></span><br />
<span style="font-family: arial, helvetica, sans-serif; white-space: pre-wrap;">predictions</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">It shows the predictions in form of probabilities. Lets round it off.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">rounded_predictions = np.round(predictions[:,0])</span></span><br />
<span style="font-family: arial, helvetica, sans-serif; white-space: pre-wrap;">rounded_predictions</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><b>Step 10: Check the accuracy</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">confusionMatrix = <b>confusion_matrix</b>(test_labels, rounded_predictions)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">accuracyScore = <b>accuracy_score</b>(test_labels, rounded_predictions)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">classificationReport = <b>classification_report</b>(test_labels, rounded_predictions)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">print(confusionMatrix)</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">print(accuracyScore * 100)</span></span><br />
<span style="font-family: arial, helvetica, sans-serif; white-space: pre-wrap;">print(classificationReport)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre-wrap;">Please note that we won't get desired accuracy with this small dataset. We need thousands of image to train our model to get desired accuracy. We can use data augmentation to increase the data. You can download thousands of images of cats and dogs from <a href="https://www.kaggle.com/c/dogs-vs-cats/data">Kaggle </a>to train this model.</span></span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-61870410594705553042019-07-06T03:39:00.001-07:002019-08-30T19:19:35.028-07:00Building a CNN model in Keras using MNIST dataset<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will implement CNN in Keras using MNIST dataset. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">To know more about CNN, you can visit </span><a href="http://theprofessionalspoint.blogspot.com/2019/05/basic-introduction-of-various-layers-in.html" style="font-family: Arial, Helvetica, sans-serif;">my this post</a><span style="font-family: "arial" , "helvetica" , sans-serif;">. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">We can download the </span><span style="font-family: "arial" , "helvetica" , sans-serif;">MNIST </span><span style="font-family: "arial" , "helvetica" , sans-serif;">dataset through Keras. The MNIST dataset contains images of handwritten digits from 0 to 9. It is divided into 60,000 training images and 10,000 testing images.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I would recommend you to build a simple neural network before jumping to CNN. You can visit <a href="http://theprofessionalspoint.blogspot.com/2019/07/building-simple-neural-network-in-keras.html">my this post</a> to build a simple neural network with Keras. You can download my Jupyter notebook containing following code of CNN from <a href="https://drive.google.com/file/d/1NwOU2UiJ8-Awe5T4QTxIU-_L88-E1KIS/view?usp=sharing">here</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://www.blogger.com/"></a><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Import required libraries</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.datasets import mnist</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.utils import to_categorical</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.models import <b>Sequential</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.layers import <b>Conv2D, MaxPooling2D, Flatten, Dense, Dropout</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.optimizers import <b>Adam</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import matplotlib.pyplot as plt<br />%matplotlib inline<b></b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Load MNIST dataset from Keras</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">(X_train, y_train), (X_test, y_test) = mnist.load_data()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Above line will download the MNIST dataset. Now, lets print the shape of the data.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train.shape, X_test.shape, y_train.shape, y_test.shape</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Output: </span><span style="font-size: 14px; white-space: pre-wrap;"><span style="font-family: "arial" , "helvetica" , sans-serif;">((60000, 28, 28), (10000, 28, 28), (60000,), (10000,))</span></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><b></b><i></i><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">It is clear from the above output that each image in the MNIST dataset has a size of 28 X 28 pixels which means that the shape of x_train is (60000, 28, 28) where 60,000 is the number of samples. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can visualize the images using matplotlib library. Lets see first image.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">plt.<b>imshow</b>(X_train[0], cmap='gray')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHooSJjBFePr4w1zGLpkLwV5JOnPDvi30maCex_r-yUJMIoQMaofxTX_5aOQ7ngqaBCztC-E9kd4qeNFMOYi4Ji6LPAok_OWpXpg9nuCg-_lhlz_R7oUSe1VLiUiIvKYZLZfyg6HYeYs55/s1600/mnist_dataset_image.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="362" data-original-width="365" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHooSJjBFePr4w1zGLpkLwV5JOnPDvi30maCex_r-yUJMIoQMaofxTX_5aOQ7ngqaBCztC-E9kd4qeNFMOYi4Ji6LPAok_OWpXpg9nuCg-_lhlz_R7oUSe1VLiUiIvKYZLZfyg6HYeYs55/s320/mnist_dataset_image.PNG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Reshape the dataset</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We have to reshape the X_train from 3 dimensions to 4 dimensions as it is a requirement to process through Keras API. We reshape X_train and X_test because our CNN accepts only a four-dimensional vector. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train = X_train.reshape(X_train.shape[0], 28, 28, 1)<br />X_test = X_test.reshape(X_test.shape[0], 28, 28, 1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">The value of "X_train.shape[0]" is 60,000. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">The value 60,000 represents the number of images in the training data, 28 represents the image size and 1 represents the number of channels. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The number of channels is set to 1 if it is in grayscale image and if the image is in RGB format, the number of channels is set to 3. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">The last number is 1, which signifies that the images are greascale.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now, lets again print the shape of the data.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train.shape, X_test.shape, y_train.shape, y_test.shape</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.94px; overflow-wrap: break-word; white-space: pre-wrap; word-break: break-all;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Output: ((60000, 28, 28, 1), (10000, 28, 28, 1), (60000,), (10000,))</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i></i><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 4: Convert the image pixels in the range between 0 and 1</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets us print first image:<br />X_train[0]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You will notice that it contains the values ranging from 0 to 255. We need to scale this data between 0 and 1 for accurate results. Ideally, all the inputs to the neural network should be between 0 and 1. So, lets do it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">First convert X_train and X_test to float data type.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train = X_train.astype('float32')<br />X_test = X_test.astype('float32')</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now, divide all the values by 255.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">X_train /= 255.0<br />X_test /= 255.0</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Again print the first image:<br />X_train[0]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You will notice that all the pixels are ranging from 0 to 1 only which is a perfect data for our neural network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Convert labels into categorical variables (one-hot encoding)</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Our labels are ranging from 0 to 9. So, we need to one-hot encode these labels so that these turn into 0 and 1.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_train, y_test</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_train[0]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">5</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_train = to_categorical(y_train)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_test = to_categorical(y_test)</span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_train, y_test</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">y_train[0]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">array([0., 0., 0., 0., 0., <b>1</b>., 0., 0., 0., 0.], dtype=float32)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please note that how the first image "5" is transformed into one hot encoded format. You can try the same with different images and note their one hot encoded representation.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Create a CNN model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model = Sequential()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Conv2D(32, kernel_size=(5,5), input_shape=(28,28,1), padding='same', activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(MaxPooling2D(pool_size=(2, 2)))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Conv2D(64, kernel_size=(5,5), padding='same', activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(MaxPooling2D(pool_size=(2, 2)))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Flatten())</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dense(1024, activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dropout(0.2))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dense(10, activation='softmax'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Explanation of above CNN model:</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> We have created a sequential model which is an in-built model in </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Keras. W</span><span style="font-family: "arial" , "helvetica" , sans-serif;">e just have to add the layers in this model as per our requirements. In this case, we have added 2 convolutional layers, 2 pooling layers, 1 flatten layer, 2 dense layers and 1 dropout layer. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2.</b> We have used 32 filters with size 5X5 each in first convolutional layer and then 64 filters in the second convolutional layer.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3</b>. We are using zero padding in each convolutional layer.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />4</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> After each convolutional layer, we are adding pooling layer with pool size of 2X2.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />5</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> We are using ReLU activation function in all hidden layers and softmax in output layer. To know more about activation functions, please visit my <a href="http://theprofessionalspoint.blogspot.com/2019/05/activation-squashing-functions-in-deep.html">this </a>and <a href="http://theprofessionalspoint.blogspot.com/2019/06/difference-between-sigmoid-and-softmax.html">this </a>post.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />6</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> </span><span style="font-family: "arial" , "helvetica" , sans-serif;">We can also specify stride attribute for convolutional and pooling layers. By default it is (1,1).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />7</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> We have flatten layer just before dense layer. Flatten layer converts the 2D matrix data to a 1D vector before building the fully connected layers.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />8</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> After that, we use a fully connected layer with 1024 neurons.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />9</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> Then we use a regularization layer called Dropout. It is configured to randomly exclude 20% of neurons in the layer in order to reduce overfitting. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Dropout randomly switches off some neurons in the network which forces the data to find new paths. Therefore, this reduces overfitting. To know more about dropout, please visit <a href="http://theprofessionalspoint.blogspot.com/2019/06/what-is-dropout-how-does-it-prevent.html">this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br />10</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>.</b> We add a dense layers at the end which is used for class prediction (0–9). That is why it has 10 neurons. It is also called output layer. This layer uses softmax activation function instead of ReLU.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<b style="font-family: Arial, Helvetica, sans-serif;">Step 7: Model Summary</b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.<b>summary</b>()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;">_________________________________________________________________<br />Layer (type) Output Shape Param # <br />=================================================================<br />conv2d_1 (Conv2D) (None, 28, 28, 32) 832 <br />_________________________________________________________________<br />max_pooling2d_1 (MaxPooling2 (None, 14, 14, 32) 0 <br />_________________________________________________________________<br />conv2d_2 (Conv2D) (None, 14, 14, 64) 51264 <br />_________________________________________________________________<br />max_pooling2d_2 (MaxPooling2 (None, 7, 7, 64) 0 <br />_________________________________________________________________<br />flatten_1 (Flatten) (None, 3136) 0 <br />_________________________________________________________________<br />dense_1 (Dense) (None, 1024) 3212288 <br />_________________________________________________________________<br />dropout_1 (Dropout) (None, 1024) 0 <br />_________________________________________________________________<br />dense_2 (Dense) (None, 10) 10250 <br />=================================================================<br />Total params: 3,274,634<br />Trainable params: 3,274,634<br />Non-trainable params: 0<br />_________________________________________________________________</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 8: Compile the model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.<b>compile</b>(Adam(lr=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are using Adam optimizer with learning rate of 0.0001 and loss function as categorical cross entropy. You can skip the learning rate parameter to the optimizer and let Keras itself decide the optimal value of the learning rate.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 9: Train the model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">history_cnn = model.<b>fit</b>(x_train, y_train, </span><span style="font-family: "arial" , "helvetica" , sans-serif;">validation_data=(x_test, y_test), </span><span style="font-family: "arial" , "helvetica" , sans-serif;">batch_size=128, epochs=5, verbose=2)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are using batch size of 128 and 5 epochs.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-family: "arial" , "helvetica" , sans-serif;">Train on 60000 samples, validate on 10000 samples<br />Epoch 1/5<br /> - 252s - loss: 0.4103 - acc: 0.8930 - val_loss: 0.1250 - val_acc: 0.9622<br />Epoch 2/5<br /> - 253s - loss: 0.1059 - acc: 0.9691 - val_loss: 0.0725 - val_acc: 0.9778<br />Epoch 3/5<br /> - 276s - loss: 0.0704 - acc: 0.9785 - val_loss: 0.0507 - val_acc: 0.9836<br />Epoch 4/5<br /> - 310s - loss: 0.0542 - acc: 0.9834 - val_loss: 0.0403 - val_acc: 0.9870<br />Epoch 5/5<br /> - 371s - loss: 0.0452 - acc: 0.9864 - val_loss: 0.0352 - val_acc: 0.9887</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: yellow;"></span><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 10: Print plots</b></span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Accuracy Plot</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVQvirIM_3o1SHxbVOQxtbheS7SDqU2DORjVtFmKtPD6oRcq4o__CluTtX2bFPRJoW30eLCVSbTjbS1FXvwaCe2ued5j1G31Q7w1ZPf6RPviUP915FcLrIdTGN0nSO6tBB7X3nQ1jQnqxI/s1600/CNN_Accuracy_Plot.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="372" data-original-width="556" height="267" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVQvirIM_3o1SHxbVOQxtbheS7SDqU2DORjVtFmKtPD6oRcq4o__CluTtX2bFPRJoW30eLCVSbTjbS1FXvwaCe2ued5j1G31Q7w1ZPf6RPviUP915FcLrIdTGN0nSO6tBB7X3nQ1jQnqxI/s400/CNN_Accuracy_Plot.PNG" width="400" /></a></div>
<div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Validation Accuracy Plot</b></span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrgeQvJJv7R_9ZHUFxnyBAUfokxEmx87exnZ732jsgfB7vByy7SjQGCy1aWv3xDalLUaxhTPB4Q8Lf3bS1rQnaDReAYjht1tp8_jca3hLh8NHAOJWM5dZ6PmzVjxlPKTsZm57ZDW0xoGrS/s1600/CNN_Validation_Accuracy_Plot.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="372" data-original-width="573" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrgeQvJJv7R_9ZHUFxnyBAUfokxEmx87exnZ732jsgfB7vByy7SjQGCy1aWv3xDalLUaxhTPB4Q8Lf3bS1rQnaDReAYjht1tp8_jca3hLh8NHAOJWM5dZ6PmzVjxlPKTsZm57ZDW0xoGrS/s400/CNN_Validation_Accuracy_Plot.PNG" width="400" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Loss Plot</b></span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdjp6ACqSYUhwGIblb917hKpOTHD7twV3rycKhaS85g56DYzG3SdPrEBTBAifR4RoDhb56GQJwU5M0GUTxHHRmloUkymBrDoB-HPrIpBgfaJ8nqlyR_M3UyfoDZXCPRNEGBf5LO3N3-3sR/s1600/CNN_Loss_Plot.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="368" data-original-width="562" height="261" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdjp6ACqSYUhwGIblb917hKpOTHD7twV3rycKhaS85g56DYzG3SdPrEBTBAifR4RoDhb56GQJwU5M0GUTxHHRmloUkymBrDoB-HPrIpBgfaJ8nqlyR_M3UyfoDZXCPRNEGBf5LO3N3-3sR/s400/CNN_Loss_Plot.PNG" width="400" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 11: Evaluate the model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">score = model.<b>evaluate</b>(x_test, y_test, verbose=0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print(score)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print('Loss:', score[0])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print('Accuracy:', score[1])</span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: yellow; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.94px; overflow-wrap: break-word; white-space: pre-wrap; word-break: break-all;"><span style="background-color: transparent; font-family: "arial" , "helvetica" , sans-serif;">[0.03515956207765266, 0.9887]<br />Loss: 0.03515956207765266<br />Accuracy: 0.9887</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i></i><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I got the accuracy score of about 98.87%. You can play around with different hyperparameters like </span><span style="font-family: "arial" , "helvetica" , sans-serif;">learning rate, </span><span style="font-family: "arial" , "helvetica" , sans-serif;">batch size, number of epochs, adding more convolutional and pooling layers, changing the number and size of filters, changing the size of strides etc.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-64592934023345980222019-07-05T00:00:00.000-07:002019-07-07T22:18:25.088-07:00All about Keras Framework in Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Keras is a widely used framework to implement neural networks in deep learning. Keras is very easy to use and understand and has a large community support. Below are the points which illustrate some strengths and limitations of Keras framework:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. High Level Framework</b>: Keras is an open source and high level neural network framework, written in Python.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Supports Multiple Backends</b>: Keras uses TensorFlow as backend by default but you can also configure it to use Theano or CNTK as backend.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Cross Platform and Easy Model Deployment</b>: Keras can run on all major operating systems. Keras supports a lot of devices and platforms, so we can deploy Keras on any device like iOS with CoreML, Android with Tensorflow Android, </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Web browser with .js support, Cloud engine, Raspberry Pi etc.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4. Multiple CPU and GPU compatible:</b> Keras has built-in support for data parallelism, so it can process large volumes of data and speed up the time needed to train it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>5. Easy to use and understand:</b> Keras is easy to use and understand. You can easily implement complex neural networks with few lines of code. You don't need to understand low level details as it is already a wrapper around complex low level frameworks like TensorFlow, Theano or CNTK. So, it is a boon for beginners.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Related links</b>: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/07/building-simple-neural-network-in-keras.html">Create a simple sequential mode in Keras</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/07/building-cnn-model-in-keras-using-mnist.html">Create a CNN model in Keras</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>6. Pre-trained models:</b> Keras contains a lot of pre-trained neural network models for our general purpose requirements. For example, for image classification, we don't need to create a CNN model from scratch. We can fine-tune an existing and well trained model called VGG16 for this purpose. Similarly, there are a lot of other models available with Keras like InceptionV3, ResNet, MobileNet, Xception, InceptionResNetV2 etc. which we just need to fine-tune as per our needs.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Related links:</b></span><br />
<a href="http://theprofessionalspoint.blogspot.com/2019/06/transfer-learning-and-fine-tuning-model.html" style="font-family: arial, helvetica, sans-serif;">What is fine-tuning?</a><br />
<a href="http://theprofessionalspoint.blogspot.com/2019/07/fine-tune-vgg16-model-for-image.html" style="font-family: arial, helvetica, sans-serif;">Fine-tuning VGG16 model</a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>7. Great community:</b> As mentioned earlier, Keras has a great community support. You can easily find a lot of tutorials, detailed articles on various concepts, solved examples and a lot more. Keras is also very well documented.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Limitations of Keras</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As stated in point 1 and 2, Keras is only a high level API which uses other frameworks like TensorFlow, Theano and CNTK to perform low level tasks. If you want to research or write your own custom algorithm in deep learning project, you should use Tensorflow instead.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-49242023411233191602019-07-02T18:58:00.002-07:002019-07-06T20:11:09.554-07:00Building a simple sequential neural network with dense layers in Keras<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets understand how can we create a simple neural network in Keras. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a simple sequential model with dense layers (fully connected layers). We will use relu as an activation function in hidden layers and softmax in outer layer and Adam as SGD.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can download my Jupyter notebook containing below code from <a href="https://drive.google.com/file/d/1mdeGLkSKtideGz3StmsqB2yPYFl90vHx/view?usp=sharing">here</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1: Import required libraries</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import numpy as np</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from random import randint</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.preprocessing import MinMaxScaler</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">import keras</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.models import <b>Sequential</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.layers import <b>Dense</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">from keras.optimizers import <b>Adam</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">from sklearn.metrics import confusion_matrix, accuracy_score, classification_report</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 2: Create training and test dataset</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a hypothetical medical data and will try to predict whether a drug has any side effect or not on the people of different age groups. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">People are divided into two age groups: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">1. 13 years to 64 years and </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">2. 65 years to 100 years. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Label equal to 1 means that drug has side effect and 0 means no side effect. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create 2100 training observations. One array contains age which acts as sample and other array contains 0 and 1 which acts as label.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_samples = []</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_labels = []</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in range(50):</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_younger = randint(13,64)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_samples.append(random_younger)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_labels.append(1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_older = randint(65,100)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_samples.append(random_older)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_labels.append(0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in range(1000):</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_younger = randint(13,64)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_samples.append(random_younger)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_labels.append(0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_older = randint(65,100)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_samples.append(random_older)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> train_labels.append(1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Convert the above lists into numpy arrays as Keras expects samples and labels in the form of numpy arrays.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_samples = np.array(train_samples)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">train_labels = np.array(train_labels)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Similarly, create a test dataset.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_samples = []</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_labels = []</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in range(10):</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_younger = randint(13,64)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_samples.append(random_younger)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_labels.append(1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_older = randint(65,100)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_samples.append(random_older)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_labels.append(0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in range(200):</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_younger = randint(13,64)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_samples.append(random_younger)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_labels.append(0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> random_older = randint(65,100)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_samples.append(random_older)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> test_labels.append(1)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_samples = np.array(test_samples)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">test_labels = np.array(test_labels)</span><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3: Scale the training and test data</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">scaler = <b>MinMaxScaler</b>(feature_range=(0,1))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">scaled_train_samples = scaler.fit_transform((train_samples).reshape(-1,1))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">scaled_test_samples = scaler.fit_transform((test_samples).reshape(-1,1))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This is a preprocessing step. We need to scale our sample data in the range of 0 and 1. This is called feature scaling. For more details on feature scaling, you can go through <a href="http://theprofessionalspoint.blogspot.com/2019/02/why-to-standardize-and-transform.html">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 4: Create a model</b></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We will create a <b>sequential model</b> which is a linear stack of layers. We can create a sequential model by passing a list of layer instances to the constructor like this:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model = <b>Sequential</b>([</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> Dense(16, input_shape=(1,), activation='relu'),</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> Dense(32, activation='relu'),</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> Dense(2, activation='softmax'),</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can also simply add layers using <b>.add()</b> method:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model = Sequential()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dense(16, input_shape=(1,), activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dense(32, activation='relu'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.add(Dense(2, activation='softmax'))</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are using <b>dense layers</b> in the above Keras code which denote fully connected layers in a neural network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For hidden layers, we are using <b>relu </b>activation function and for outer layer, we are using softmax activation function. To know the difference between <b>relu </b>and <b>softmax </b>activation functions, please consider <a href="http://theprofessionalspoint.blogspot.com/2019/06/difference-between-sigmoid-and-softmax.html">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 5: Model Summary</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.<b>summary</b>()</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">It will show the description of all the layers and parameters.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 6: Compile a model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.<b>compile</b>(Adam(lr=0.0001), loss='sparse_categorical_crossentropy', metrics=['accuracy'])</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We need to pass the optimizer we want to use, learning rate, loss function and metrics. We are using Adam as an optimizer. This is a variant of SGD (Stochastic Gradient Descent). There are a lot of other optimizers. To go in detail, you can consider visiting <a href="http://theprofessionalspoint.blogspot.com/2019/06/momentum-and-adaptive-learning-based.html">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 7: Train a model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">model.<b>fit</b>(scaled_train_samples, train_labels, validation_split=0.1, </span><span style="font-family: arial, helvetica, sans-serif;">batch_size=10, epochs=20, shuffle=True, verbose=2)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We need to pass training sample and label data, validation set, batch size, epochs, shuffle and verbose parameters. Validation set helps in removing the overfitting and increasing the generalization capabilities of the network. By default, shuffle is always true. These parameters are called hyperparameters and we need to tune these parameters. You can try with different batch sizes and epochs and observe the change in the results.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 8: Predict from the model</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">predictions = model.<b>predict</b>(scaled_test_samples, batch_size=10, verbose=0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in predictions:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> print(i)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Above code will give us the predictions in form of probabilities. If we need exact predictions, we need to use following code. Instead of <b>predict</b>, we need to use </span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>predict_classes </b>function.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">rounded_predictions = model.<b>predict_classes</b>(scaled_test_samples, batch_size=10, verbose=0)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">for i in rounded_predictions:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> print(i)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 9: Check accuracy</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We are going to use confusion matrix, accuracy score and classification report to check the accuracy of our neural network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">confusionMatrix = <b>confusion_matrix</b>(test_labels, rounded_predictions)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">accuracyScore = <b>accuracy_score</b>(test_labels, rounded_predictions)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">classificationReport = <b>classification_report</b>(test_labels, rounded_predictions)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print(confusionMatrix)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print(accuracyScore * 100)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">print(classificationReport)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Hyperparameter Tuning:</b> In steps 6, 7 and 8, we are using a lot of hyperparameters. Network does not learn these parameters by itself. So, we need to tune these parameters explicitly in order to improve the performance and accuracy of the network. For more information on hyperparameters, you can go through </span><a href="http://theprofessionalspoint.blogspot.com/2019/06/hyperparameter-tuning-in-neural.html" style="font-family: arial, helvetica, sans-serif;">my this post</a><span style="font-family: "arial" , "helvetica" , sans-serif;">.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Related</b>: <a href="http://theprofessionalspoint.blogspot.com/2019/07/building-cnn-model-in-keras-using-mnist.html">Build a CNN model using Keras framework</a></span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com1tag:blogger.com,1999:blog-4423496461696137369.post-351539542189016822019-06-27T20:12:00.001-07:002019-09-14T20:04:57.141-07:00100+ Basic Deep Learning Interview Questions and Answers<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have listed down some basic deep learning interview questions with answers. These deep learning interview questions cover many concepts like perceptrons, neural networks, weights and biases, activation functions, gradient descent algorithm, CNN (ConvNets), CapsNets, RNN, LSTM, regularization techniques, dropout, hyperparameters, transfer learning, fine-tuning a model, autoencoders, NLP (Natural Language Processing), deep learning frameworks like TensorFlow and Keras, Google Cloud Vision API etc. I will keep adding more and more deep learning interview questions in this list. So, stay tuned.</span><br />
<b style="font-family: arial, helvetica, sans-serif;"><br /></b>
<b style="font-family: arial, helvetica, sans-serif;">Note</b><span style="font-family: "arial" , "helvetica" , sans-serif;">: For Machine Learning Interview Questions, refer </span><a href="http://theprofessionalspoint.blogspot.com/2019/01/100-basic-machine-learning-interview.html" style="font-family: arial, helvetica, sans-serif;">this link</a><span style="font-family: "arial" , "helvetica" , sans-serif;">.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Introduction</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">1. What is Deep Learning? How is it different from machine learning? What are the pros and cons of deep learning over machine learning? <a href="http://theprofessionalspoint.blogspot.com/2019/05/a-comparison-between-machine-learning.html" target="_blank">Answer</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">2. How does deep learning mimic the behavior of human brain? How will you compare an artificial neuron to a biological neuron?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Perceptron</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">3. What is a <b>Perceptron</b>? How does it work? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">What is a multi-layer perceptron?</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">4. What are the various limitations of a Perceptron? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Why cannot we implement XOR gate using Perceptron?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/perceptron-artificial-neuron.html" target="_blank">Answers to above questions</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Neural Networks</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">5. What are the various layers in a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">6. What are the various types of a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">7. What are <b>Deep </b>and <b>Shallow </b>neural networks? What are the advantages and disadvantages of deep neural networks over shallow neural networks?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/perceptron-artificial-neuron.html" target="_blank">Answers to above questions</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Weights and Bias</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">8. What is the importance of weights and biases in a neural network? What are the things to keep in mind while initializing weights and biases? <a href="http://theprofessionalspoint.blogspot.com/2019/05/perceptron-artificial-neuron.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">9. What is <b>Xavier Weight Initialization</b> technique? How is it helpful in initializing the weights? How does weight initialization vary for different types of activation functions? <a href="http://theprofessionalspoint.blogspot.com/2019/06/xavier-weight-initialization-technique.html" target="_blank">Answer</a> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">10. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Explain forward and backward propagation in a neural network. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How does a neural network update weights and biases during back propagation? <i>(See Gradient Descent section for answer)</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Activation Functions</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">11. What do you mean by activation functions in neural networks? Why do we call them squashing functions? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How do activation functions bring non-linearity in neural networks?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">12. Explain various activation functions like <b>Step (</b></span><b style="font-family: Arial, Helvetica, sans-serif;">Threshold</b><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>)</b>, </span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Logistic </b>(S</span><b style="font-family: Arial, Helvetica, sans-serif;">igmoid)</b><span style="font-family: "arial" , "helvetica" , sans-serif;">, </span><b style="font-family: Arial, Helvetica, sans-serif;">Hyperbolic Tangent (Tanh)</b><span style="font-family: "arial" , "helvetica" , sans-serif;">, </span><span style="font-family: "arial" , "helvetica" , sans-serif;">and </span><b style="font-family: Arial, Helvetica, sans-serif;">ReLU (Rectified Linear Unit)</b><span style="font-family: "arial" , "helvetica" , sans-serif;">. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">What are the various advantages and disadvantages of using these activation functions? </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/activation-squashing-functions-in-deep.html" target="_blank">Answers to above questions</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">13<b>. Dying and Leaky ReLU:</b> What do you mean by Dying ReLU? When a neuron is considered as dead in a neural network? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How does l</span><span style="font-family: "arial" , "helvetica" , sans-serif;">eaky ReLU</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> help in dealing with dying ReLU? </span><a href="http://theprofessionalspoint.blogspot.com/2019/06/dying-relu-causes-and-solutions-leaky.html" style="font-family: arial, helvetica, sans-serif;" target="_blank">Answer</a><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">14. What is the difference between <b>Sigmoid </b>and <b>Softmax</b> activation functions? <a href="http://theprofessionalspoint.blogspot.com/2019/06/difference-between-sigmoid-and-softmax.html" target="_blank">Answer</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Batches</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">15. Explain the terms: </span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Epochs</b>, <b>Batches </b>and <b>Iterations </b>in neural networks.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></span><span style="font-family: "arial" , "helvetica" , sans-serif;">16. What do you mean by <b>Batch Normalization</b>? What are its various advantages? <a href="http://theprofessionalspoint.blogspot.com/2019/06/batch-normalization-in-neural-networks.html" target="_blank">Answer</a></span><br />
<br />
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">Loss Function</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">17. What is the difference between </span><b style="font-family: arial, helvetica, sans-serif;">categorical_crossentropy </b><span style="font-family: "arial" , "helvetica" , sans-serif;">and </span><b style="font-family: arial, helvetica, sans-serif;">sparse_categorical_crossentropy</b><span style="font-family: "arial" , "helvetica" , sans-serif;">? Which one to use and when?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Hint</b>: For one hot encoded labels, use categorical_crossentropy. Otherwise, use sparse_categorical_crossentropy.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Gradient Descent</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">18. What is Gradient Descent? How is it helpful in minimizing the loss function? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">What are its various types? </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">19. Explain <b>Batch</b>, <b>Stochastic</b>, and <b>Mini Batch</b> Gradient Descent. What are the advantages and disadvantages of these Gradient Descent methods? <a href="http://theprofessionalspoint.blogspot.com/2019/06/gradient-descent-algorithm-in-deep.html" target="_blank">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">20. Explain these terms in context of SGD: <b>Momentum</b>, <b>Nesterov Momentum</b>, <b>AdaGrad</b>, <b>AdaDelta</b>, <b>RMSprop</b>, <b>Adam</b>. <a href="http://theprofessionalspoint.blogspot.com/2019/06/momentum-and-adaptive-learning-based.html" target="_blank">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">21. What is the difference between <b>Local and Global Minima</b>? What are the ways to avoid local minima? <a href="http://theprofessionalspoint.blogspot.com/2019/06/global-and-local-minima-in-gradient.html" target="_blank">Answer</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">22. Explain <b>Vanishing </b>and <b>Exploding </b>Gradients.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">23. What is <b>Learning Rate</b>? How does low and high learning rate affect the performance and accuracy of a neural network? <a href="http://theprofessionalspoint.blogspot.com/2019/06/hyperparameter-tuning-in-neural.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">24. If loss in a neural network is not decreasing during training period after so many iterations, what could be the possible reasons?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Hint</b>: Think of low / high learning rate, local and global minima (may be it </span><span style="font-family: "arial" , "helvetica" , sans-serif;">stuck at local minima</span><span style="font-family: "arial" , "helvetica" , sans-serif;">), high regularization parameter etc.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>CNN (ConvNets)</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">25. What is Convolutional Neural Network? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Explain various layers in a CNN? </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">26. What are the <b>Filters (Kernels)</b> in CNN? What is <b>Stride</b>?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">27. What do you mean by </span><b style="font-family: arial, helvetica, sans-serif;">Padding </b><span style="font-family: "arial" , "helvetica" , sans-serif;">in CNN? What is the difference between <b>Z</b></span><b style="font-family: arial, helvetica, sans-serif;">ero Padding</b><span style="font-family: "arial" , "helvetica" , sans-serif;"> and </span><b style="font-family: arial, helvetica, sans-serif;">Valid Padding</b><span style="font-family: "arial" , "helvetica" , sans-serif;">?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">28. What do you mean by <b>Pooling </b>in CNN? What are the various types of pooling? Explain <b>Max Pooling, Min Pooling, Average Pooling</b> and <b>Sum Pooling</b>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">29. What are the various hyperparameters in CNN which need to be tuned while training process?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">30. How is CNN different from traditional fully connected neural networks? Why we cannot use fully connected neural networks for image recognition?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">31. Suppose we have an input of n X n dimension and filter of f X f dimension. If we slide this filter over the input in the convolutional layer, what will be the dimension of the resulting output?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/basic-introduction-of-various-layers-in.html" target="_blank">Answers to above questions</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">CapsNets</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">32. What is Capsule Neural Network (<b>CapsNets</b>)? How is it different from CNN (ConvNets)? </span><a href="http://theprofessionalspoint.blogspot.com/2019/06/capsule-neural-networks-enhancement-of.html" style="font-family: arial, helvetica, sans-serif;" target="_blank">Answer</a><br />
<br />
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">Computer Vision</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">33. What is computer vision? How does deep learning help in solving various computer vision problems? <a href="https://machinelearningmastery.com/applications-of-deep-learning-for-computer-vision/">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">RNN</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">34. Explain RNN (Recurrent Neural Network). Why is RNN best suited for sequential data?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">35. What do you mean by feedback loop in RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">36. What are the various types of RNN? Explain with example:<b> One to One</b>, <b>One to Many</b>, <b>Many to One</b>, and <b>Many to Many</b> RNN.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">37. What is Bidirectional RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">38. What are the various issues with RNN? Explain Vanishing and Exploding Gradients. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">What are the various ways to solve these gradient issues in RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">39. What are the various advantages and disadvantages of RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">40. What are the various applications of RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">41. What are the differences between CNN and RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>LSTM</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">42. How does LSTM (Long Short Term Memory) solve Vanishing Gradient issue in RNN?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">43. What are the gated cells in LSTM? What are the various types of gates used in LSTM?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">44. What are the various applications of LSTM?</span><br />
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/basic-introduction-to-rnn-recurrent.html" target="_blank">Answers to all questions of RNN and LSTM</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">Regularization</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">45. What are the main causes of overfitting and underfitting in a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">46. What are the various regularization techniques used in a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">47. Explain <b>L1 and L2 Regularization</b> techniques used in a neural network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">48. What is <b>Dropout</b>? How does it prevent overfitting in a neural network? What are its various advantages and disadvantages? <a href="http://theprofessionalspoint.blogspot.com/2019/06/what-is-dropout-how-does-it-prevent.html" target="_blank">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">49. What is <b>Data Augmentation</b>? </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How does it prevent overfitting in a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">50. What is <b>Early Stopping</b>? How does it prevent overfitting in a neural network?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/06/regularization-techniques-used-in.html" target="_blank">Answers to above questions</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Learnable Parameters and Hyperparameters</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">51. What are the learnable parameters in a neural network? Explain with an example.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">52. What are the various hyperparameters used in a neural network? What are the various ways to optimize these hyper-parameters?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/06/hyperparameter-tuning-in-neural.html" target="_blank">Answers to above questions</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">53. How will you manually calculate number of weights and biases in a fully connected neural network? Explain with an example. <a href="https://www.youtube.com/watch?v=pg3hJpSopHQ">YouTube video</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">54. How will you manually calculate number of weights and biases in a convolutional neural network (CNN)? Explain with an example. <a href="https://www.youtube.com/watch?v=gmBfb6LNnZs">YouTube video</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Transfer Learning</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">55. What do you mean by <b>Transfer Learning</b> and <b>Fine-tuning </b>a model? What are its various advantages? What are the various steps to fine-tune a model? <a href="http://theprofessionalspoint.blogspot.com/2019/06/transfer-learning-and-fine-tuning-model.html" target="_blank">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Autoencoders</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">56. What are <b>Autoencoders</b>?</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> What are the various components of an autoencoder? Explain <b>encoder</b>, <b>decoder </b>and <b>bottleneck</b>. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How does an autoencoder work?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">57. What do you mean by <b>latent space representation</b> and <b>reconstruction loss</b> in an autoencoder?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">58. What are the various properties of an autoencoder?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">59. What are the various types of an autoencoder? Explain <b>Undercomplete </b>autoencoder, <b>Sparse</b> autoencoder, <b>Denoising </b>autoencoder, <b>Convolutional </b>autoencoder, <b>Contractive </b>autoencoders and <b>Deep </b>autoencoders.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">60. How do we add regularization capabilities to autoencoders?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">61. What are the various applications of an autoencoder?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">62. What are the various hyperparameters we need to tune in an autoencoder?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">63. How will you compare Autoencoders with PCA (Principal Component Analysis)?</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">64. What is <b>RBM (</b></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Restricted Boltzman Machine)</b>? What is the difference between an Autoencoder and RBM?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/06/autoencoders-in-deep-learning.html" target="_blank">Answers to above questions</a></span><br />
<br />
<b style="font-family: arial, helvetica, sans-serif; font-size: x-large;">NLP (Natural Language Processing)</b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">65. What are the various steps involved in preprocessing of the raw data in NLP? Explain Tokenization, Stemming, Lemmatization and Vectorization.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">66. What is the difference between </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Stemming and Lemmatization?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">67. What are the various types of </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Vectorization</span><span style="font-family: "arial" , "helvetica" , sans-serif;">? Explain </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Count Vectorization, N-grams Vectorization and Term Frequency - Inverse Document Frequency (TF-IDF).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/09/preprocessing-of-raw-data-in-nlp.html">Answers to above questions</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-size: large;"><b>Frameworks</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">68. What are the various frameworks available to implement deep learning models? What should be the characteristics of an ideal deep learning framework? <a href="http://theprofessionalspoint.blogspot.com/2019/06/basic-frameworks-to-implement-deep.html" target="_blank">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">TensorFlow</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">69. Explain TensorFlow architecture.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">70. What is a Tensor? Explain Tensor Datatypes and Ranks.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">71. What are Constants, Placeholders and Variables in a TensorFlow? Why do we need to initialize variables explicitly?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">72. What is a Computational Graph? What are the nodes and edges in it? How to build and run the graph using session? What are its various advantages?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">73. What is a Tensor Board? How is it useful?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">74. What is a TensorFlow Pipeline? How is it useful?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">75. Explain these terms: Feed Dictionary and Estimators</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="http://theprofessionalspoint.blogspot.com/2019/05/tensorflow-tensors-computational-graphs.html" target="_blank">Answers to above questions</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">76. Write a sample code to demonstrate constants, placeholders and variables in TensorFlow? </span><a href="http://theprofessionalspoint.blogspot.com/2019/06/constants-placeholders-variables-and.html" style="font-family: arial, helvetica, sans-serif;" target="_blank">Answer</a><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">77. Write a sample code using TensorFlow to demonstrate gradient descent? <a href="http://theprofessionalspoint.blogspot.com/2019/06/implement-simple-gradient-descent.html" target="_blank">Answer</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">78. Implement a Linear Classification Model using TensorFlow Estimator. <a href="http://theprofessionalspoint.blogspot.com/2019/06/implement-linear-classification-model.html" target="_blank">Answer</a></span><br />
<br />
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">Keras</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">79. What do you know about Keras framework? What are its various advantages and limitations? <a href="http://theprofessionalspoint.blogspot.com/2019/07/all-about-keras-framework-in-deep.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">80. How will you build a basic sequential model using Keras? <a href="http://theprofessionalspoint.blogspot.com/2019/07/building-simple-neural-network-in-keras.html">Answer</a> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">81. How will you solve a regression problem using sequential model in Keras? <a href="http://theprofessionalspoint.blogspot.com/2019/08/solving-regression-problem-using.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">82. How will you build a basic CNN model using Keras? <a href="http://theprofessionalspoint.blogspot.com/2019/07/building-cnn-model-in-keras-using-mnist.html">Answer</a> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">83. How will you build a basic LSTM model using Keras?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">84. What are the various pre-trained models available in Keras? How are these pre-trained models useful for us?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">85. How will you use VGG16 model to recognize a given image? <a href="http://theprofessionalspoint.blogspot.com/2019/08/image-recognition-using-pre-trained.html">Answer</a></span><br />
<a href="https://www.blogger.com/"></a><a href="https://www.blogger.com/"></a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">86. How will you fine-tune VGG16 model for image classification? <a href="http://theprofessionalspoint.blogspot.com/2019/07/fine-tune-vgg16-model-for-image.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent; color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent; color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">87. How will you fine-tune MobileNet model for image classification? What is the difference between VGG16 and MobileNet model?</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent; color: black; display: inline; float: none; font-family: "arial" , "helvetica" , sans-serif; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><br /></span></span>
<b style="font-family: arial, helvetica, sans-serif;"><span style="font-size: large;">Google Cloud Vision API</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">88. How will you use Google Cloud Vision API for Label Detection? </span><a href="http://theprofessionalspoint.blogspot.com/2019/09/image-recognition-label-detection-using.html" style="font-family: arial, helvetica, sans-serif;">Answer</a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">89. How will you use Google Cloud Vision API for Text Detection (OCR: Optical Character Recognition)? <a href="http://theprofessionalspoint.blogspot.com/2019/09/image-recognition-text-detection.html">Answer</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b></b><i></i><u></u><sub></sub><sup></sup><strike></strike><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Some of the above questions don't have answers by now. I am still writing answers for these questions and will keep this list updated. Although above list does not contain 100+ questions as claimed in the title of the post, but very soon I will take the count beyond 100.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com1tag:blogger.com,1999:blog-4423496461696137369.post-34245866258045976012019-06-25T07:33:00.000-07:002019-06-25T07:33:43.274-07:00Xavier Weight Initialization Technique in Neural Networks<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Weight initialization is the most important step while training the neural network. If weights are high, it may lead to exploding gradient. If weights are low, it may lead to vanishing gradient. Due to these issues, our model may take a long time to converge to global minima or sometimes it may never converge. So, weight initialization should be done with care. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Normally, weights are randomly initialized at the beginning. We use Gaussian distribution to </span><span style="font-family: "arial" , "helvetica" , sans-serif;">randomly distribute these weights such that the <b>mean</b> of the distribution is <b>zero</b> and <b>standard deviation</b> is <b>one</b>. But the problem with this approach was that variance or standard deviation tend to change in next layers which lead to explode or vanish the gradients.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Xavier Weight Initialization Technique</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">With each passing layer, we want the variance or standard deviation to remain the same. This helps us keep the signal from exploding to a high value or vanishing to zero. In other words, we need to initialize the weights in such a way that the variance remains the same with each passing layer. This initialization process is known as Xavier initialization. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In Xavier initialization technique, we need to pick the weights from a Gaussian distribution with <b>zero mean</b> and a <b>variance</b> of <b>1/N</b> (instead of 1), where N specifies the number of input neurons.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Notes</b>: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> Initially, it was suggested to take variance of 1/(Nin + Nout) instead of 1/N. Nin is the number of weights coming into the neuron and Nout is the number of weights going out of the neuron. But it was computationally complex, so it was discarded and we take only 1/N as variance.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2.</b> In Keras, Xavier technique is used by default to initialize the weights in the neural network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>For ReLU activation function</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If we are using ReLU as activation function in hidden layers, we need to go through following steps to implement Xavier initialization technique:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> Generate random weights from a Gaussian distribution having mean 0 and a standard deviation of 1.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2</b>. Multiply those random weights with the <b>square root of (2/n)</b>. Here n is number of input units for that layer.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>For other activation functions like Sigmoid or Hyperbolic Tangent</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If we are using Sigmoid or Tanh as activation function in hidden layers, we need to go through following steps to implement Xavier initialization technique:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> Generate random weights from a Gaussian distribution having mean 0 and a standard deviation of 1.</span><br />
<b style="font-family: arial, helvetica, sans-serif;"><br /></b>
<b style="font-family: arial, helvetica, sans-serif;">2.</b><span style="font-family: arial, helvetica, sans-serif;"> Multiply those random weights with the </span><b style="font-family: arial, helvetica, sans-serif;">square root of (1/n)</b><span style="font-family: arial, helvetica, sans-serif;">. Here n is number of input units for that layer.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-39791760555782348452019-06-24T19:32:00.002-07:002019-06-24T20:04:05.026-07:00Batch Normalization in Neural Networks in Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Batch normalization (<b>batchnorm</b>) is a technique to improve performance and accuracy of a neural network. Many times, normalization and standardization terms are used interchangeably. For more details on normalization and standardization, you can visit<a href="http://theprofessionalspoint.blogspot.com/2019/03/feature-scaling-techniques.html" target="_blank"> my this article</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Batch normalization occurs per batch, that is why, it is called batch normalization. We normalize </span><span style="font-family: "arial" , "helvetica" , sans-serif;">(mean = 0, standard deviation = 1)</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><span style="font-family: "arial" , "helvetica" , sans-serif;">the output of a layer before applying the activation function, and then feed it into the next layer in a neural network. So, </span><span style="font-family: "arial" , "helvetica" , sans-serif;">instead of just normalizing the inputs to the network, we normalize the inputs to each hidden layer within the network.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Advantages of Batch Normalization</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. Solves internal covariate shift:</b> In a neural network, each hidden unit’s input distribution changes every time when there is a parameter update in the previous layer. This is called internal covariate shift. This makes training slow and requires a very small learning rate and a good parameter initialization. This problem is solved by normalizing the layer’s inputs over a mini-batch.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Solves Vanishing and Exploding Gradient issues:</b> Unstable gradients like vanishing gradients and exploding gradients are the common issues which occur while training a neural network. By normalizing the outputs of each layer, we can significantly deal with this issue.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Training becomes faster: </b>In traditional neural networks, if we use higher learning rate, we may face exploding gradient issue. Also, if we use higher learning rate, there are possibilities that network may not converge at all and keeps oscillating around the global minima. Due to this, we usually prefer lower learning rate in traditional neural networks. But, with lower learning rate, as the networks get deeper, gradients get smaller during back propagation, and so require even more iterations which increases training period. But, if we normalize the output of each layer, we can safely use higher learning rate due to which we can drastically reduce the training period.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4. Solves dying ReLU problem:</b> ReLUs often die out during training the deep neural networks and many neurons stop contributing. But, with batch normalization, we can regulate the output of each hidden layer, which prevents this issue in deep neural networks. For more detail, on dying ReLU, you can refer <a href="http://theprofessionalspoint.blogspot.com/2019/06/dying-relu-causes-and-solutions-leaky.html" target="_blank">my this article</a>.</span><br />
<b style="font-family: arial, helvetica, sans-serif;"><br /></b><b style="font-family: arial, helvetica, sans-serif;">5. Introduces regularization:</b><span style="font-family: "arial" , "helvetica" , sans-serif;"> Batch normalization all provides some sort of regularization to the neural network which increases the generalization capabilities on the network.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-516038545580959972019-06-20T20:38:00.001-07:002019-06-20T20:38:48.492-07:00Transfer Learning and Fine Tuning a model in Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Arial, Helvetica, sans-serif;">Transfer learning and fine-tuning terms are very similar in many ways and widely used almost interchangeably. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Fine-tuning:</b> Suppose you already have an efficient deep learning model which performs task A. Now you have to perform a task B which is quite similar to task A. You don't need to create a separate model from scratch for task B. Just fine-tune the existing model which is efficiently performing task A. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Example</b>: You have a well trained model which identifies all types of cars. Car model has already learned a lot of features like edges, shapes, textures, head lights, door handles, tyres, windshield </span><span style="font-family: Arial, Helvetica, sans-serif;">etc. Now you have to create a model which can identify trucks. We know that many features of cars and trucks are similar. So, why to create a new model for trucks from scratch. Lets just tweak the existing car model to create a new model for truck. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Transfer Learning:</b> We can transfer the learning from the existing model on cars to new model on trucks. So, transfer learning happens while fine-tuning an existing model.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Advantages of Transfer Learning and Fine Tuning:</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Creating a new model is a very tough and time consuming task. We need to decide a lot of things while creating a model like:</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">1. Different types of layers to use (fully connected, convoluted, capsule, LSTM etc.)</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">2. How many layers to use?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">3. How many nodes in a layer?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">4. Which activation function to use in which layer?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">5. Which regularization techniques to use?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">6. Which optimizer to use?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">7. Tuning various hyperparameters like initializing weights, learning rate, momentum, batch size, number of epochs etc.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So, if we can fine-tune an existing model, we can very well escape from above tasks and save our time and energy.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>How to fine-tune a model? </b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">We need to make some reasonable changes and tweaks to our existing model to create a new model. Below are some basic steps to fine-tune an existing model:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>1. Remove output layer:</b> First of all remove output layer which was identifying cars. Add a new output layer which will now identify trucks.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>2. Add and remove hidden layers:</b> Trucks have some features different from cars. So accordingly, add some hidden layers which will learn new features of trucks. Remove those hidden layers which are not required in case of trucks.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>3. Freeze the unchanged layers:</b> </span><span style="font-family: Arial, Helvetica, sans-serif;">Freeze the layers which are maintained (not changed) so that no weight update happens on them when we again train this model on the new data with trucks. Weight should only be updated on new hidden layers. </span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-89835051386509665352019-06-18T19:57:00.002-07:002019-06-18T19:59:28.371-07:00Capsule Neural Networks: An enhancement of Convolutional Neural Networks (ConvNets vs CapsNets)<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Capsule Neural Networks can be seen as an enhancement of Convolutional Neural Networks. In order to understand capsule neural networks, lets first recap convolutional neural networks (CNN). In CNN, initial layers detect simple features like edges, curves, color gradients etc. Deeper convolutional layers start combining the simple features into comparatively complex features and so on. But in doing so, <b>CNN does not take care of orientational and relative spatial relationships between the features </b>or components. So, sometimes, CNN can be easily tricked. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For example, in <b>face recognition</b>, CNN does not take care of placements of eyes, nose, mouth, lips etc. Even if lips are near to eyes or eyes are below the mouth, it will still consider it a face. If all the features or components of face are available, it will consider it as a face without taking care of the orientation and placement of those components. Capsule networks take care of this.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have written a <a href="http://theprofessionalspoint.blogspot.com/2019/05/basic-introduction-of-various-layers-in.html" target="_blank">separate post on CNN</a>. Please go through it for detailed information on CNN.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Pooling layer problem in CNN:</b> Pooling layer is used to perform down-sampling the data due to which a lot of information is lost. These layers reduce the spatial resolution, so their outputs are invariant to small changes in the inputs. This is a problem when detailed information must be preserved throughout the network. With CapsNets, detailed pose information (such as precise object position, rotation, thickness, skew, size, and so on) is preserved throughout the network. Small changes to the inputs result in small changes to the outputs—information is preserved. This is called "equivariance."</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Capsule</b>: Human brain is organized into modules called capsules. Considering this fact, concept of capsule was put forward by Hilton. A capsule can be considered as a group of neurons. We can add as many neurons to a capsule to capture different dimensions of an image like scale thickness, stroke thickness, width, skew, translation etc. It can maintain information such as equivariance, hue, pose, albedo, texture, deformation, speed, and location of the object.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Dynamic Routing Algorithm:</b> Human brain has a mechanism to route information among capsules. On similar mechanism, dynamic routing algorithm was suggested by Hilton. This algorithm allows capsules to communicate with each other. For more details, please visit this article:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<a href="https://arxiv.org/abs/1710.09829" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">Dynamic Routing Between Capsules</a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Squashing Function:</b> Instead of ReLU, a new squashing function was suggested by Hilton known as novel squashing function. It is used to normalize the magnitude of vectors so that it falls between 0 and 1. The outputs from these squash functions tell us how to route data through various capsules that are trained to learn different concepts.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Limitations of Capsule Neural Networks</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><br /></b></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1.</b> As compared to the CNN, the training time for the capsule network is slower because of its computational complexity.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2.</b> It has been tested over MNIST dataset, but how will it behave on complex dataset, is still unknown.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3.</b> This concept is still under research. So, it has a lot of scope for improvement.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I would suggest to go through <a href="https://papers.nips.cc/paper/6975-dynamic-routing-between-capsules.pdf" target="_blank">this PDF</a> for more details on Capsule Neural Networks.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-44967471293263062282019-06-15T19:15:00.001-07:002019-07-15T18:13:50.627-07:00Difference between Sigmoid and Softmax function in deep learning<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Softmax function can be understood as a generalized version of a sigmoid function or an extension of a sigmoid function. Softmax function is usually used in the output layers of neural networks. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Following are some of the <b>differences between Sigmoid and Softmax function</b>:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. </b>The sigmoid function is used for the two-class (binary) classification problem, whereas the softmax function is used for the <b>multi-class classification problem</b>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2.</b> <b>Sum of all softmax units are supposed to be 1</b>. In sigmoid, it’s not really necessary. Sigmoid just makes output between 0 to 1. The softmax enforces that the sum of the probabilities of all the output classes are equal to one, so in order to increase the probability of a particular class, softmax must correspondingly decrease the probability of at least one of the other classes. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">When you use a softmax, basically you get a probability of each class (join distribution and a multinomial likelihood) whose sum is bound to be one. In case, you use sigmoid for multi class classification, it’d be like a marginal distribution and a Bernoulli likelihood.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Formula for Sigmoid and Softmax</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sigmoid function:</span><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG_tkTtnrFcY7aLf7iZ3nZWPm2MSzV5C22uIMrfpJr0pxQXE7qWvD7o_0T_8jsriMqsuzVn7CV1_GzXMV9obH3WfsBjYacF_5-KvDQx1CcGlILkDHjW_wxRhRcpOLtyGjtX02p6wYDLPa9/s1600/sigmoid_formula.PNG" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="74" data-original-width="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG_tkTtnrFcY7aLf7iZ3nZWPm2MSzV5C22uIMrfpJr0pxQXE7qWvD7o_0T_8jsriMqsuzVn7CV1_GzXMV9obH3WfsBjYacF_5-KvDQx1CcGlILkDHjW_wxRhRcpOLtyGjtX02p6wYDLPa9/s1600/sigmoid_formula.PNG" /></a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Softmax function:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDaz4MJZo_xaw8QXI1UjCBb-y388YEUIFjDoHweMZPeij3kQuHH_ocqarBQKD_gTuEdqve3d0TiXjqef4TUKesvBmjbk36Q9JsetgoxcXZKcEPF91Tyn9munJrFIcFRpSTo8s5jCogMuwG/s1600/softmax_function.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="152" data-original-width="355" height="85" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDaz4MJZo_xaw8QXI1UjCBb-y388YEUIFjDoHweMZPeij3kQuHH_ocqarBQKD_gTuEdqve3d0TiXjqef4TUKesvBmjbk36Q9JsetgoxcXZKcEPF91Tyn9munJrFIcFRpSTo8s5jCogMuwG/s200/softmax_function.PNG" width="200" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Let me illustrate the point 2 with an example here. Lets say, we have 6 inputs: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">[1,2,3,4,5,6]</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If we pass these inputs through the sigmoid function, we will get following output:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">[0.5, 0.73, 0.88, 0.95, 0.98, 0.99] </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sum of the above output units is 5.03 which is greater than 1. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">But in case of softmax, the sum of output units is always 1. Lets see how? Pass the same input to softmax function, and we get following output:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">[0.001, 0.009, 0.03, 0.06, 0.1, 0.8] which sums up to 1.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4.</b> Sigmoid is usually used as an activation function in hidden layers (but we use ReLU nowadays) while Softmax is used in output layers.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A general rule of thumb is to use ReLU as an activation function in hidden layers and softmax in output layer in a neural networks. For more information on activation functions, please visit <a href="http://theprofessionalspoint.blogspot.com/2019/05/activation-squashing-functions-in-deep.html" target="_blank">my this post</a>.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0tag:blogger.com,1999:blog-4423496461696137369.post-61796211516646001182019-06-14T19:56:00.001-07:002019-06-21T18:32:05.096-07:00Regularization Techniques used in Neural Networks in Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Ideally, the neural networks should never underfit and overfit and maintain good generalization capabilities. For this purpose, we use various regularization techniques in our neural networks. Below is the list of some of the regularization techniques which are commonly used to improve the performance and accuracy of the neural networks in deep learning.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>1. L1 and L2 Regularization</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">L1 and L2 are the most common types of regularization techniques used in machine learning as well as in deep learning algorithms. These update the general cost function by adding another term known as the regularization penalty. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For more details, please go through <a href="http://theprofessionalspoint.blogspot.com/2019/03/difference-between-ridge-regression-l2.html" target="_blank">my this article</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>2. Dropout</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Dropout can be seen as temporarily deactivating or ignoring neurons in the hidden layers of a network. Probabilistically dropping out nodes in the network is a simple and effective regularization method. We can switch off some neurons in a layer so that they do not contribute any information or learn any information and the responsibility falls on other active neurons to learn harder and reduce the error.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For more details on dropout, please consider visiting <a href="http://theprofessionalspoint.blogspot.com/2019/06/what-is-dropout-how-does-it-prevent.html" target="_blank">my this post</a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>3. Data Augmentation</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">Creating new data by making reasonable modifications to the existing data is called data augmentation. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Lets take an example of our MNIST dataset (hand written digits). We can easily generate thousands of new similar images by rotating, flipping, scaling, shifting, zooming in and out, cropping, changing or varying the color of the existing images.</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can use data augmentation technique when our model is overfitting due to less data.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In many cases in deep learning, increasing the amount of data is not a difficult task as we discussed above the case of MNIST dataset. In machine learning, this task is not that easy as we need labeled data which is not easily available. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>4. Early Stopping</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">While training a neural network, there will be a point during training when the model will stop generalizing and start learning the noise in the training dataset. This leads to overfitting.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">One approach to solve this problem is to treat the number of training epochs as a hyperparameter and train the model multiple times with different values, then select the number of epochs that result in the best performance. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The downside of this approach is that it requires multiple models to be trained and discarded. This can be computationally inefficient and time-consuming.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Another approach is <b>early stopping</b>. The model is evaluated on a validation dataset after each epoch. If the performance of the model on the validation dataset starts to degrade (e.g. loss begins to increase or accuracy begins to decrease), then the training process is stopped. The model at the time when the training is stopped, is then used and is known to have good generalization performance.</span></div>
Naresh Kumarhttp://www.blogger.com/profile/03334034022779238705noreply@blogger.com0