TensorFlow library was developed by the Google Brain Team for complex numeric calculations (like numpy). It relies on a lot of matrix multiplications. Later on, Google started using it as a library for deep learning. First stable version of TensorFlow appeared in 2017. It is an open source library under Apache Open Source license.
TensorFlow allows you to create large-scale neural networks with many layers like CNN, RNN etc. TensorFlow is a computational framework used to build deep learning models. It is an open source library for numerical computation and large scale machine learning.
Tensors and TensorFlow
A tensor is a vector or matrix of n-dimensions that represents all types of data. We can say that a tensor is a collection of feature vectors (i.e. array) of n-dimensions. Tensors can be considered as dynamically sized multi-dimensional array. The shape of the data is the dimensionality of the matrix or array. One dimensional tensor is known as scalar.
“TensorFlow” has been derived from the operations which neural networks perform on the tensors. It’s literally a flow of tensors. Tensor goes inside, flows through various nodes in neural network, and then comes out. That is why, it is called TensorFlow.
TensorFlow is made up of two terms – Tensor and Flow: In TensorFlow, the term tensor refers to the representation of data as multi-dimensional array whereas the term flow refers to the series of operations that one performs on tensors.
Tensor Ranks: 0 (scaler), 1 (vector), 2 (matrix), 3 (3-tensor)......., n (n-tensor)
Why TensorFlow?
1. It provides both C++ and Python APIs.
2. It has faster compilation time as compared to other deep learning libraries like Keras and PyTorch.
3. It supports both CPU and GPU computing devices.
4. Available both on mobile and desktop.
5. TensorFlow runtime is a cross platform library.
Versions of TensorFlow
1. TensorFlow with CPU support only
2. TensorFlow with GPU support
The model can be trained and used on GPUs as well as CPUs.
Google Colaboratory: It is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. With Colaboratory you can write and execute code, save and share your analyses, and access powerful computing resources, all for free from your browser.
Graph (Computational graph)
Graph is made up of nodes and edges. Series of TensorFlow operations are arranged as nodes in the computational graph.
Nodes: Each nodes take 0 or more tensors as input and produces a tensor as output. Node carries the mathematical operation and produces an endpoints outputs.
Properties of a node: unique label (name), dimension (shape), data type (dtype)
Datatypes: float32, float64, int8, int16, int32, int64, uint8, string and bool
Types of nodes: constant, placeholder, Variable, SparseTensor
In case you have not provided the data type explicitly, TensorFlow will infer the type of the constant / variable from the initialized value.
Must initialize a variable in TensorFlow
Constants are initialized when you call tf.constant but variables are not initialized when you call tf.Variable.
To initialize all the variables in a TensorFlow program, you must explicitly call a special operation as shown below:
init = tf.global_variables_initializer()
sess.run(init)
Variables must be initialized before a graph is used for the first time.
TensorFlow variables are in-memory buffers that contain tensors, but unlike normal tensors that are only instantiated when a graph is run and are immediately deleted afterwards, variables survive across multiple executions of a graph.
Edges: Edges explain the input/output relationships between nodes. Edge of the nodes is the tensor, i.e. a way to populate the operation with data. Each operation is called an op node.
Executing a Graph: There are two steps involved while executing a graph:
1. Building a Computational Graph: Just create nodes and assign them operations.
2. Running a Computational Graph: We need to run the computational graph within a session. Session is also called TensorFlow Runtime. Session places the graph operations onto devices, such as CPUs or GPUs, and provides methods to execute them. A session encapsulates the control and state of the TensorFlow runtime i.e. it stores the information about the order in which all the operations will be performed and passes the result of already computed operation to the next operation in the pipeline.
Advantages of Computational Graph
1. Parallel execution: The operations assigned to different nodes of a computational graph can be performed in parallel, thus, providing a better performance in terms of computations.
Nodes and edges can be spread over several clusters of computers (in distributed manner).
2. Portability: The portability of the graph allows to preserve the computations for immediate or later use. The graph can be saved and executed in the future.
Pipeline, Batches, Epochs, Iterations and Estimators
Pipeline and Batches: If you have a dataset of 50 GB, and your computer has only 16 GB of memory, then the machine will crash. In this situation, you need to build a TensorFlow pipeline. The pipeline will load the data in batches. Each batch will be pushed to the pipeline and be ready for the training. It allows you to use parallel computing. It means TensorFlow will train the model across multiple CPUs or GPUs.
Tip: Use Pipeline if you have a large dataset. Use Pandas for less than 10GB data.
Epoch: An epoch defines how many times you want the model to see the data. One epoch is counted when all your data is once forward and backward propagated through the entire neural network.
Lets take an example. Suppose you have a dataset with 1200 observations. You are going to use a pipeline and have created 3 batches containing 400 observations each. Now, it will take 3 iterations (1200 / 400) to completely propagate the data forward and backward through the neural network to complete one epoch.
Estimators: It is a Tensorflow API used to implement algorithms. We can import following estimators APIs to solve a lot of classification and regression problems.
tf.estimator.LinearRegressor
tf.estimator.LinearClassifier
tf.estimator.BoostedTreesRegressor
tf.estimator.BoostedTreesClassifier
tf.estimator.DNNClassifier
tf.estimator.DNNLinearCombinedClassifier
Estimators are used for creating computational graphs, initializing variables, training the model and saving checkpoint and logging files for Tensorboard. In order to user estimators we need to create feature columns and input functions.
Input functions are used for passing input data to the model for training and evaluation. Feature columns are specifications for how the model should interpret the input data. We will see these concepts in detail when we solve a problem using TensorFlow in my future posts.
TensorBoard
TensorBoard enables to monitor graphically and visually what TensorFlow is doing. TensorFlow is based on graph computation; it allows the developer to visualize the construction of the neural network with Tensorboad.
TensorFlow allows you to create large-scale neural networks with many layers like CNN, RNN etc. TensorFlow is a computational framework used to build deep learning models. It is an open source library for numerical computation and large scale machine learning.
Tensors and TensorFlow
A tensor is a vector or matrix of n-dimensions that represents all types of data. We can say that a tensor is a collection of feature vectors (i.e. array) of n-dimensions. Tensors can be considered as dynamically sized multi-dimensional array. The shape of the data is the dimensionality of the matrix or array. One dimensional tensor is known as scalar.
“TensorFlow” has been derived from the operations which neural networks perform on the tensors. It’s literally a flow of tensors. Tensor goes inside, flows through various nodes in neural network, and then comes out. That is why, it is called TensorFlow.
TensorFlow is made up of two terms – Tensor and Flow: In TensorFlow, the term tensor refers to the representation of data as multi-dimensional array whereas the term flow refers to the series of operations that one performs on tensors.
Tensor Ranks: 0 (scaler), 1 (vector), 2 (matrix), 3 (3-tensor)......., n (n-tensor)
Why TensorFlow?
1. It provides both C++ and Python APIs.
2. It has faster compilation time as compared to other deep learning libraries like Keras and PyTorch.
3. It supports both CPU and GPU computing devices.
4. Available both on mobile and desktop.
5. TensorFlow runtime is a cross platform library.
Versions of TensorFlow
1. TensorFlow with CPU support only
2. TensorFlow with GPU support
The model can be trained and used on GPUs as well as CPUs.
Google Colaboratory: It is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. With Colaboratory you can write and execute code, save and share your analyses, and access powerful computing resources, all for free from your browser.
Graph (Computational graph)
Graph is made up of nodes and edges. Series of TensorFlow operations are arranged as nodes in the computational graph.
Nodes: Each nodes take 0 or more tensors as input and produces a tensor as output. Node carries the mathematical operation and produces an endpoints outputs.
Properties of a node: unique label (name), dimension (shape), data type (dtype)
Datatypes: float32, float64, int8, int16, int32, int64, uint8, string and bool
Types of nodes: constant, placeholder, Variable, SparseTensor
In case you have not provided the data type explicitly, TensorFlow will infer the type of the constant / variable from the initialized value.
Must initialize a variable in TensorFlow
Constants are initialized when you call tf.constant but variables are not initialized when you call tf.Variable.
To initialize all the variables in a TensorFlow program, you must explicitly call a special operation as shown below:
init = tf.global_variables_initializer()
sess.run(init)
Variables must be initialized before a graph is used for the first time.
TensorFlow variables are in-memory buffers that contain tensors, but unlike normal tensors that are only instantiated when a graph is run and are immediately deleted afterwards, variables survive across multiple executions of a graph.
Edges: Edges explain the input/output relationships between nodes. Edge of the nodes is the tensor, i.e. a way to populate the operation with data. Each operation is called an op node.
Executing a Graph: There are two steps involved while executing a graph:
1. Building a Computational Graph: Just create nodes and assign them operations.
2. Running a Computational Graph: We need to run the computational graph within a session. Session is also called TensorFlow Runtime. Session places the graph operations onto devices, such as CPUs or GPUs, and provides methods to execute them. A session encapsulates the control and state of the TensorFlow runtime i.e. it stores the information about the order in which all the operations will be performed and passes the result of already computed operation to the next operation in the pipeline.
Advantages of Computational Graph
1. Parallel execution: The operations assigned to different nodes of a computational graph can be performed in parallel, thus, providing a better performance in terms of computations.
Nodes and edges can be spread over several clusters of computers (in distributed manner).
2. Portability: The portability of the graph allows to preserve the computations for immediate or later use. The graph can be saved and executed in the future.
Pipeline, Batches, Epochs, Iterations and Estimators
Pipeline and Batches: If you have a dataset of 50 GB, and your computer has only 16 GB of memory, then the machine will crash. In this situation, you need to build a TensorFlow pipeline. The pipeline will load the data in batches. Each batch will be pushed to the pipeline and be ready for the training. It allows you to use parallel computing. It means TensorFlow will train the model across multiple CPUs or GPUs.
Tip: Use Pipeline if you have a large dataset. Use Pandas for less than 10GB data.
Epoch: An epoch defines how many times you want the model to see the data. One epoch is counted when all your data is once forward and backward propagated through the entire neural network.
Lets take an example. Suppose you have a dataset with 1200 observations. You are going to use a pipeline and have created 3 batches containing 400 observations each. Now, it will take 3 iterations (1200 / 400) to completely propagate the data forward and backward through the neural network to complete one epoch.
Estimators: It is a Tensorflow API used to implement algorithms. We can import following estimators APIs to solve a lot of classification and regression problems.
tf.estimator.LinearRegressor
tf.estimator.LinearClassifier
tf.estimator.BoostedTreesRegressor
tf.estimator.BoostedTreesClassifier
tf.estimator.DNNClassifier
tf.estimator.DNNLinearCombinedClassifier
Estimators are used for creating computational graphs, initializing variables, training the model and saving checkpoint and logging files for Tensorboard. In order to user estimators we need to create feature columns and input functions.
Input functions are used for passing input data to the model for training and evaluation. Feature columns are specifications for how the model should interpret the input data. We will see these concepts in detail when we solve a problem using TensorFlow in my future posts.
TensorBoard
TensorBoard enables to monitor graphically and visually what TensorFlow is doing. TensorFlow is based on graph computation; it allows the developer to visualize the construction of the neural network with Tensorboad.
No comments:
Post a Comment