2. Introduction to Variables

A TensorFlow variable is the recommanded way to represent shared, persistent state your program manipulates.
This guide covers how to create, update, and manage instances of tf.Variable in Tensorflow.
Higher level libraries like tf.keras use lf.Variable to store model parameters.

import tensorflow as tf

Create a variable

To create a variable, provide an initial value. The tf.Variable will have the same dtype as the initialization value.
A variable looks and acts like a tensor, and is a data structure backed by a tf.Tensor.

float_var = tf.Variable([[1.0, 2.0], [3.0, 4.0]])
bool_var = tf.Variable([False, False, False, True])
complex_var = tf.Variable([5 + 3j, 6 + 3j])

print(float_var)
print(bool_var)
print(complex_var)

print("Shape of float variable: ", float_var.shape)
print("DType of float variable: ", float_var.dtype)
print("Numpy of float variable: \n", float_var.numpy())

<tf.Variable 'Variable:0' shape=(2, 2) dtype=float32, numpy=
array([[1., 2.],
       [3., 4.]], dtype=float32)>
<tf.Variable 'Variable:0' shape=(4,) dtype=bool, numpy=array([False, False, False,  True])>
<tf.Variable 'Variable:0' shape=(2,) dtype=complex128, numpy=array([5.+3.j, 6.+3.j])>

You can reassign the tensor using tf.Variable.assign .
Calling assign does not (usually) allocate a new tensor; instead, the existing tensor's memory is reused.

var = tf.Variable([2.0, 3.0])
print("Before var address: ", id(var))
var.assign([1, 2])
print("After var address: ", id(var))

Before var address:  1946689182536
After var address:  1946689182536

Lifecycles, naming, and watching

In Python-based TensorFlow, tf.Variable instance have the same lifecycle as other Python objects. When there are no references to a variable it is automatically dealoocated.
Variable names are preserved when saving and loading models. By default, variables in models will acquire unique variable names automatically, so you don't need to assign them yourself unless you want to.
Although variables are important for differentiation, some variables will not need to be differentiated. You can turn off gradients for a variable by setting trainable to false at creation.

tensor = tf.Variable([2,3], trainable=False, name="Mark")
print("Untrainable tensor: ", tensor)

Untrainable tensor:  <tf.Variable 'Mark:0' shape=(2,) dtype=int32, numpy=array([2, 3])>

Placing variables and tensors

For better performance, TensorFlow will attempt to place tensors and variables on the fastest device compatible with its dtype. However, you can override this. Place a float tensor and a variable on the CPU, even if a GPU is available.

with tf.device('CPU:0'):
    a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
    b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
    c = tf.matmul(a, b)
print(c)

tf.Tensor(
[[22. 28.]
 [49. 64.]], shape=(2, 2), dtype=float32)

It's possible to set the location of a variable or tensor on one device and do the computation on another device. This will introduce delay, as data needs to be copied between the devices.

with tf.device('CPU:0'):
    a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
    b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])

with tf.device('GPU:0'):
    c = tf.matmul(a, b)
print(c)

tf.Tensor(
[[22. 28.]
 [49. 64.]], shape=(2, 2), dtype=float32)