2. The Functional API

The main idea is that a deep learning model is usually a directed acyclic graph(DAG) of layers. So the functional API is a way to build graphs of layers.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

Create model by funtional API

To build this model using the functional API:

1. Creating an input node(keras.input) which return information about the shape and dtype of the input.

2. Add a layer node(keras.layers) by calling a layer on this input object.

3. Create a model(keras.Model) by specifying its inputs and outputs in the graph in layers.

# create a input node
inputs = keras.Input(shape=(784,))

# Add a layer node
x = layers.Dense(64, activation='relu')(inputs)
outputs = layers.Dense(10)(x)

# create a model
model = keras.Model(inputs, outputs)

model.summary()

Model: "model_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_3 (InputLayer)        [(None, 784)]             0         

 dense_4 (Dense)             (None, 64)                50240     

 dense_5 (Dense)             (None, 10)                650       

=================================================================
Total params: 50,890
Trainable params: 50,890
Non-trainable params: 0
_________________________________________________________________

Use the same graph of layers to define multiple models

In the functional API, models are created by specifying their inputs and outputs in a graph of layers.
That means that a single graph of layers can be used to generate multiple models.
Layer instance can use only once in each model instance. If Layer instance use more than once, it will share weight.

# create a input node
inputs = keras.Input(shape=(64,))

# Add a layer node
layer1 = layers.Dense(64)
layer2 = layers.Dense(10)

x = layer1(inputs)
outputs = layer2(x)

'''share weight
x = layer1(inputs)
outputs = layer1(x)
'''

# create a model
model1 = keras.Model(inputs, outputs)
model2 = keras.Model(inputs, outputs)

model1.summary()
model2.summary()

Model: "model_20"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_13 (InputLayer)       [(None, 64)]              0         

 dense_24 (Dense)            (None, 64)                4160      

 dense_25 (Dense)            (None, 10)                650       

=================================================================
Total params: 4,810
Trainable params: 4,810
Non-trainable params: 0
_________________________________________________________________
Model: "model_21"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_13 (InputLayer)       [(None, 64)]              0         

 dense_24 (Dense)            (None, 64)                4160      

 dense_25 (Dense)            (None, 10)                650       

=================================================================
Total params: 4,810
Trainable params: 4,810
Non-trainable params: 0
_________________________________________________________________

All models are callable, just like layers

You can treat any model as if it were a layer by invoking it on an Input or on the output of another layer.

# create a input node
inputs = keras.Input(shape=(64,))

# Add a layer node
x = layers.Dense(10, activation='relu')(inputs)
first_outputs = layers.Dense(64)(x)
x = layers.Dense(10)(first_outputs)
second_outputs = layers.Dense(64)(x)

first_model = keras.Model(inputs, first_outputs)
second_model = keras.Model(first_outputs, second_outputs)

x = first_model(inputs)
y = second_model(x)

'''share weight
x = first_model(inputs)
x = first_model(x)
y = first_model(x)
'''

final_model1 = keras.Model(inputs, y)
final_model2 = keras.Model(inputs, y)
final_model1.summary()
final_model2.summary()

Model: "model_24"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_14 (InputLayer)       [(None, 64)]              0         

 model_22 (Functional)       (None, 64)                1354      

 model_23 (Functional)       (None, 64)                1354      

=================================================================
Total params: 2,708
Trainable params: 2,708
Non-trainable params: 0
_________________________________________________________________
Model: "model_25"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_14 (InputLayer)       [(None, 64)]              0         

 model_22 (Functional)       (None, 64)                1354      

 model_23 (Functional)       (None, 64)                1354      

=================================================================
Total params: 2,708
Trainable params: 2,708
Non-trainable params: 0
_________________________________________________________________

Manipulate complex graph topology

Models with mutiple inputs and outputs

The Functional API makes it easy to manipulate multiple inputs and outputs.

Models with residual connections

The Functional API can handle residual connections.

# multiple inputs and outputs
input1 = keras.Input(shape=(None,), name='input1')
input2 = keras.Input(shape=(None,), name='input2')
input3 = keras.Input(shape=(8,), name='input3')

embedding1 = layers.Embedding(10, 8)(input1)
embedding2 = layers.Embedding(10, 8)(input2)
#embedding3 = layers.Embedding(10, 8)(input3)

feature1 = layers.LSTM(32)(embedding1)
feature2 = layers.LSTM(16)(embedding2)

x = layers.concatenate([feature1, feature2, input3])

pred1 = layers.Dense(1, name='output1')(x)
pred2 = layers.Dense(3, name='output2')(x)

model = keras.Model(
    inputs = [input1, input2, input3],
    outputs = [pred1, pred2]
)

# Plot the model
keras.utils.plot_model(model, show_shapes=True)

# residual connection
input = keras.Input(shape=(64,),)
x = layers.Dense(32)(input)
output1 = layers.Dense(32)(x)

x = layers.Dense(32)(output1)
x = layers.Dense(32)(x)
output2 = layers.add([x, output1])

x = layers.Dense(32)(output2)
x = layers.Dense(32)(x)
output3 = layers.add([x, output2])

model = keras.Model(input, output3)

# Plot the model
keras.utils.plot_model(model, show_shapes=True)

Extract and reuse nodes in the graph of layers

Because the graph of layers you are manipulating is a static data structure, it can be accessed and inspected. And this is how you are able to plot functional models as images.
This also mean that you access the activations of intermediate layers and reuse them elsewhere.

# intermediate activations of the model
features_list = [layer.output for layer in model.layers]

# model that returns the values of the intermediate layer activation
feature_model = keras.Model(inputs=model.input, outputs=features_list)

# Plot the model
keras.utils.plot_model(feature_model, show_shapes=True)

Extend the API using custom layers

It's easy to extend the API by creating your own layers. All layers subclass the Layer class and implement:

• call method, that specifies the computation done by the layer.
• build method, that creates the weights of the layer (you can create weights in __init__ as well).

When to use the functional API

In general, the functional API is higher-level, easier and safer, and has a number of features that subclassed models do not support.
However, model subclassing provides greater flexibility when building models that are not easily expressible as directed acyclic graphs of layers.

Funtional API strengths:

The following properties are also true for Sequential models (which are also data structures), but not true subclased models (which are Python bytecode, not data structures).

- Less verbose:

There is no super(Myclass, self).__init__(...), no def call(self, ...):, etc:

- Model validation while defining its connectivity graph:
In the functional API, the input specification (shape and dtype) is created in advance (using Input). Every time you call a layer, the layer checks that the specification passed to it matches its assumptions, and it will raise a helpful error message if not.

This guarantees that any model you can build with the functional API will run. All debugging happens statically during the model construction and not at execution time.

- A functional model is plottable and inspectable:
You can plot the model as a graph, and you can easily access intermediate nodes in this graph.

- A functional model can be serialized or cloned:
Becuase a functional model is a data structure, it is safely serializable and can be saved as a single file that allows you to recreate the exact same model without having access to any of the original code.

Functional API weakness:

- It does not support dynamic architectures:
The functional API only treats models as DAGs of layers.