Introduction to modules, layers, and models
Contents
5. Introduction to modules, layers, and models¶
在用 tensorflow 做 machine learning的時候，會需要去 define, save, and restore a model.
在 tf 中， model 可被定義為：
A function that computes something on tensors (a forward pass)
Some variables that can be updated in response to training
在這份文件中，將 go below the surface of Keras to see how TensorFlow models are defined.
This looks at how TensorFlow collects variables and models, as well as how they are saved and restored.
5.1. Setup¶
import tensorflow as tf
from datetime import datetime
%load_ext tensorboard
5.2. tf.Module: Defining models and layers in TensorFlow¶
大部分的 model 都是由 layers 所組成.
Layers 其實就是 functions，而這個 function 是由可被重複使用的數學結構所定義，裡面有可訓練的變數。
在 tensorflow 中，大部分 high-level 的 layers 和 models，都是built on the same foundational class:
tf.Module.
5.2.1. toy example¶
class SimpleModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)
self.a_variable = tf.Variable(5.0, name="train_me")
self.non_trainable_variable = tf.Variable(5.0, trainable=False, name="do_not_train_me")
def __call__(self, x):
return self.a_variable * x + self.non_trainable_variable
simple_module = SimpleModule(name="simple")
simple_module(tf.constant(5.0))
<tf.Tensor: shape=(), dtype=float32, numpy=30.0>
來看一下，我們剛剛定義了一個 module (你可以叫他 module，也可以叫他 layer，都可以)，他最一開始就繼承了
tf.Moudle這個 class可以看到，這個 class，有
__call__這個 method，這就是一般 python callable 的定義方式，沒什麼特別再來看一下
__init__裡面，定義了兩個會用的屬性，也就是兩個tf.Variable。其中一個是 trainable (可微分)，另一個是不給 train (不可微分)然後，這個 module 要做的事情，就是，當你輸入 x 時，他會幫你乘上一個數，然後再加上一個數
從結果來看，輸入 5 後，得到 32
那，繼承
tf.Module這個 class 有什麼好處？好處就是，已經有寫好一些 method 和 attribute，會幫你省很多力氣。例如，他會自動幫你蒐集總共定義了多少個 tf.Variables，以及， trainable 的 variable 有哪些，
# all variables
simple_module.variables
(<tf.Variable 'train_me:0' shape=() dtype=float32, numpy=5.0>,
<tf.Variable 'do_not_train_me:0' shape=() dtype=float32, numpy=5.0>)
可以看到，他用一個 tuple，把所有 variable 給蒐集起來。名稱分別是
train_me和do_not_train_me.另外，自動微分最常用的，就是抓出 trainable variables 來做微分。這他也幫你蒐集好了：
# trainable variable
simple_module.trainable_variables
(<tf.Variable 'train_me:0' shape=() dtype=float32, numpy=5.0>,)
5.2.2. 自訂 layer¶
接著，我們可以來定義一個自己的
Dense(linear) layer:
class Dense(tf.Module):
def __init__(self, in_features, out_features, name=None):
super().__init__(name=name)
# Dense layer 的第一個參數，是 weight 矩陣 `w`，起始值用 normal 來生，shape 是 in_features x out_features
self.w = tf.Variable(
tf.random.normal([in_features, out_features]), name='w')
# Dense layer 的第二個參數，是 bias 向量 `b`，起始值給 0 向量， shape 為 1 x out_features
self.b = tf.Variable(tf.zeros([out_features]), name='b')
def __call__(self, x):
# 當我們 call Dense 的時候，就是輸入一個 x tensor (shape 為 1 x in_features)，然後去計算 x w + b，得到 shape 為 1 x out_features 的向量
y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)
可以想像， w 和 b 都是 trainable variable，之後就要靠自動微分來更新 w 和 b 的值
dense_1 = Dense(in_features = 3, out_features = 6)
dense_1.trainable_variables
(<tf.Variable 'b:0' shape=(6,) dtype=float32, numpy=array([0., 0., 0., 0., 0., 0.], dtype=float32)>,
<tf.Variable 'w:0' shape=(3, 6) dtype=float32, numpy=
array([[-0.2585963 , -0.72111183, -0.47151518, 1.1848954 , 1.4632958 ,
1.9215399 ],
[ 1.2155559 , -0.9481618 , -0.17013389, 1.3932179 , -1.2731404 ,
0.14196219],
[-1.3073918 , 0.33516636, 0.5894352 , -0.90030533, 0.88234293,
1.3543645 ]], dtype=float32)>)
可以看到，初始化後的 dense_1，有兩個 trainable variable，而且起始 weight 也列出來給你看了
可以用用看這個 dense 層的功能：輸入 1 x in_features 的 tensor，輸出 1 x out_features 的 tensor
x = tf.constant([[1.0, 2.1, 3.2]])
out = dense_1(x)
out
<tf.Tensor: shape=(1, 6), dtype=float32, numpy=
array([[0. , 0. , 1.0573964, 1.2296758, 1.6131986, 6.553627 ]],
dtype=float32)>
可以看到， output 是一個 1x6 的 tensor
5.2.3. 自訂 model¶
那如果我們想自定一個 model (從 input 一個 tensor，到 output 一個結果出來。重點在定義中間的 forward propagation 過程)，就可以沿用剛剛的定義方式。
假設，我想做一個 2 個 dense 的 NN (都是linear)，那我可以這樣定義：
class SequentialModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)
# 在這邊定義好，我等等會用到的 layer 有哪些
self.dense_1 = Dense(in_features=3, out_features=3)
self.dense_2 = Dense(in_features=3, out_features=2)
def __call__(self, x):
# forward logic
x = self.dense_1(x)
return self.dense_2(x)
# You have made a model!
my_model = SequentialModule(name="the_model")
# Call it, with random results
print("Model results:", my_model(tf.constant([[2.0, 2.0, 2.0]])))
Model results: tf.Tensor([[4.243844 0. ]], shape=(1, 2), dtype=float32)
tf.Module instances will automatically collect, recursively, any tf.Variable or tf.Module instances assigned to it. This allows you to manage collections of tf.Modules with a single model instance, and save and load whole models.
print("Submodules:", my_model.submodules)
Submodules: (<__main__.Dense object at 0x14bae3df0>, <__main__.Dense object at 0x10d486550>)
for var in my_model.variables:
print(var, "\n")
<tf.Variable 'b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>
<tf.Variable 'w:0' shape=(3, 3) dtype=float32, numpy=
array([[-2.247787 , 1.3312311, 1.2570276],
[-1.1049702, 1.2348688, 1.1046218],
[-1.5440533, -0.994251 , 1.4764204]], dtype=float32)>
<tf.Variable 'b:0' shape=(2,) dtype=float32, numpy=array([0., 0.], dtype=float32)>
<tf.Variable 'w:0' shape=(3, 2) dtype=float32, numpy=
array([[ 0.64399236, -0.01036486],
[ 0.13026635, -0.6883245 ],
[-0.00606108, 2.0914354 ]], dtype=float32)>
5.2.4. Waiting to create variables¶
剛剛可以看到，我們定義的 dense ，要定義 input tensor 的 shape (i.e. in_features 是多少
這有點麻煩，而且和 keras 內建的
tf.keras.layers.Dense不同keras 的 Dense 只要知道 out_features 就好， in_features 他會直接去讀你丟給他的 input tensor 來決定
所以，我們稍微改一下原本的 code，就可以做到這件事：
class FlexibleDenseModule(tf.Module):
# Note: No need for `in_features`
def __init__(self, out_features, name=None):
super().__init__(name=name)
self.out_features = out_features
# 加入這行
self.is_built = False # 起始狀態時，是 False
def __call__(self, x):
# 第一次 call 時 (self.is_built 為 False 時)
if not self.is_built:
# 在這裡才定義 w 和 b
self.w = tf.Variable(
tf.random.normal([x.shape[-1], self.out_features]), name='w')
self.b = tf.Variable(tf.zeros([self.out_features]), name='b')
self.is_built = True # 並修改狀態
y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)
# Used in a module
class MySequentialModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)
self.dense_1 = FlexibleDenseModule(out_features=3)
self.dense_2 = FlexibleDenseModule(out_features=2)
def __call__(self, x):
x = self.dense_1(x)
return self.dense_2(x)
my_model = MySequentialModule(name="the_model")
print("Model results:", my_model(tf.constant([[2.0, 2.0, 2.0]])))
Model results: tf.Tensor([[0. 0.]], shape=(1, 2), dtype=float32)
5.2.5. Save & load¶
剛剛不管是自己定義的 layer，或是 model，因為都繼承自
tf.Module，所以都可以追蹤到所有的 trainable variable 的 weights那在訓練過程中，我們當然就可以把這些 weights 給存下來，之後就可以從這個 weight 繼續 train 下去
存檔的方式有兩種：
checkpoint: 這就是只有存 weight，沒有存 module/layer structure。所以之後讀取時，要先建立一個一樣 module/layer structure 的 object 後，再把存好的 weight 塞回去
SaveModel: 這是把 module/layer structure 以及 對應的 weight，全都存下來。之後只要 load 這個 model 就好，大師兄就全都回來了。
5.2.5.1. checkpoints (save & load weights)¶
作法如下：
先用
checkpoint = tf.train.Checkpoint(model = my_model_obj)，來建立一個 checkpoint 物件再用
checkpoint.write('checkpoint_path')，來把 checkpoint 檔存出來
# 先建立一個 checkpoint 物件，他追蹤的是我剛剛 train 到一半的 my_model
checkpoint = tf.train.Checkpoint(model=my_model)
checkpoint
<tensorflow.python.training.tracking.util.Checkpoint at 0x141a2b490>
# 再把結果寫出來
chkp_path = "my_checkpoint"
checkpoint.write(chkp_path)
'my_checkpoint'
看一下資料夾，寫出兩個檔案：
data 本身 (i.e. my_checkpoint.data-00000-of-00001)。裡面包含 variable values and their attribute lookup paths
index file for metadata (i.e. my_checkpoint.index)。功能是 keeps track of what is actually saved and the numbering of checkpoints
!ls my_checkpoint*
my_checkpoint.data-00000-of-00001 my_checkpoint.index
You can look inside a checkpoint to be sure the whole collection of variables is saved, sorted by the Python object that contains them.
tf.train.list_variables(chkp_path)
[('_CHECKPOINTABLE_OBJECT_GRAPH', []),
('model/dense_1/b/.ATTRIBUTES/VARIABLE_VALUE', [3]),
('model/dense_1/w/.ATTRIBUTES/VARIABLE_VALUE', [3, 3]),
('model/dense_2/b/.ATTRIBUTES/VARIABLE_VALUE', [2]),
('model/dense_2/w/.ATTRIBUTES/VARIABLE_VALUE', [3, 2])]
During distributed (multi-machine) training they can be sharded, which is why they are numbered (e.g., ‘00000-of-00001’). In this case, though, there is only have one shard.
如果之後要把 weight 給 load 進來，那做法是：
先建立一個和之前一樣架構的 model/layer.
建立 checkpoint 物件，去追蹤這個新建立好的 model/layer
用這個新的 checkpoint 物件的
restore('')method，把剛剛的 weight 給 load 回來：
new_model = MySequentialModule()
new_checkpoint = tf.train.Checkpoint(model=new_model)
new_checkpoint.restore("my_checkpoint")
# Should be the same result as above
new_model(tf.constant([[2.0, 2.0, 2.0]]))
<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[0., 0.]], dtype=float32)>
5.2.5.2. SavedModel¶
另一種儲存的方式，是直接把 model 的 structure 和 weight 全都存起來
作法是：
用
tf.saved_model.save(my_model_obj, "a_folder_path_to_save_model").用
new_model = tf.saved_model.load("a_folder_path_to_save_model")來讀檔
看一下範例：
tf.saved_model.save(my_model, "the_saved_model")
INFO:tensorflow:Assets written to: the_saved_model/assets
可以看到，檔案被存到
the_saved_model這個資料夾中。看一下這個資料夾裡有什麼東西：
assets資料夾： 空的variables資料夾variables.data-00000-of-00001: 這就是剛剛 checkpoint 的 data 檔
variables.index: 這就是剛剛 checkpoint 的 index 檔
saved_model.pb檔案： a protocol buffer describing the functionaltf.Graph.
# Inspect the SavedModel in the directory
!ls -l the_saved_model
total 24
drwxr-sr-x 2 kbuilder kokoro 4096 Oct 26 01:29 assets
-rw-rw-r-- 1 kbuilder kokoro 14702 Oct 26 01:29 saved_model.pb
drwxr-sr-x 2 kbuilder kokoro 4096 Oct 26 01:29 variables
讀檔時，這樣讀：
new_model = tf.saved_model.load("the_saved_model")
new_model, created from loading a saved model, is an internal TensorFlow user object without any of the class knowledge. It is not of type SequentialModule.
isinstance(new_model, SequentialModule)
False
This new model works on the already-defined input signatures. You can’t add more signatures to a model restored like this.
print(my_model([[2.0, 2.0, 2.0]]))
print(my_model([[[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]))
tf.Tensor([[0. 0.]], shape=(1, 2), dtype=float32)
tf.Tensor(
[[[0. 0.]
[0. 0.]]], shape=(1, 2, 2), dtype=float32)
Thus, using SavedModel, you are able to save TensorFlow weights and graphs using tf.Module, and then load them again.
5.3. Keras models and layers¶
剛剛教了用
tf.Module來建立 layer 和 model這邊開始，要來看 Keras 是怎麼用
tf.Module的
5.3.1. Keras layers¶
tf.keras.layers.Layer是 Keras 的 layer 的 base class，他繼承自tf.Module.所以，我們剛剛是在
tf.Module的基礎下，建立自己的 layer。現在，可以在tf.keras.layers.Layer的基礎下，建立 layer。那這樣的好處是，就可以繼承更多 keras 的 layer 所擁有的 attribute, methods，使得，未來用 keras 的 fit, compile 等功能時，他去吃你定義的 dense，都能取得他預期要拿到的東西
換句話說，如果你後續想用 keras 的 compile, fit 等功能，那你的 layer，必須繼承 keras layer 的 class，而不是
tf.Module.我們來看已下範例，寫法基本上和剛剛沒差別，只有
__call__要改成call，因為 keras 自己有定義 call method
class MyDense(tf.keras.layers.Layer):
# Adding **kwargs to support base Keras layer arguments
def __init__(self, in_features, out_features, **kwargs):
super().__init__(**kwargs)
# This will soon move to the build step; see below
self.w = tf.Variable(
tf.random.normal([in_features, out_features]), name='w')
self.b = tf.Variable(tf.zeros([out_features]), name='b')
def call(self, x):
y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)
simple_layer = MyDense(name="simple", in_features=3, out_features=3)
Keras layers have their own __call__ that does some bookkeeping described in the next section and then calls call(). You should notice no change in functionality.
simple_layer([[2.0, 2.0, 2.0]])
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=array([[0. , 0. , 6.451925]], dtype=float32)>
5.3.2. The build step¶
再來，是 keras 的 layer 寫法中，很重要的
buildstep.還記得前面用
tf.Module來寫 layer 時，為了不要每次都定義 input_feature，所以會先寫一個self.is_built = False來說明，目前還沒被 build。然後當 input tensor 進來後，才定義 w 和 b 的 shape，並把self.is_built改為 true，表示已經 build 完那，keras 這邊，就直接定義一個 method 叫 build，就是直接用來取 input_feature 用的
buildis called exactly once, and it is called with the shape of the input. It’s usually used to create variables (weights).我們來改寫一下剛剛的 layer
class FlexibleDense(tf.keras.layers.Layer):
# Note the added `**kwargs`, as Keras supports many arguments
def __init__(self, out_features, **kwargs):
super().__init__(**kwargs)
self.out_features = out_features
def build(self, input_shape): # Create the state of the layer (weights)
self.w = tf.Variable(
tf.random.normal([input_shape[-1], self.out_features]), name='w')
self.b = tf.Variable(tf.zeros([self.out_features]), name='b')
def call(self, inputs): # Defines the computation from inputs to outputs
return tf.matmul(inputs, self.w) + self.b
上面的寫法，有幾個重點要注意：
__init__裡面，多了**kwargs**，因為，繼承自tf.keras.layers.Layer時，他已經 support 很多其他的 input arguments 了.build(self, input_shape)這個 method中，input_shape 是 keras 會自動幫你讀出 input tensor 的 shape。所以下面就會用 input_shape[-1] 來當作 input_feature 數。build這個 method，你不會真的拿來用，他就是一個輔助 method，當你第一次把 input tensor 丟進去時，他會自己啟用
接著，我們實例化這個 class
# Create the instance of the layer
flexible_dense = FlexibleDense(out_features=3)
目前為止，我們還沒丟 input tensor 進去，所以 build 方法還沒被啟用，這時候，我們的 layer，還沒追蹤到任何 variable:
flexible_dense.variables
[]
一旦我丟一個 input tensor 進去後，就會啟用 build 方法，weight 就被 initialize 了：
# Call it, with predictably random results
print("Model results:", flexible_dense(tf.constant([[2.0, 2.0, 2.0], [3.0, 3.0, 3.0]])))
Model results: tf.Tensor(
[[-0.81454504 -4.95683 1.3514652 ]
[-1.2218176 -7.435245 2.027198 ]], shape=(2, 3), dtype=float32)
flexible_dense.variables
[<tf.Variable 'flexible_dense_1/w:0' shape=(3, 3) dtype=float32, numpy=
array([[-0.74692434, -1.0260396 , 0.7583028 ],
[ 0.05714832, -0.89261234, 0.0625408 ],
[ 0.2825035 , -0.5597631 , -0.14511095]], dtype=float32)>,
<tf.Variable 'flexible_dense_1/b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>]
由於
buildmethod 只會被 call 一次，所以如果之後 input 的 tensor，shape 和 起始話的時候不同，那就會報 error
try:
print("Model results:", flexible_dense(tf.constant([[2.0, 2.0, 2.0, 2.0]])))
except tf.errors.InvalidArgumentError as e:
print("Failed:", e)
Failed: Matrix size-incompatible: In[0]: [1,4], In[1]: [3,3] [Op:MatMul]
5.3.3. Keras models¶
前面在定義 layer 和 model 時，都是繼承自
tf.Module這個 class但當我們要定義 custom keras layer 時，我們會繼承自
tf.keras.layers.Layer(此 class 繼承自tf.Module)，這樣就能保有很多 keras layer 的好的特性同樣的，當我們要定義 custom keras model 時，我們也會繼承一個 keras 的 class，就是
tf.keras.Model(此 class 繼承自tf.keras.layers.Layer)這樣做的好處是，我們定義好的 Model，可以輕易的被 used, nested, and saved in the same way as Keras layers.
而且，還會有許多 extra functionality that makes them easy to train, evaluate, load, save, and even train on multiple machines.
來寫吧：
class MySequentialModel(tf.keras.Model):
def __init__(self, name=None, **kwargs):
super().__init__(**kwargs)
self.dense_1 = FlexibleDense(out_features=3)
self.dense_2 = FlexibleDense(out_features=2)
def call(self, x):
x = self.dense_1(x)
return self.dense_2(x)
可以看到，比剛剛寫 layer 簡單，因為不用寫
buildmethod。這部分在定義 custom layer 時已經做完了。只要單純的__init__和call就好實例化這個 model，一樣的，一開始找不到 weight，因位還沒有 tensor 被餵進來，還不知道 input shape，就不會 initialze weights
# You have made a Keras model!
my_sequential_model = MySequentialModel(name="the_model")
my_sequential_model.variables
[]
丟個 input tensor 進去，就可以看到結果，以及 variables 了
# Call it on a tensor, with random results
print("Model results:", my_sequential_model(tf.constant([[2.0, 2.0, 2.0]])))
Model results: tf.Tensor([[-2.5422215 2.2206373]], shape=(1, 2), dtype=float32)
my_sequential_model.variables
[<tf.Variable 'my_sequential_model/flexible_dense_2/w:0' shape=(3, 3) dtype=float32, numpy=
array([[-0.31155884, 0.35502377, 0.13043131],
[ 0.2252853 , -0.0766846 , -1.9106293 ],
[ 0.2285221 , 1.0349969 , 0.52284014]], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_2/b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_3/w:0' shape=(3, 2) dtype=float32, numpy=
array([[-0.8085942 , 0.611606 ],
[-1.0512786 , 0.78703386],
[-0.17862263, 0.00820879]], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_3/b:0' shape=(2,) dtype=float32, numpy=array([0., 0.], dtype=float32)>]
my_sequential_model.submodules
(<__main__.FlexibleDense at 0x142e349a0>,
<__main__.FlexibleDense at 0x13692cfa0>)
my_sequential_model.summary()
Model: "my_sequential_model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flexible_dense_2 (FlexibleDe multiple 12
_________________________________________________________________
flexible_dense_3 (FlexibleDe multiple 8
=================================================================
Total params: 20
Trainable params: 20
Non-trainable params: 0
_________________________________________________________________
5.3.4. functional API¶
另外一種建立 model 的方式，是直接用 functional API，這不僅可以幫我們減少一些時間，也可以獲得一些額外的好處 (e.g. model.summary()時，可發現 output shape 都跑出來了)
functional API 和剛剛 subclass 的寫法，最大差別在，你要先定義 input 的 shape (by
tf.keras.Input(shape = [3,]))The
input_shapeargument in this case does not have to be completely specified; you can leave some dimensions asNone.Note: You do not need to specify
input_shapeor anInputLayerin a subclassed model; these arguments and layers will be ignored.
inputs = tf.keras.Input(shape=[3,])
x = FlexibleDense(3)(inputs)
x = FlexibleDense(2)(x)
my_functional_model = tf.keras.Model(inputs=inputs, outputs=x)
my_functional_model.summary()
Model: "functional_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 3)] 0
_________________________________________________________________
flexible_dense_4 (FlexibleDe (None, 3) 12
_________________________________________________________________
flexible_dense_5 (FlexibleDe (None, 2) 8
=================================================================
Total params: 20
Trainable params: 20
Non-trainable params: 0
_________________________________________________________________
my_functional_model(tf.constant([[2.0, 2.0, 2.0]]))
<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[-0.3817817, -3.4696531]], dtype=float32)>
5.4. Saving Keras models¶
Keras models can be checkpointed, and that will look the same as tf.Module.
Keras models can also be saved with tf.saved_model.save(), as they are modules. However, Keras models have convenience methods and other functionality:
my_sequential_model.save("exname_of_file")
WARNING:tensorflow:From /Users/hanklee/.pyenv/versions/3.8.0/lib/python3.8/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
WARNING:tensorflow:From /Users/hanklee/.pyenv/versions/3.8.0/lib/python3.8/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
INFO:tensorflow:Assets written to: exname_of_file/assets
Just as easily, they can be loaded back in:
reconstructed_model = tf.keras.models.load_model("exname_of_file")
WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
Keras SavedModels also save metric, loss, and optimizer states.
This reconstructed model can be used and will produce the same result when called on the same data:
reconstructed_model(tf.constant([[2.0, 2.0, 2.0]]))
<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[-2.5422215, 2.2206373]], dtype=float32)>
There is more to know about saving and serialization of Keras models, including providing configuration methods for custom layers for feature support. Check out the guide to saving and serialization.
6. What’s next¶
If you want to know more details about Keras, you can follow the existing Keras guides here.
Another example of a high-level API built on tf.module is Sonnet from DeepMind, which is covered on their site.
6.1. 待完成內容：¶
Keras layers have a lot more extra features including:
Optional losses
Support for metrics
Built-in support for an optional
trainingargument to differentiate between training and inference useget_configandfrom_configmethods that allow you to accurately store configurations to allow model cloning in Python
Read about them in the full guide to custom layers and models.