(三)Tensorflow神经网络函数:加和矩阵内积

import tensorflow as tf

1 tf.nn.bias_add()

【tf.nn.bias_add】
功能:二维矩阵a的行与矩阵b的列相加;三维矩阵，a的第三维度，b的行相加，实现偏置共享；

1.0 二维矩阵

【Demo】

import tensorflow as tf

# 矩阵维度:2x3
data = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度:3x1
bias = tf.constant([1, 2, 3])
# 矩阵维度:2x3
bias_add = tf.nn.bias_add(data, bias)

with tf.Session() as sess:
    data, bias,  bias_add= sess.run([data, bias, bias_add])
    print("data: {}".format(data))
    print("bias: {}".format(bias))
    print("bias add: {}".format(bias_add))

【Result】

data: [[1 2 3]
 [4 5 6]]
bias: [1 2 3]
bias add: [[2 4 6]
 [5 7 9]]

1.2 三维矩阵

【Demo】

import tensorflow as tf
import numpy as np

data = tf.constant([[[1, 2], [3, 4], [5, 6]], [[7, 8], [9, 10], [11, 12]]], dtype=tf.float64)
bias = tf.constant([1, 2], dtype=tf.float64)
bias_add = tf.nn.bias_add(data, bias)

with tf.Session() as sess:
    data, bias, bias_add = sess.run([data, bias, bias_add])
    print("Shape of data: {}".format(data.shape))
    print("Shape of bias: {}".format(bias.shape))
    
    print("Data value: {}".format(data))
    print("Bias value: {}".format(bias))
    print("Data add bias: {}".format(bias_add))

【Result】

Shape of data: (2, 3, 2)
Shape of bias: (2,)
Data value: [[[ 1.  2.]
  [ 3.  4.]
  [ 5.  6.]]

 [[ 7.  8.]
  [ 9. 10.]
  [11. 12.]]]
Bias value: [1. 2.]
Data add bias: [[[ 2.  4.]
  [ 4.  6.]
  [ 6.  8.]]

 [[ 8. 10.]
  [10. 12.]
  [12. 14.]]]

矩阵a维度：2x3x2，第三维度为2；
矩阵b维度：2x？，行为2；
矩阵a的第三维度与矩阵b的行相同，矩阵b的每个元素与矩阵a的第一和第二维度元素分别相加，神经网络里的偏置共享。

2 tf.add()

【tf.add】
功能:矩阵a的行与矩阵b的列相加;
【Demo】

import tensorflow as tf

# 矩阵维度:2x3
data = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度:3x1
bias = tf.constant([1, 2, 3])
# 矩阵维度:2x3
add_func = tf.add(data, bias)

with tf.Session() as sess:
    data, bias,  add_func= sess.run([data, bias, add_func])
    print("data: {}".format(data))
    print("bias: {}".format(bias))
    print("fucntion add: {}".format(add_func)) 

【Result】

data: (2, 3)
bias: (3,)
data: [[1 2 3]
 [4 5 6]]
bias: [1 2 3]
function add: [[2 4 6]
 [5 7 9]]

3 tf.add_n()

【tf.add_n】
功能:矩阵a的行与矩阵b的列相加;
【Demo】

import tensorflow as tf
# 矩阵维度:2x3
data = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度:3x1
bias = tf.constant([1, 2, 3])
# 矩阵维度:2x3
add_n = tf.add_n([data, bias])

with tf.Session() as sess:
    data, bias,  add_n= sess.run([data, bias, add_n])
    print("data: {}".format(data))
    print("bias: {}".format(bias))
    print("n add: {}".format(add_n))

【Result】

data: [[1 2 3]
 [4 5 6]]
bias: [1 2 3]
n add: [[ 2  4  6]
 [ 8 10 12]]

4 tf.nn.relu_layer()

【tf.nn.relu_layer】
功能:矩阵a1和矩阵a2内积加上偏置b；
行+列
$$y=x\ast weights+biases$$
其中，若$x\ast weights$为[10,30]则$biases$为[30,1]，biases行数为内积矩阵的列数。
【Demo】

import tensorflow as tf

# 矩阵维度2x3
data_2 = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度3x2
data_3 = tf.constant([[1, 2], [3, 4], [5, 6]])
# 矩阵维度2x?
bias_2 = tf.constant([1, 2])
# 矩阵维度2x2
relu_layer = tf.nn.relu_layer(data_2, data_3, bias_2)

with tf.Session() as sess:
    data_2, data_3, bias_2, relu_layer = sess.run([data_2, data_3, bias_2, relu_layer])
    print("data2 : {}".format(data_2))
    print("data3 : {}".format(data_3))
    print("bias2 : {}".format(bias_2))
    print("relu layer: {}".format(relu_layer))

【Result】

data2 : [[1 2 3]
 [4 5 6]]
data3 : [[1 2]
 [3 4]
 [5 6]]
bias2 : [1 2]
relu layer: [[23 30]
 [50 66]]

5 直接相加+

功能:矩阵a的行与矩阵b的列相加;
【Demo】

import tensorflow as tf

# 矩阵维度:2x3
data = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度:3x1
bias = tf.constant([1, 2, 3])
# 矩阵维度:2x3
direct_add = data + bias

with tf.Session() as sess:
    data, bias,  direct_add= sess.run([data, bias, direct_add])
    print("data: {}".format(data))
    print("bias: {}".format(bias))
    print("direct add: {}".format(direct_add)) 

【Result】

data: (2, 3)
bias: (3,)
data: [[1 2 3]
 [4 5 6]]
bias: [1 2 3]
direct add: [[2 4 6]
 [5 7 9]]

6 tf.matmul()

【tf.matmul】
功能:矩阵a和b内积.
$$y=mat1\ast mat2$$
【Demo】

import tensorflow as tf
# 矩阵维度2x3
data_2 = tf.constant([[1, 2, 3], [4, 5, 6]])
# 矩阵维度3x2
data_3 = tf.constant([[1, 2], [3, 4], [5, 6]])
# 矩阵维度2x2
matmul = tf.matmul(data_2, data_3)

with tf.Session() as sess_1:
    data_2, data_3, matmul = sess_1.run([data_2, data_3, matmul])
    print("data2 : {}".format(data_2))
    print("data3 : {}".format(data_3))
    print("matmul : {}".format(matmul))

【Result】

data2 : [[1 2 3]
 [4 5 6]]
data3 : [[1 2]
 [3 4]
 [5 6]]
bias2 : [1 2]
matmul : [[22 28]
 [49 64]]

7 总结

• Tensorflow矩阵加法tf.nn.bias_add和tf.add_n,tf.add及直接使用加号"+"功能相同,均为矩阵a的行与矩阵b的列相加,即a矩阵的列与矩阵b的行维度相同;
• tf.nn.relu_layer先计算矩阵内积,功能与tf.matmul相同,然后使用tf特有的加法,功能与tf.nn.add_n, tf.nn.bias_add相同.

【参考文献】
[1]https://tensorflow.google.cn/api_docs/python/
[2]https://blog.youkuaiyun.com/mieleizhi0522/article/details/80416668
[3]https://tensorflow.google.cn/versions/r1.12/api_docs/python/tf/nn/relu_layer

---

持续更新ing