tensorlayer学习日志3_chapter2_2.6.3

最新推荐文章于 2020-10-29 14:48:57 发布

zouqiuyi

最新推荐文章于 2020-10-29 14:48:57 发布

阅读量1.1k

点赞数

分类专栏： tensorlayer tensorflow 文章标签： tensorlayer tensorflow

本文链接：https://blog.youkuaiyun.com/weixin_42025210/article/details/81545653

版权

tensorlayer 同时被 2 个专栏收录

18 篇文章

订阅专栏

tensorflow

18 篇文章

订阅专栏

第2.6.3，教材提供的示例在：https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_mlp_dropout2.py

与上一节2.6.2一样，我直接删了tl.logging.set_verbosity(tl.logging.DEBUG)这条，并直接把act=tf.identity定义了。

示例里是少了tl.layers.set_name_reuse(reuse)这行的，但教材是有的，如果这行没有的话是会报错:

Exception: Layer 'MLP/input' already exists, please choice other 'name' or reuse this layer

import time
import tensorflow as tf
import tensorlayer as tl

tf.logging.set_verbosity(tf.logging.DEBUG)
sess = tf.InteractiveSession()
X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 784))
x = tf.placeholder(tf.float32, shape=[None, 784], name='x')
y_ = tf.placeholder(tf.int64, shape=[None], name='y_')

def mlp(x, is_train=True, reuse=False):
    with tf.variable_scope("MLP", reuse=reuse):
        tl.layers.set_name_reuse(reuse)
        network = tl.layers.InputLayer(x, name='input')
        network = tl.layers.DropoutLayer(network, keep=0.8, is_fix=True, is_train=is_train, name='drop1')
        network = tl.layers.DenseLayer(network, n_units=800, act=tf.nn.relu, name='relu1')
        network = tl.layers.DropoutLayer(network, keep=0.5, is_fix=True, is_train=is_train, name='drop2')
        network = tl.layers.DenseLayer(network, n_units=800, act=tf.nn.relu, name='relu2')
        network = tl.layers.DropoutLayer(network, keep=0.5, is_fix=True, is_train=is_train, name='drop3')
        network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output')
    return network
        

net_train = mlp(x, is_train=True, reuse=False)
net_test = mlp(x, is_train=False, reuse=True)

y = net_train.outputs
cost = tl.cost.cross_entropy(y, y_, name='xentropy')

y2 = net_test.outputs
cost_test = tl.cost.cross_entropy(y2, y_, name='xentropy2')
correct_prediction = tf.equal(tf.argmax(y2, 1), y_)
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

train_params = tl.layers.get_variables_with_name('MLP', True, False)
train_op = tf.train.AdamOptimizer(learning_rate=0.0001).minimize(cost, var_list=train_params)

tl.layers.initialize_global_variables(sess)

n_epoch = 100
batch_size = 500
print_freq = 10

for epoch in range(n_epoch):
    start_time = time.time()
    for X_train_a, y_train_a in tl.iterate.minibatches(X_train, y_train, batch_size, shuffle=True):
        sess.run(train_op, feed_dict={x: X_train_a, y_: y_train_a})

    if epoch + 1 == 1 or (epoch + 1) % print_freq == 0:
        print("Epoch %d of %d took %fs" % (epoch + 1, n_epoch, time.time() - start_time))
        train_loss, train_acc, n_batch = 0, 0, 0
        for X_train_a, y_train_a in tl.iterate.minibatches(X_train, y_train, batch_size, shuffle=True):
            err, ac = sess.run([cost_test, acc], feed_dict={x: X_train_a, y_: y_train_a})
            train_loss += err
            train_acc += ac
            n_batch += 1
        print("   train loss: %f" % (train_loss / n_batch))
        print("   train acc: %f" % (train_acc / n_batch))
        val_loss, val_acc, n_batch = 0, 0, 0
        for X_val_a, y_val_a in tl.iterate.minibatches(X_val, y_val, batch_size, shuffle=True):
            err, ac = sess.run([cost_test, acc], feed_dict={x: X_val_a, y_: y_val_a})
            val_loss += err
            val_acc += ac
            n_batch += 1
        print("   val loss: %f" % (val_loss / n_batch))
        print("   val acc: %f" % (val_acc / n_batch))

print('Evaluation')
test_loss, test_acc, n_batch = 0, 0, 0
for X_test_a, y_test_a in tl.iterate.minibatches(X_test, y_test, batch_size, shuffle=True):
    err, ac = sess.run([cost_test, acc], feed_dict={x: X_test_a, y_: y_test_a})
    test_loss += err
    test_acc += ac
    n_batch += 1
print("   test loss: %f" % (test_loss / n_batch))
print("   test acc: %f" % (test_acc / n_batch))

运行好久~~2794.8s就是46分钟后。。。运行如下：

[TL] Load or Download MNIST > data\mnist
[TL] data\mnist\train-images-idx3-ubyte.gz
[TL] data\mnist\t10k-images-idx3-ubyte.gz
[TL] InputLayer  MLP/input: (?, 784)
[TL] DropoutLayer MLP/drop1: keep:0.800000 is_fix:True
[TL] DenseLayer  MLP/relu1: 800 relu
[TL] DropoutLayer MLP/drop2: keep:0.500000 is_fix:True
[TL] DenseLayer  MLP/relu2: 800 relu
[TL] DropoutLayer MLP/drop3: keep:0.500000 is_fix:True
[TL] DenseLayer  MLP/output: 10 identity
[TL] InputLayer  MLP/input: (?, 784)
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/relu1: 800 relu
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/relu2: 800 relu
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/output: 10 identity
[TL]   [*] geting variables with MLP
Epoch 1 of 100 took 27.410568s
   train loss: 0.624696
   train acc: 0.801320
   val loss: 0.558072
   val acc: 0.823700
Epoch 10 of 100 took 27.335564s
   train loss: 0.239704
   train acc: 0.931060
   val loss: 0.217166
   val acc: 0.938300
Epoch 20 of 100 took 26.756531s
   train loss: 0.169821
   train acc: 0.951940
   val loss: 0.159543
   val acc: 0.956200
Epoch 30 of 100 took 27.015545s
   train loss: 0.129604
   train acc: 0.963120
   val loss: 0.128502
   val acc: 0.964000
Epoch 40 of 100 took 26.892538s
   train loss: 0.103222
   train acc: 0.971300
   val loss: 0.109796
   val acc: 0.968700
Epoch 50 of 100 took 26.840535s
   train loss: 0.082205
   train acc: 0.977580
   val loss: 0.095634
   val acc: 0.973200
Epoch 60 of 100 took 27.062548s
   train loss: 0.067935
   train acc: 0.981800
   val loss: 0.086476
   val acc: 0.975500
Epoch 70 of 100 took 26.795533s
   train loss: 0.057200
   train acc: 0.984800
   val loss: 0.079900
   val acc: 0.977000
Epoch 80 of 100 took 26.509516s
   train loss: 0.047904
   train acc: 0.987560
   val loss: 0.074532
   val acc: 0.978100
Epoch 90 of 100 took 27.449570s
   train loss: 0.040132
   train acc: 0.989900
   val loss: 0.070099
   val acc: 0.979600
Epoch 100 of 100 took 27.296561s
   train loss: 0.034271
   train acc: 0.991280
   val loss: 0.067506
   val acc: 0.981000
Evaluation
   test loss: 0.065505
   test acc: 0.979700
[Finished in 2794.8s]

因为is_train=True是指启用droupout，所以net_train的MLP显示与net_test是不一样的

net_train = mlp(x, is_train=True, reuse=False)

# 这里的reuse必须选False,选True会报错如下：
ValueError: Variable MLP/relu1/W does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

net_test = mlp(x, is_train=False, reuse=True) 

# 这里的reuse必须选True,选False会报错如下：
Exception: Layer 'MLP/input' already exists, please choice other 'name' or reuse this layer

[TL] InputLayer  MLP/input: (?, 784)
[TL] DropoutLayer MLP/drop1: keep:0.800000 is_fix:True
[TL] DenseLayer  MLP/relu1: 800 relu
[TL] DropoutLayer MLP/drop2: keep:0.500000 is_fix:True
[TL] DenseLayer  MLP/relu2: 800 relu
[TL] DropoutLayer MLP/drop3: keep:0.500000 is_fix:True
[TL] DenseLayer  MLP/output: 10 identity

~~~~~~~~~~~上面的是net_train的，下面是net_test的~~~~~~~~

[TL] InputLayer  MLP/input: (?, 784)
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/relu1: 800 relu
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/relu2: 800 relu
[TL] skip DropoutLayer
[TL] DenseLayer  MLP/output: 10 identity

下面的net_test的dropout层全都是skip跳过的，没执行

然后我把这里的printable改为true,尝试运行一下，出来一点新东东：

train_params = tl.layers.get_variables_with_name('MLP', train_only=True, printable=True)

因为没有droupout层的数值，所以我想这应该是从MKL函数网络中取出来用的reuse参数

[TL]   [*] geting variables with MLP
[TL]   got   0: MLP/relu1/W:0     (784, 800)
[TL]   got   1: MLP/relu1/b:0     (800,)
[TL]   got   2: MLP/relu2/W:0     (800, 800)
[TL]   got   3: MLP/relu2/b:0     (800,)
[TL]   got   4: MLP/output/W:0    (800, 10)
[TL]   got   5: MLP/output/b:0    (10,)
[Cancelled]

好了，最后回头想想这节的本意，应该是对付有多个数据集要输入，但又同时可用到同个参数设置的网络，像是dropout层就可以有选择要不要用了。只要机器够强大，就可以一次性把这些多个数据集全搞了~~