深度神经网络模型解析：基于TensorFlow的多层感知机与Dropout实现

原创于 2025-06-02 09:04:46 发布 · 363 阅读

CC 4.0 BY-SA版权

深度神经网络模型解析：基于TensorFlow的多层感知机与Dropout实现

deeplearning-models A collection of various deep learning architectures, models, and tips 项目地址: https://gitcode.com/gh_mirrors/de/deeplearning-models

引言

在深度学习领域，多层感知机(MLP)是最基础也是最重要的神经网络架构之一。本文将深入探讨如何在TensorFlow中实现带有Dropout正则化的多层感知机模型，用于解决经典的MNIST手写数字分类问题。

Dropout技术原理

Dropout是一种强大的正则化技术，通过在训练过程中随机"丢弃"部分神经元，可以有效防止神经网络过拟合。

Dropout应用位置分析

对于使用ReLU激活函数的网络，Dropout可以有两种应用方式：

传统方式：线性层输出 -> ReLU激活 -> Dropout
优化方式：线性层输出 -> Dropout -> ReLU激活

这两种方式在使用ReLU时会产生相同的结果，但第二种方式在实现上可能更高效。让我们通过一个具体例子来说明：

假设某层输出为：[-1, -2, -3, 4, 5, 6]

方式1：

经过ReLU：[0, 0, 0, 4, 5, 6]
50% Dropout(假设丢弃2,4,6单元)：[0*2, 0, 0*2, 0, 5*2, 0] = [0, 0, 0, 0, 10, 0]

方式2：

50% Dropout(同样丢弃2,4,6单元)：[-1*2, 0, -3*2, 0, 5*2, 0]
经过ReLU：[0, 0, 0, 0, 10, 0]

可以看到，两种方式最终结果一致，但方式2在实现上可能更高效。

基础实现方案

1. 数据准备

首先加载MNIST数据集，该数据集包含60,000个训练样本和10,000个测试样本，每个样本是28x28像素的手写数字图像。

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("./", one_hot=True)

2. 模型参数设置

# 超参数
learning_rate = 0.1
training_epochs = 20
batch_size = 64
dropout_keep_proba = 0.5  # 保留神经元的概率

# 网络结构
n_hidden_1 = 128  # 第一隐藏层神经元数
n_hidden_2 = 256  # 第二隐藏层神经元数
n_input = 784     # 输入维度(28x28)
n_classes = 10    # 输出类别数

3. 网络架构定义

使用TensorFlow的低级API构建网络：

g = tf.Graph()
with g.as_default():
    # 定义占位符
    keep_proba = tf.placeholder(tf.float32, None, name='keep_proba')
    tf_x = tf.placeholder(tf.float32, [None, n_input], name='features')
    tf_y = tf.placeholder(tf.float32, [None, n_classes], name='targets')
    
    # 定义权重和偏置
    weights = {
        'h1': tf.Variable(tf.truncated_normal([n_input, n_hidden_1], stddev=0.1)),
        'h2': tf.Variable(tf.truncated_normal([n_hidden_1, n_hidden_2], stddev=0.1)),
        'out': tf.Variable(tf.truncated_normal([n_hidden_2, n_classes], stddev=0.1))
    }
    biases = {
        'b1': tf.Variable(tf.zeros([n_hidden_1])),
        'b2': tf.Variable(tf.zeros([n_hidden_2])),
        'out': tf.Variable(tf.zeros([n_classes])))
    }
    
    # 网络结构
    layer_1 = tf.add(tf.matmul(tf_x, weights['h1']), biases['b1'])
    layer_1 = tf.nn.relu(layer_1)
    layer_1 = tf.nn.dropout(layer_1, keep_prob=keep_proba)
    
    layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
    layer_2 = tf.nn.relu(layer_2)
    layer_2 = tf.nn.dropout(layer_2, keep_prob=keep_proba)
    
    out_layer = tf.add(tf.matmul(layer_2, weights['out']), biases['out'], name='logits')
    
    # 损失函数和优化器
    loss = tf.nn.softmax_cross_entropy_with_logits(logits=out_layer, labels=tf_y)
    cost = tf.reduce_mean(loss, name='cost')
    optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
    train = optimizer.minimize(cost, name='train')
    
    # 准确率计算
    correct_prediction = tf.equal(tf.argmax(tf_y, 1), tf.argmax(out_layer, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name='accuracy')

4. 模型训练与评估

with tf.Session(graph=g) as sess:
    sess.run(tf.global_variables_initializer())
    
    for epoch in range(training_epochs):
        avg_cost = 0.
        total_batch = mnist.train.num_examples // batch_size
        
        for i in range(total_batch):
            batch_x, batch_y = mnist.train.next_batch(batch_size)
            _, c = sess.run(['train', 'cost:0'], 
                          feed_dict={'features:0': batch_x,
                                    'targets:0': batch_y,
                                    'keep_proba:0': dropout_keep_proba})
            avg_cost += c
        
        # 计算训练集和验证集准确率
        train_acc = sess.run('accuracy:0', 
                           feed_dict={'features:0': mnist.train.images,
                                      'targets:0': mnist.train.labels,
                                      'keep_proba:0': 1.0})
        valid_acc = sess.run('accuracy:0', 
                           feed_dict={'features:0': mnist.validation.images,
                                      'targets:0': mnist.validation.labels,
                                      'keep_proba:0': 1.0})
        
        print(f"Epoch: {epoch+1:03d} | AvgCost: {avg_cost/(i+1):.3f} | Train/Valid ACC: {train_acc:.3f}/{valid_acc:.3f}")
    
    # 最终测试集评估
    test_acc = sess.run(accuracy, 
                       feed_dict={'features:0': mnist.test.images,
                                  'targets:0': mnist.test.labels,
                                  'keep_proba:0': 1.0})
    print(f'Test ACC: {test_acc:.3f}')

高级API实现方案

TensorFlow提供了更高级的tf.layers API，可以简化网络构建过程。

关键区别

使用tf.layers.dense替代手动定义权重和偏置
Dropout参数使用"丢弃率"(dropout_rate)而非"保留概率"(keep_prob)
通过training标志控制Dropout是否生效

实现代码

g = tf.Graph()
with g.as_default():
    # 定义占位符
    is_training = tf.placeholder(tf.bool, name='is_training')
    tf_x = tf.placeholder(tf.float32, [None, n_input], name='features')
    tf_y = tf.placeholder(tf.float32, [None, n_classes], name='targets')
    
    # 使用tf.layers构建网络
    layer_1 = tf.layers.dense(tf_x, n_hidden_1, activation=tf.nn.relu,
                             kernel_initializer=tf.truncated_normal_initializer(stddev=0.1))
    layer_1 = tf.layers.dropout(layer_1, rate=dropout_rate, training=is_training)
    
    layer_2 = tf.layers.dense(layer_1, n_hidden_2, activation=tf.nn.relu,
                             kernel_initializer=tf.truncated_normal_initializer(stddev=0.1))
    layer_2 = tf.layers.dropout(layer_2, rate=dropout_rate, training=is_training)
    
    out_layer = tf.layers.dense(layer_2, n_classes, activation=None, name='logits')
    
    # 损失函数和优化器
    loss = tf.nn.softmax_cross_entropy_with_logits(logits=out_layer, labels=tf_y)
    cost = tf.reduce_mean(loss, name='cost')
    optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
    train = optimizer.minimize(cost, name='train')
    
    # 准确率计算
    correct_prediction = tf.equal(tf.argmax(tf_y, 1), tf.argmax(out_layer, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name='accuracy')