Handwritten numeral recognition & Tensorboard

Tensorboard&Tensorflow

As the most popular functional parts, Tensorflow and Tensorboard have a strong power to build and visualize your neural network which make your project much more understandable. Today, I am trying to explain the usage of Tensorboard and put the construction of neural network into practice.

Tensorboard

`Our ideal network:

The network visualized by tensorboard:

In this flow chart, all the data are measured by ‘tensor’, a special data type in tensorflow. Therefore we can clearly see the flow of those tensors.

Simple network

import Database

We will use the database handwritten numeral recognition in ‘mnist’ as our ‘Train’ and ‘Test’ data.

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets(r"C:\Users\ghx\PycharmProjects\untitled\.idea\mnist", one_hot=True)

visualize the unprocessed data:

Define the neural layer

Our basic thought is to create a function and deliver the shape parameter and specific inputs to return us the outputs. Here comes the code(the structural frames in tensorboard are also added ):

def add_layer(inputs,in_size,out_size,activation_function=None):
    with tf.name_scope('input_layer'):
        # add input layer and return output layer
        with tf.name_scope('weights'):
            Weights = tf.Variable(tf.random_normal([in_size,out_size]))
            tf.summary.histogram('/weight',Weights)
        with tf.name_scope('biases'):
            biases = tf.Variable(tf.zeros([1,out_size]))+ 0.1
            tf.summary.histogram('/biase', biases)
        with tf.name_scope('Wx_plus_b'):
            Wx_plus_b = tf.matmul(inputs,Weights) + biases
        if activation_function is None:
            outputs = Wx_plus_b
        else:
            outputs =activation_function(Wx_plus_b)
        tf.summary.histogram('/Outputs', outputs)
        return outputs

Cross_entropy&Optimizer

In this project we focus on how to identify the number behind the handwritten pictures, meaning that above all it’s a classification qusetion. Hence, we use cross_entropy as the optimization target.

with tf.name_scope('cross_entropy'):
    # error between prediction and training label
    cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction),reduction_indices=[1]))#loss
    tf.summary.scalar('cross_entropy',cross_entropy)

with tf.name_scope('train_step'):
    # train method -> GradiantDescent
    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

Train the network and caculate the accuracy

def compute_accuracy(v_xs, v_ys):
    global prediction
    with tf.name_scope('accuracy'):
        y_pre = sess.run(prediction,feed_dict={xs: v_xs})
        with tf.name_scope('correct_prediction'):
            # tf.arg_max(y_pre,1) the predicted label
            # judge whether v_pre and v_ys share the label
            correct_prediction = tf.equal(tf.arg_max(y_pre,1),tf.arg_max(v_ys, 1))
        with tf.name_scope('accuracy'):
            accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
        result = sess.run(accuracy,feed_dict={xs:v_xs, ys:v_ys})
        tf.summary.histogram('/result',result)
    return result

with tf.name_scope('inputs'):
sess = tf.Session()
merged = tf.summary.merge_all()
writer = tf.summary.FileWriter('number',sess.graph)
sess.run(tf.global_variables_initializer())

for i in range(1000):
    batch_xs,batch_ys = mnist.train.next_batch(100)# SGD
    sess.run(train_step, feed_dict={xs: batch_xs, ys: batch_ys})
    if i % 50 == 0:
        print(compute_accuracy(mnist.test.images, mnist.test.labels))
        rs = sess.run(merged,feed_dict={xs: batch_xs, ys: batch_ys})
        writer.add_summary(rs,i)

Print the accuracy every 50 steps:
0.1826
0.6357
0.7335
0.7789
0.8061
0.8181
0.8313
0.8384
0.8453
0.8528
0.8539
0.8592
0.8614
0.8612
0.8645
0.8665
0.866
0.8721
0.8731
0.8737

CNN neural network

I am not going to explain the rationale of CNN as I am also a naive guy when dealing with these questions. I will just try to build the core parts of CNN which are the convolutional layer and pooling layer.

conv1 = tf.layers.conv2d(inputs=image,filter=16,kernel_size=5,stride=1,padding='same'
                        activation=tf.nn.relu)  #(28,28,16)
pooling1=tf.layers.max_poolong2d(conv1,pool_size=2,strides=2) #(14,14,16)
conv2=tf.layers.conv2d(pooling1,filter=32,5,1,'same',tf.nn.relu) #(14,14,32)
pooling2=tf.layers.max_poolong2d(conv2,2,2) #(7,7,32)
flat=tf.reshape(pooling2,[-1,7*7*32]) # 1568 cols
output=tf.layers.dense(flat,10)  
 
#loss
loss = tf.losses.softmax_cross_entropy(onehot_labels=tf_y, logits=output)           # compute cost
train_op = tf.train.AdamOptimizer(lr).minimize(loss)
 

every 100 steps print the train accuracy:

Finally, test the CNN network by passing the mnist.test into it:

The CNN network is inspired by those two blogs:
https://blog.youkuaiyun.com/greedystar/article/details/80444519
https://blog.youkuaiyun.com/xiexu911/article/details/88634970

The statement: All the materials are used to study python, if this article involves all infringement, please contact me to delete.