使用fashion mnsit数据集
研究tensorflow 深度学习框架
fashion mnist 使用28x28的灰度图片
# -*- coding: utf-8 -*-
# TensorFlow and tf.keras
from com.tfmnist.demo.utils import mnist_reader
from tensorflow import keras
import tensorflow as tf
# Helper libraries
import numpy as np
import matplotlib.pyplot as plt
if __name__ == '__main__':
'''
matplotlib 是一个 Python 的 2D绘图库
image 和标签数据集
'''
train_images, train_labels = mnist_reader.load_mnist('data', kind='train')
test_images, test_labels = mnist_reader.load_mnist('data', kind='t10k')
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
print("train_images ", train_images.shape, " test_images", test_images.shape)
'''
预处理数据
使用2D绘图库来显示 image图片
'''
plt.figure()
plt.imshow(train_images[0])
plt.colorbar()
plt.grid(False)
'''
显示训练集中的前 25 张图像,并在每张图像下显示类别名称
'''
plt.figure(figsize=(10, 10))
for i in range(25):
plt.subplot(5, 5, i + 1)
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(train_images[i], cmap=plt.cm.binary)
plt.xlabel(class_names[train_labels[i]])
遇到的异常
- 在使用 matplotlib 库显示 numpy数组image图片时,出现了 TypeError: Invalid dimensions for image data异常
pyplot.imshow()方法显示二维数组的图片导致dimensions 非法数据维度,因为fashion mnist 使用28x28的灰度图片,查看train_images.shape

是一维数组的格式,需要转换为二维数组,然后使用imshow显示
image_temp=train_images[0].reshape(28, 28)
plt.imshow(image_temp)
使用numpy reshape()方法把数组维度设置为28x28的二维数组
转换完成之后,查看下image_temp的shape

然后运行出来的效果图如下

或者在mnist_reader.load_mnist 里面修改
with gzip.open(images_path, 'rb') as imgpath:
images = np.frombuffer(imgpath.read(), dtype=np.uint8, offset=16).reshape(len(labels), 28, 28)
把np.frombuffer(imgpath.read(), dtype=np.uint8, offset=16) , reshape 为三维数组reshape(len(labels), 28, 28)
- 在macbook os系统上运行相同的代码,图片无显示
需要在结尾再调用plt.show() 来显示图片,效果如下图

显示数据没有问题,继续下一步
预处理数据
我们将28,28的数组的像素值缩小到 0 到 1 之间,然后将其馈送到神经网络模型。为此,将图像组件的数据类型从整数转换为浮点数,然后除以 255。以下是预处理图像的函数
train_images = train_images / 255.0
test_images = test_images / 255.0
构建模型
构建神经网络需要先配置模型的层,然后再编译模型。
model = keras.Sequential([
keras.layers.Flatten(input_shape=(28, 28)),
keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(10, activation=tf.nn.softmax)
])
该网络中的第一层 tf.keras.layers.Flatten 将图像格式从二维数组(28x28 像素)转换成一维数组(28 * 28 = 784 像素)。可以将该层视为图像中像素未堆叠的行,并排列这些行。该层没有要学习的参数;它只改动数据的格式。
在扁平化像素之后,该网络包含两个 tf.keras.layers.Dense 层的序列。这些层是密集连接或全连接神经层。第一个 Dense 层具有 128 个节点(或神经元)。第二个(也是最后一个)层是具有 10 个节点的 softmax 层,该层会返回一个具有 10 个概率得分的数组,这些得分的总和为 1。每个节点包含一个得分,表示当前图像属于 10 个类别中某一个的概率。
编译模型
指定目标函数或者损失函数为 sparse_categorical_crossentropy,
损失函数对应的优化器(数学不好不懂这个算法),
和数据训练的指标为accuracy 精准度
model.compile(optimizer=tf.train.AdamOptimizer(),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
下一步训练模型
model.fit(train_images, train_labels, epochs=5)
调用fit进行训练,训练次数epochs设置为5次,下面是训练,5次训练的所有日志
Epoch 1/5
32/60000 [..............................] - ETA: 2:58 - loss: 2.5790 - acc: 0.0938
1184/60000 [..............................] - ETA: 7s - loss: 1.3121 - acc: 0.5591
2624/60000 [>.............................] - ETA: 4s - loss: 1.0030 - acc: 0.6562
4032/60000 [=>............................] - ETA: 3s - loss: 0.8903 - acc: 0.6949
5664/60000 [=>............................] - ETA: 2s - loss: 0.8147 - acc: 0.7228
7456/60000 [==>...........................] - ETA: 2s - loss: 0.7597 - acc: 0.7403
9344/60000 [===>..........................] - ETA: 2s - loss: 0.7349 - acc: 0.7480
11136/60000 [====>.........................] - ETA: 1s - loss: 0.7056 - acc: 0.7584
12864/60000 [=====>........................] - ETA: 1s - loss: 0.6844 - acc: 0.7654
14432/60000 [======>.......................] - ETA: 1s - loss: 0.6660 - acc: 0.7712
16128/60000 [=======>......................] - ETA: 1s - loss: 0.6460 - acc: 0.7777
17856/60000 [=======>......................] - ETA: 1s - loss: 0.6351 - acc: 0.7810
19680/60000 [========>.....................] - ETA: 1s - loss: 0.6213 - acc: 0.7863
21536/60000 [=========>....................] - ETA: 1s - loss: 0.6091 - acc: 0.7899
23392/60000 [==========>...................] - ETA: 1s - loss: 0.5992 - acc: 0.7932
24992/60000 [===========>..................] - ETA: 1s - loss: 0.5910 - acc: 0.7960
26816/60000 [============>.................] - ETA: 1s - loss: 0.5842 - acc: 0.7982
28672/60000 [=============>................] - ETA: 1s - loss: 0.5765 - acc: 0.8006
30464/60000 [==============>...............] - ETA: 0s - loss: 0.5687 - acc: 0.8028
32320/60000 [===============>..............] - ETA: 0s - loss: 0.5612 - acc: 0.8053
34112/60000 [================>.............] - ETA: 0s - loss: 0.5560 - acc: 0.8068
35904/60000 [================>.............] - ETA: 0s - loss: 0.5508 - acc: 0.8084
37728/60000 [=================>............] - ETA: 0s - loss: 0.5471 - acc: 0.8092
39552/60000 [==================>...........] - ETA: 0s - loss: 0.5414 - acc: 0.8112
41408/60000 [===================>..........] - ETA: 0s - loss: 0.5363 - acc: 0.8129
43168/60000 [====================>.........] - ETA: 0s - loss: 0.5319 - acc: 0.8144
44960/60000 [=====================>........] - ETA: 0s - loss: 0.5277 - acc: 0.8159
46816/60000 [======================>.......] - ETA: 0s - loss: 0.5239 - acc: 0.8166
48640/60000 [=======================>......] - ETA: 0s - loss: 0.5204 - acc: 0.8173
50464/60000 [========================>.....] - ETA: 0s - loss: 0.5168 - acc: 0.8185
52064/60000 [=========================>....] - ETA: 0s - loss: 0.5147 - acc: 0.8190
53888/60000 [=========================>....] - ETA: 0s - loss: 0.5116 - acc: 0.8199
55744/60000 [==========================>...] - ETA: 0s - loss: 0.5080 - acc: 0.8212
57536/60000 [===========================>..] - ETA: 0s - loss: 0.5064 - acc: 0.8216
59328/60000 [============================>.] - ETA: 0s - loss: 0.5037 - acc: 0.8225
60000/60000 [==============================] - 2s 31us/step - loss: 0.5035 - acc: 0.8226
Epoch 2/5
32/60000 [..............................] - ETA: 5s - loss: 0.5432 - acc: 0.8438
1920/60000 [..............................] - ETA: 1s - loss: 0.3944 - acc: 0.8594
3744/60000 [>.............................] - ETA: 1s - loss: 0.3801 - acc: 0.8691
5600/60000 [=>............................] - ETA: 1s - loss: 0.3834 - acc: 0.8662
7360/60000 [==>...........................] - ETA: 1s - loss: 0.3863 - acc: 0.8628
8832/60000 [===>..........................] - ETA: 1s - loss: 0.3855 - acc: 0.8628
10368/60000 [====>.........................] - ETA: 1s - loss: 0.3850 - acc: 0.8633
12288/60000 [=====>........................] - ETA: 1s - loss: 0.3800 - acc: 0.8639
14112/60000 [======>.......................] - ETA: 1s - loss: 0.3785 - acc: 0.8652
15968/60000 [======>.......................] - ETA: 1s - loss: 0.3769 - acc: 0.8657
17888/60000 [=======>......................] - ETA: 1s - loss: 0.3784 - acc: 0.8653
19744/60000 [========>.....................] - ETA: 1s - loss: 0.3828 - acc: 0.8637
21568/60000 [=========>....................] - ETA: 1s - loss: 0.3833 - acc: 0.8631
23520/60000 [==========>...................] - ETA: 1s - loss: 0.3824 - acc: 0.8637
25440/60000 [===========>..................] - ETA: 0s - loss: 0.3793 - acc: 0.8645
26848/60000 [============>.................] - ETA: 0s - loss: 0.3779 - acc: 0.8650
28448/60000 [=============>................] - ETA: 0s - loss: 0.3762 - acc: 0.8659
30144/60000 [==============>...............] - ETA: 0s - loss: 0.3777 - acc: 0.8653
31744/60000 [==============>...............] - ETA: 0s - loss: 0.3771 - acc: 0.8652
33632/60000 [===============>..............] - ETA: 0s - loss: 0.3769 - acc: 0.8656
35552/60000 [================>.............] - ETA: 0s - loss: 0.3780 - acc: 0.8653
37408/60000 [=================>............] - ETA: 0s - loss: 0.3778 - acc: 0.8653
39264/60000 [==================>...........] - ETA: 0s - loss: 0.3775 - acc: 0.8656
41184/60000 [===================>..........] - ETA: 0s - loss: 0.3771 - acc: 0.8653
43072/60000 [====================>.........] - ETA: 0s - loss: 0.3759 - acc: 0.8656
44864/60000 [=====================>........] - ETA: 0s - loss: 0.3759 - acc: 0.8653
46784/60000 [======================>.......] - ETA: 0s - loss: 0.3768 - acc: 0.8648
48736/60000 [=======================>......] - ETA: 0s - loss: 0.3760 - acc: 0.8655
50624/60000 [========================>.....] - ETA: 0s - loss: 0.3744 - acc: 0.8663
52480/60000 [=========================>....] - ETA: 0s - loss: 0.3734 - acc: 0.8667
54368/60000 [==========================>...] - ETA: 0s - loss: 0.3736 - acc: 0.8664
56224/60000 [===========================>..] - ETA: 0s - loss: 0.3744 - acc: 0.8661
58176/60000 [============================>.] - ETA: 0s - loss: 0.3740 - acc: 0.8659
60000/60000 [==============================] - 2s 28us/step - loss: 0.3737 - acc: 0.8658
Epoch 3/5
32/60000 [..............................] - ETA: 3s - loss: 0.2923 - acc: 0.9375
1920/60000 [..............................] - ETA: 1s - loss: 0.3462 - acc: 0.8755
3680/60000 [>.............................] - ETA: 1s - loss: 0.3449 - acc: 0.8799
5600/60000 [=>............................] - ETA: 1s - loss: 0.3356 - acc: 0.8788
7488/60000 [==>...........................] - ETA: 1s - loss: 0.3292 - acc: 0.8811
9280/60000 [===>..........................] - ETA: 1s - loss: 0.3294 - acc: 0.8807
11104/60000 [====>.........................] - ETA: 1s - loss: 0.3279 - acc: 0.8811
12960/60000 [=====>........................] - ETA: 1s - loss: 0.3331 - acc: 0.8820
14912/60000 [======>.......................] - ETA: 1s - loss: 0.3347 - acc: 0.8803
16800/60000 [=======>......................] - ETA: 1s - loss: 0.3335 - acc: 0.8804
18624/60000 [========>.....................] - ETA: 1s - loss: 0.3376 - acc: 0.8791
20576/60000 [=========>....................] - ETA: 1s - loss: 0.3361 - acc: 0.8790
22496/60000 [==========>...................] - ETA: 1s - loss: 0.3360 - acc: 0.8788
24320/60000 [===========>..................] - ETA: 0s - loss: 0.3359 - acc: 0.8786
26240/60000 [============>.................] - ETA: 0s - loss: 0.3343 - acc: 0.8792
28160/60000 [=============>................] - ETA: 0s - loss: 0.3335 - acc: 0.8793
30080/60000 [==============>...............] - ETA: 0s - loss: 0.3351 - acc: 0.8791
31904/60000 [==============>...............] - ETA: 0s - loss: 0.3350 - acc: 0.8793
33824/60000 [===============>..............] - ETA: 0s - loss: 0.3337 - acc: 0.8797
35744/60000 [================>.............] - ETA: 0s - loss: 0.3325 - acc: 0.8799
37696/60000 [=================>............] - ETA: 0s - loss: 0.3353 - acc: 0.8786
39584/60000 [==================>...........] - ETA: 0s - loss: 0.3367 - acc: 0.8781
41312/60000 [===================>..........] - ETA: 0s - loss: 0.3382 - acc: 0.8773
43200/60000 [====================>.........] - ETA: 0s - loss: 0.3382 - acc: 0.8775
45120/60000 [=====================>........] - ETA: 0s - loss: 0.3376 - acc: 0.8772
47072/60000 [======================>.......] - ETA: 0s - loss: 0.3369 - acc: 0.8777
48928/60000 [=======================>......] - ETA: 0s - loss: 0.3370 - acc: 0.8777
50816/60000 [========================>.....] - ETA: 0s - loss: 0.3366 - acc: 0.8775
52768/60000 [=========================>....] - ETA: 0s - loss: 0.3368 - acc: 0.8775
54688/60000 [==========================>...] - ETA: 0s - loss: 0.3374 - acc: 0.8773
56544/60000 [===========================>..] - ETA: 0s - loss: 0.3379 - acc: 0.8771
58464/60000 [============================>.] - ETA: 0s - loss: 0.3373 - acc: 0.8771
60000/60000 [==============================] - 2s 27us/step - loss: 0.3369 - acc: 0.8773
Epoch 4/5
32/60000 [..............................] - ETA: 3s - loss: 0.2904 - acc: 0.8750
1984/60000 [..............................] - ETA: 1s - loss: 0.3182 - acc: 0.8745
3840/60000 [>.............................] - ETA: 1s - loss: 0.3295 - acc: 0.8753
5632/60000 [=>............................] - ETA: 1s - loss: 0.3332 - acc: 0.8722
7552/60000 [==>...........................] - ETA: 1s - loss: 0.3290 - acc: 0.8770
9504/60000 [===>..........................] - ETA: 1s - loss: 0.3263 - acc: 0.8794
11456/60000 [====>.........................] - ETA: 1s - loss: 0.3223 - acc: 0.8818
13248/60000 [=====>........................] - ETA: 1s - loss: 0.3194 - acc: 0.8824
15168/60000 [======>.......................] - ETA: 1s - loss: 0.3173 - acc: 0.8832
17088/60000 [=======>......................] - ETA: 1s - loss: 0.3175 - acc: 0.8840
18848/60000 [========>.....................] - ETA: 1s - loss: 0.3173 - acc: 0.8834
20672/60000 [=========>....................] - ETA: 1s - loss: 0.3169 - acc: 0.8827
22624/60000 [==========>...................] - ETA: 1s - loss: 0.3155 - acc: 0.8828
24576/60000 [===========>..................] - ETA: 0s - loss: 0.3154 - acc: 0.8830
26496/60000 [============>.................] - ETA: 0s - loss: 0.3161 - acc: 0.8827
28448/60000 [=============>................] - ETA: 0s - loss: 0.3160 - acc: 0.8830
30304/60000 [==============>...............] - ETA: 0s - loss: 0.3159 - acc: 0.8831
32224/60000 [===============>..............] - ETA: 0s - loss: 0.3163 - acc: 0.8832
34208/60000 [================>.............] - ETA: 0s - loss: 0.3147 - acc: 0.8839
36160/60000 [=================>............] - ETA: 0s - loss: 0.3133 - acc: 0.8849
38048/60000 [==================>...........] - ETA: 0s - loss: 0.3132 - acc: 0.8848
40000/60000 [===================>..........] - ETA: 0s - loss: 0.3132 - acc: 0.8850
41920/60000 [===================>..........] - ETA: 0s - loss: 0.3139 - acc: 0.8851
43872/60000 [====================>.........] - ETA: 0s - loss: 0.3155 - acc: 0.8844
45760/60000 [=====================>........] - ETA: 0s - loss: 0.3157 - acc: 0.8842
47648/60000 [======================>.......] - ETA: 0s - loss: 0.3153 - acc: 0.8842
49600/60000 [=======================>......] - ETA: 0s - loss: 0.3160 - acc: 0.8840
51552/60000 [========================>.....] - ETA: 0s - loss: 0.3155 - acc: 0.8844
53504/60000 [=========================>....] - ETA: 0s - loss: 0.3148 - acc: 0.8847
55264/60000 [==========================>...] - ETA: 0s - loss: 0.3150 - acc: 0.8846
57120/60000 [===========================>..] - ETA: 0s - loss: 0.3153 - acc: 0.8843
59072/60000 [============================>.] - ETA: 0s - loss: 0.3145 - acc: 0.8845
60000/60000 [==============================] - 2s 27us/step - loss: 0.3143 - acc: 0.8845
Epoch 5/5
32/60000 [..............................] - ETA: 5s - loss: 0.3525 - acc: 0.8438
1888/60000 [..............................] - ETA: 1s - loss: 0.2724 - acc: 0.9062
3776/60000 [>.............................] - ETA: 1s - loss: 0.2815 - acc: 0.8986
5664/60000 [=>............................] - ETA: 1s - loss: 0.2959 - acc: 0.8948
7584/60000 [==>...........................] - ETA: 1s - loss: 0.2919 - acc: 0.8965
9536/60000 [===>..........................] - ETA: 1s - loss: 0.2890 - acc: 0.8983
11392/60000 [====>.........................] - ETA: 1s - loss: 0.2916 - acc: 0.8977
13312/60000 [=====>........................] - ETA: 1s - loss: 0.2934 - acc: 0.8963
14784/60000 [======>.......................] - ETA: 1s - loss: 0.2967 - acc: 0.8946
16672/60000 [=======>......................] - ETA: 1s - loss: 0.2957 - acc: 0.8950
18592/60000 [========>.....................] - ETA: 1s - loss: 0.2944 - acc: 0.8954
20512/60000 [=========>....................] - ETA: 1s - loss: 0.2965 - acc: 0.8939
22432/60000 [==========>...................] - ETA: 1s - loss: 0.2957 - acc: 0.8938
24352/60000 [===========>..................] - ETA: 0s - loss: 0.2959 - acc: 0.8942
26240/60000 [============>.................] - ETA: 0s - loss: 0.2942 - acc: 0.8946
28096/60000 [=============>................] - ETA: 0s - loss: 0.2949 - acc: 0.8941
30016/60000 [==============>...............] - ETA: 0s - loss: 0.2952 - acc: 0.8941
31872/60000 [==============>...............] - ETA: 0s - loss: 0.2942 - acc: 0.8941
33664/60000 [===============>..............] - ETA: 0s - loss: 0.2952 - acc: 0.8934
35488/60000 [================>.............] - ETA: 0s - loss: 0.2955 - acc: 0.8933
37376/60000 [=================>............] - ETA: 0s - loss: 0.2954 - acc: 0.8928
39296/60000 [==================>...........] - ETA: 0s - loss: 0.2953 - acc: 0.8925
41184/60000 [===================>..........] - ETA: 0s - loss: 0.2952 - acc: 0.8925
43040/60000 [====================>.........] - ETA: 0s - loss: 0.2944 - acc: 0.8930
44896/60000 [=====================>........] - ETA: 0s - loss: 0.2936 - acc: 0.8932
46816/60000 [======================>.......] - ETA: 0s - loss: 0.2946 - acc: 0.8929
48736/60000 [=======================>......] - ETA: 0s - loss: 0.2956 - acc: 0.8925
50528/60000 [========================>.....] - ETA: 0s - loss: 0.2953 - acc: 0.8928
52416/60000 [=========================>....] - ETA: 0s - loss: 0.2955 - acc: 0.8926
54336/60000 [==========================>...] - ETA: 0s - loss: 0.2954 - acc: 0.8928
56224/60000 [===========================>..] - ETA: 0s - loss: 0.2952 - acc: 0.8929
58048/60000 [============================>.] - ETA: 0s - loss: 0.2961 - acc: 0.8925
59936/60000 [============================>.] - ETA: 0s - loss: 0.2955 - acc: 0.8925
60000/60000 [==============================] - 2s 27us/step - loss: 0.2954 - acc: 0.8925
使用测试数据在模型进行测试
test_loss, test_acc = model.evaluate(test_images, test_labels)
print('Test accuracy:', test_acc)
32/10000 [..............................] - ETA: 9s
2848/10000 [=======>......................] - ETA: 0s
6112/10000 [=================>............] - ETA: 0s
9504/10000 [===========================>..] - ETA: 0s
10000/10000 [==============================] - 0s 19us/step
Test accuracy: 0.8776
0.8925 0.8776测试数据集结果小于训练结果,表示过拟合
551



被折叠的 条评论
为什么被折叠?



