第一个实验:用神经网络实现异或门
测试数据如下:
[0,0]->[0]
[1,0]->[1]
[1,1]->[0]
[0,1]->[1]
数据量很小,但异或门远没有我想象的好写,原来在第一层的时候我使用的激活函数是relu,第一层有两个神经元;输出层使用的激活函数是softmax;学习率是0.01;loss function选的是梯度下降算法,weights和biases都是随机产生的非零数据。
后来发现这样训练出来的神经网络的预测值是[0.5,0.5,0.5,0.5]。
最后参考了别人的程序,改成了如下配置**第一层有16个神经元(其实影响不大),第一层使用的激活函数是relu;输出层使用的激活函数是sigmoid;loss function选的是adam,这样修改之后就开始像准确值逼近了!
看来深度学习会设计才是最重要的。
下面附上代码:
import tensorflow as tf
learning_rate = 0.01
x_data = [[0.,0.],[1.,0.],[1.,1.],[0.,1.]]
x = tf.placeholder("float", shape = [None,2])
y_data = [0,1,0,1]
y = tf.placeholder("float",shape=[None,1])
weights = {
'w1':tf.Variable(tf.random_normal([2,16])),
'w2':tf.Variable(tf.random_normal([16,1]))
}
biases = {
'b1':tf.Variable(tf.random_normal([1])),
'b2':tf.Variable(tf.random_normal([1]))
}
def dnn(_X,_weights,_biases):
d1 = tf.matmul(_X, _weights['w1'])+_biases['b1']
d1 = tf.nn.relu(d1)
d2 = tf.matmul(d1,_weights['w2'])+_biases['b2']
d2 = tf.nn.sigmoid(d2)
return d2
pred = dnn(x, weights, biases)
cost = tf.reduce_mean(tf.square(y-pred))
optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost)
correct_pred = tf.equal(tf.argmax(pred,1),tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred,tf.float32))
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
step = 1
for _ in range(1500):
batch_xs = tf.reshape(x_data,shape=[-1,2])
batch_ys = tf.reshape(y_data,shape=[-1,1])
#print(batch_xs)
#print(batch_ys)
sess.run(optimizer,feed_dict={x:sess.run(batch_xs),y:sess.run(batch_ys)})
acc = sess.run(accuracy,feed_dict={x:sess.run(batch_xs),y:sess.run(batch_ys)})
loss = sess.run(cost,feed_dict = {x:sess.run(batch_xs),y:sess.run(batch_ys)})
#print("Step "+str(step)+",Minibatch Loss = "+"{:.6f}".format(loss)+", Training Accuracy = "+"{:.5f}".format(acc))
step += 1
if(step%100==0):
print("Step "+str(step)+" loss "+"{:.6f}".format(loss))
print(sess.run(pred,feed_dict={x:sess.run(batch_xs)}))
# print(sess.run(weights))
# print(sess.run(biases))
print("Optimization Finished!")