作业内容:实现一个两层的神经网络,包括FP和BP,在CIFAR-10上测试
计算score,score function用的是ReLU:max(0,W*X):
计算loss,用的是softmax+L2正则化:
用BP计算梯度:
利用SGD训练模型:
先是挑选mini batch:
更新:
最后精度达到29%。
很低,所以模型需要debug ,可以画出plot the loss function and the accuracies on the training and validation sets during optimization;visualize the weights that were learned in the first layer of the network。
有几点可以考虑:
1.hidden layer size,
2.learning rate,
3. numer of training epochs,
4.regularization strength.