8、神经网络参数初始化与优化全解析

神经网络参数初始化与优化全解析

1. 网络参数初始化方法

1.1 零初始化

零初始化时，所有神经元的权重都被设为零。在这种情况下，模型无法找到决策边界。以下是相关代码示例：

x_min <- min(testX[, 1]) - 0.2
x_max <- max(testX[, 1]) + 0.2
y_min <- min(testX[, 2]) - 0.2
y_max <- max(testX[, 2]) + 0.2
grid <- as.matrix(expand.grid(seq(x_min, x_max, by = step), seq(y_min,
y_max, by = step)))
Z <- predict_model(init_zero$parameters, t(grid), hidden_layer_act = "relu",
output_layer_act = "sigmoid")
Z <- ifelse(Z == 0, 1, 2)
g2 <- ggplot() + geom_tile(aes(x = grid[, 1], y = grid[, 2],
fill = as.character(Z)), alpha = 0.3, show.legend = F) +
geom_point(data = test_data, aes(x = x, y = y, color = as.character(testY)),
size = 1) + theme_bw(base_size = 15) + coord_fixed(ratio = 0.8) +
them