Tensorflow（三十二） —— 梯度爆炸

解决方案

import tensorflow as tf
from tensorflow import keras

# solve gradient exploding
with tf.GradientTape() as tape:
    logits = model(x)
    loss = tf.losses.categorical_crossentropy(y,logits,from_logits=True)
grads = tape.gradient(loss,model.trainable_variables)
grads = [tf.clip_by_norm(g,15) for g in grads]

optimizer.apply_gradients(zip(grads,model.trainable_variables))

本文为参考龙龙老师的“深度学习与TensorFlow 2入门实战“课程书写的学习笔记

by CyrusMay 2022 04 16