[python]如何自己跟踪训练过程中的损失值并可视化出来

1 跟踪训练函数的损失值
如果跑的代码中没有显示损失函数的变化趋势，而自己需要根据这个来调整超参，那么可以自己编写函数去实现这一个需求。
首先应该将每次的损失值记录并储存下来。这里以CornerNet的代码为例，代码链接如下：
CornerNet
在train.py函数中，损失值在如下的部分得到：

with stdout_to_tqdm() as save_stdout:
    for iteration in tqdm(range(start_iter + 1, max_iteration + 1), file=save_stdout, ncols=80):
        training = pinned_training_queue.get(block=True)
        training_loss = nnet.train(**training)

        if display and iteration % display == 0:
            print("training loss at iteration {}: {}".format(iteration, training_loss.item()))
        del training_loss

        # if val_iter and validation_db.db_inds.size and iteration % val_iter == 0:
        #     nnet.eval_mode()
        #     validation = pinned_validation_queue.get(block=True)
        #     validation_loss = nnet.validate(**validation)
        #     print("validation loss at iteration {}: {}".format(iteration, validation_loss.item()))
        #     nnet.train_mode()

        if iteration % snapshot == 0:
            nnet.save_params(iteration)

        if iteration % stepsize == 0:
            learning_rate /= decay_rate
            nnet.set_lr(learning_rate)

如果想要记录并储存好其值，加入以下代码即可：

with stdout_to_tqdm() as save_stdout:
    for iteration in tqdm(range(start_iter + 1, max_iteration + 1), file=save_stdout, ncols=80):
        training = pinned_training_queue.get(block=True)
        training_loss = nnet.train(**training)
        loss = training_loss.cpu()
        loss_ = str(loss.data.numpy())
        with open('path', 'a') as f:
            f.write(str(iteration))
            f.write(' ')
            f.write(loss_)
            if iteration < max_iteration:
                f.write(' \r\n')
        if display and iteration % display == 0:
            print("training loss at iteration {}: {}".format(iteration, training_loss.item()))
        del training_loss

        # if val_iter and validation_db.db_inds.size and iteration % val_iter == 0:
        #     nnet.eval_mode()
        #     validation = pinned_validation_queue.get(block=True)
        #     validation_loss = nnet.validate(**validation)
        #     print("validation loss at iteration {}: {}".format(iteration, validation_loss.item()))
        #     nnet.train_mode()

        if iteration % snapshot == 0:
            nnet.save_params(iteration)

        if iteration % stepsize == 0:
            learning_rate /= decay_rate
            nnet.set_lr(learning_rate)

加入的部分为：

        loss = training_loss.cpu()
        loss_ = str(loss.data.numpy())
        with open('path', 'a') as f:
            f.write(str(iteration))
            f.write(' ')
            f.write(loss_)
            if iteration < max_iteration:
                f.write(' \r\n')

解释一下代码
由于深度学习计算loss的时候基本上loss都是cuda的一个tensor变量，储存在cuda中的。是不能够直接复制过来的，所以需要用.cpu()把cuda中的值转移给cpu中储存
然后由于此时转移过去以后还是一个variable变量（就是可以backpropogation来计算grad的一种变量），所以需要用variable.data把其中的数据单独取出来，但是此时还是一个tensor，需要转换成numpy，所以再通过一个.numpy()
f.write（）中的必须是一个string类型的数据，所以要转成string
最后if语句是为了在最后一行的时候不要再写入回车了。
path:就是你想把记录下来的数据储存在哪个文件夹下，比如./loss.txt，就是储存在当前路径下的txt文件。这里建议保存成txt文件，以后要用的话就很方便处理。这个txt文件不需要自己新建一个空白，没有的话python会自己建立一个的
open函数中我选择用a参数，而不是用w，这是因为w会抹去之前写过的重新写，最后只剩下最后一对数据了，而我们的目的明显不是这样的，a是从上一次的位置接着写，这就nice了。

2 根据保存好的训练过程中的损失值可视化
“”"

Note: The code is used to show the change trende via the whole training procession.
First: You need to mark all the loss of every iteration
Second: You need to write these data into a txt file with the format like:
…
iter loss
iter loss
…
Third: the path is the txt file path of your loss

“”"

import matplotlib.pyplot as plt

def read_txt(path):
with open(path, ‘r’) as f:
lines = f.readlines()
splitlines = [x.strip().split(’ ') for x in lines]
return splitlines

Referenced from Tensorboard(a smooth_loss function:https://blog.youkuaiyun.com/charel_chen/article/details/80364841)

def smooth_loss(path, weight=0.85):
iter = []
loss = []
data = read_txt(path)
for value in data:
iter.append(int(value[0]))
loss.append(int(float(value[1])))
# Note a str like ‘3.552’ can not be changed to int type directly
# You need to change it to float first, can then you can change the float type ton int type
last = loss[0]
smoothed = []
for point in loss:
smoothed_val = last * weight + (1 - weight) * point
smoothed.append(smoothed_val)
last = smoothed_val
return iter, smoothed

if name == “main”:
path = ‘./loss.txt’
loss = []
iter = []
iter, loss = smooth_loss(path)
plt.plot(iter, loss, linewidth=2)
plt.title(“Loss-iters”, fontsize=24)
plt.xlabel(“iters”, fontsize=14)
plt.ylabel(“loss”, fontsize=14)
plt.tick_params(axis=‘both’, labelsize=14)
plt.savefig(’./loss_func.png’)
plt.show()

这里借用了tensorboard 平滑损失曲线代码
————————————————
版权声明：本文为优快云博主「Kurt Sun」的原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.youkuaiyun.com/weixin_43748786/article/details/96432391