Hello Word!

【代码】mmcv报错 mmcv/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol。

报错：pytorch DDP 模型卡住代码# 具体卡住的代码yolov5训练代码 train.py 中有一句： scaler.step(optimizer) # optimizer.step程序运行到第二个epoch的时候，卡住了，具体卡在调用语句：/home/xxx/lib/python3.7/site-packages/torch/cuda/amp/grad_scaler.py中的 if not sum(v.item() for v

现象：已经重复发生过，而且掉进程的卡bus id不一样Pytorch 版本 1.7.0 ，卡：titan rtx x 8很奇怪的bug，目前分析的原因：batch size过大，每张卡显存占的太满，导致训练的时候显存爆了？问题：那为什么一开始不爆呢？程序有显存泄漏？显卡过热（这个目前应该不是引起的原因，因为散热风挡已经开到最大，温度最多70度左右）...

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [25088, 512]], which is output 0 of TBackward, is at version 4; expected version 3 instead. Hint: enable anomaly detection

报错：Traceback (most recent call last): File "train_noPfc.py", line 201, in <module> main(args_) File "train_noPfc.py", line 160, in main f_masked, focc_masked, output, output_occ, f_diff, out = backbone(img1, img2) File "/home/user1/min

就一个前向推理，也报错了。。。代码：from torchvision import modelsmodel = models.vgg19(pretrained=True)output = model(input.cuda())报错：RuntimeError: Legacy autograd function with non-static forward method is deprecated.完整报错：Traceback (most recent call last): File

Traceback (most recent call last): File "/home/user1/pjs/frvt_pytorch/batch_run/2branch_alter_1update_2pfc_MMD_ori_auto/recognition/arcface_torch/tools/visualize.py", line 276, in <module> mask = grad_cam(input, target_index) File "/home/user

训练过程中多个loss回传产生了GPU显存不够用的情况（即使是设置batch_size最小也不行），在backward函数中去掉retain_graph=True之后，情况没有出现。我这里出现这个情况的原因：因为不同loss求完之后没有算均值，可能返回的是一个tensor，要通过 .mean() 把它变成标量。解决：criterion = torch.nn.CrossEntropyLoss()output = module_a(fc1Features,label)arcLoss = criteri

报错：RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1解决：把DP改成DDP解决了报错的代码：# DP模式, module_a是一个分类器module_a = torch.nn.DataParallel(module_a)改完的代码：local_rank = 0# DDP模式mod

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [25088, 512]], which is output 0 of TBackward, is at version 4; expected version 3 instead. Hint: enable anomaly detection

报错：Traceback (most recent call last): File "train.py", line 166, in <module> main(args_) File "train.py", line 118, in main f_clean_masked, f_occ_masked, fc, fc_occ = backbone(img1, img2) File "/home/user1/miniconda3/envs/py377/lib/pyt

sss

报错：RuntimeError: Expected 4-dimensional input for 4-dimensional weight [512, 512, 3, 3], but got 2-dimensional input of size [4, 512] instead

报错：Traceback (most recent call last): File "topFAR_COX_py2_fc1_pytorch_PY3.py", line 199, in <module> IDimage_features_dict = getfeatures_dict(model, IDimage_list, IDimage_path, featurelen) File "topFAR_COX_py2_fc1_pytorch_PY3.py", line 106,

完整报错：Traceback (most recent call last): File "topFAR_COX_py2_fc1_pytorch.py", line 181, in <module> model = constructmodel(args) File "topFAR_COX_py2_fc1_pytorch.py", line 36, in constructmodel dist.init_process_group(backend='nccl', ini

完整报错：Traceback (most recent call last): File "topFAR_COX_py2_fc1_pytorch.py", line 181, in <module> model = constructmodel(args) File "topFAR_COX_py2_fc1_pytorch.py", line 37, in constructmodel torch.cuda.set_device(rank) File "/home/u

你的模型或者参数没有放到GPU上，解决：backbone = eval("backbones.{}".format(args.network))(False, dropout=dropout, fp16=args.fp16).to(rank)最后的 .to(rank) 做到了这一点。在我的代码中rank=0

No handlers could be found for logger “mxboard.event_file_writer”解决：$ pip install tensorboard在代码中加入：import logginglogging.basicConfig(level=logging.DEBUG)https://github.com/reminisce/mxboard-demo/blob/master/train_mnist.py

1， loss很快下降到0附近，后面不再动弹。验证集准确率始终保持在50%-60%附近图：问题：可能是传入的gt标签有错误，比如每次传入的都是同一个标签。需检查

手动写mxnet预测代码对单个图像进行预测时，报错：Traceback (most recent call last): File "/home/user1/pjs/frvt/arcface_Siamese_offline/recognition/tools/eval_on_train_set.py", line 163, in <module> fc7_mod.init_params(arg_params=fc7_overall, aux_params=None) File

报错：mxnet.base.MXNetError: Error in operator conv0: [17:40:27] src/operator/nn/convolution.cc:152: Check failed: dshp.ndim() == 4U (3 vs. 4) : Input data should be 4D in batch-num_filter-y-x明明输入数据是4维的，为什么报错？因为用collections.namedtuple装载数据，进行前向预测时，没有给data外面加

Traceback (most recent call last): File "/home/user1/pjs/frvt/mask/arcface_Siamese_offline/recognition/train_0305.py", line 497, in <module> main() File "/home/user1/pjs/frvt/mask/arcface_Siamese_offline/recognition/train_0305.py", line 493,

报错：src/base.cc:49: GPU context requested, but no GPUs found.src/storage/storage.cc???? Check failed: e == cudaSuccess || e == cudaErrorCudartUnloading: CUDA: no CUDA-capable device is detected[14:07:55] src/base.cc:49: GPU context requested, but no GPUs

nan表示结果太大，正无穷或者负无穷可能的原因：1，学习率太大，比如应该从0.1改为0.01或0.001。如果还不行，那应该是代码有问题2， 你loss算的不对，导致太大3， 你梯度算的不对，如果loss是手动构建的， 请手动推导梯度计算公式，并正确实现...

loading ['/data/user1/log/frvt/models/r34_webface_mask/siamese/siamese34-arcface-webfaceSiamese/model-0009.params']Traceback (most recent call last): File "train_0305.py", line 447, in <module> main() File "train_0305.py", line 443, in main

Traceback (most recent call last): File "train_parall_fc7.py", line 409, in <module> main() File "train_parall_fc7.py", line 406, in main train_net(args) File "train_parall_fc7.py", line 401, in train_net epoch_end_callback = epoch_

pytorch 1.7高版本训练保存的模型在1.4低版本无法加载，报错：torch.load('/home/user1/model_best_b.pth.tar')Traceback (most recent call last): File "/data/user1/pkgs/conda/envs/drc/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code exec(co

mxnet 1.6 自定义OP （计算metric）训练报错：src/imperative/./imperative_utils.h:72: Check failed: inputs[i]->ctx().dev_mask() == ctx.dev_mask() (1 vs. 2) : Operator broadcast_add require all inputs live on the same context. But the first argument is on gpu(0) while

哥们，你这个模型load_state_dict加载参数之后是不会将模型作为返回值的。所以你不要去接收返回参数。只是load_state_dict就可以了。错误的代码：# 这里你用列表接收了load_state_dict的返回值，但它其实不是load_state_dict之后得到的模型，而是一个_IncompatibleKeys的什么鬼models = [m.load_state_dict(ckpts[i]['state_dict']) for i,m in enumerate(models)]正确

报错：MXNetError: Check failed: assign(&dattr, vec.at(i)): Incompatible attr in node at 0-th output: expected [51249,512], got [59643,512]这种奇奇怪怪的报错其实很头疼，mxnet给出的提示根本很难定位到具体代码，详细：Traceback (most recent call last): File "train_0723.py", line 494, in

Traceback (most recent call last): File "image_iter.py", line 42, in <module> C = FaceImageIter(20, (3,112,112), path_imgrec='/home/user1/data/deepglint/train.rec') File "image_iter.py", line 39, in __init__ s = self.imgrec.read_idx(0)

报错：$ pythonPython 3.7.7 (default, Mar 26 2020, 15:48:22)[GCC 7.3.0] :: Anaconda, Inc. on linuxType "help", "copyright", "credits" or "license" for more information.>>> import mxnet as mx>>> mx.cpuAttributeError: module 'mxnet' has

Segmentation fault: 11Stack trace: [bt] (0) /data/user1/pkgs/conda/envs/drc/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x40c6b50) [0x7f2dfe4f8b50] [bt] (1) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f2edc06b4b0] [bt] (2) /lib/x86_64-linux-gn

tf.version ‘2.3.0’keras.version ‘2.4.3’tensoflow代码：out = model.predict(x_test)out = np.array(out)报错：---------------------------------------------------------------------------InvalidArgumentError Traceback (most recent call las

Traceback (most recent call last): File "train_0723.py", line 455, in <module> main() File "train_0723.py", line 451, in main train_net(args) File "train_0723.py", line 445, in train_net epoch_end_callback=epoch_cb) File "/home/us

代码：criterion = nn.BCEWithLogitsLoss(reduction='none')loss = criterion(output, target)loss.mul_(weights)报错：Traceback (most recent call last):File “/home/user1/main_cs_0708.py”, line 391, in main()File “/home/user1/main_cs_0708.py”, line 301, in mai

报错：Traceback (most recent call last): File "/data/user1/pkgs/conda/envs/drc/lib/python3.7/site-packages/graphviz/backend.py", line 129, in render subprocess.check_call(args, stderr=stderr, **POPEN_KWARGS) File "/data/user1/pkgs/conda/envs/drc/lib/

sklearn 0.19各种报错还不能升级：报错1（使用时报错）：ImportError: cannot import name 'multilabel_confusion_matrix'报错2（升级时报错）：$ pip install git+http://github.com/scikit-learn/scikit-learn.gitLooking in indexes: https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.t

命令输出缓慢，并且报ERR1，重启机器2，重装驱动3，维修或换显卡

画图出不来UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.解决：import matplotlib.pyplot as pltplt.switch_backend('TkAgg')okie@https://stackoverflow.com/questions/56656777/userwarning-matplotlib-is-curre