在BBDM-LAB摸爬滚打的日子(1)

本文纠正了关于交叉验证的误解,并详细介绍了如何通过5折交叉验证来正确地划分数据集进行模型评估,同时讨论了如何计算AUC等评估指标。

今天是2018年5月25日,纠正了对交叉验证的错误认识,还在努力实现AUC和AUPr的计算。 MATLAB的矩阵计算思维我还掌握不了,好难受……

交叉验证,以5折交叉验证为例。
已知数据为n个links、一个反应矩阵interaction。
5折交叉验证所说的“将数据集分为5份,其中4份为训练集,另一份为测试集”,这里的“数据集”指的是links,而不是interaction。

解释一下为什么划分的不是interaction。假如划分的数据集是interaction,很有可能取出的测试集值全为0(矩阵稀疏的时候很有可能是这种情况),也就是说,没有真正例和假正例,根本就计算不了评估指标(AUC等),没有实际意义。

model_evaluation(real, predict)
那么,输入模型评估函数的参数real和predict是什么呢?
‘test_index = find(train_interaction==0)’
‘real = interaction (test_index)’
‘predict = scores(test_index)’
上面三行是MATLAB代码。
注意,train_interaction是把测试集包含的links全置为0之后的反应矩阵。
所以,参与计算AUC等指标的interaction中的元素 包括:
1. 未知的link(值本就为0的元素) 未知的link就当作不存在这个link
2. 测试集中的link(值本来为1,后置为0的元素)

TP 真正例:预测为真且属于测试集的link
FP 假正例:预测为真但属于未知link
TN 真反例:预测为假且属于未知link
FN 假反例:预测为假但属于测试集的link

(bbdm) zzl@dlteam-X299-WU8:~/BBDM$ nvidia-smi Mon Nov 10 15:13:08 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 570.124.04 Driver Version: 570.124.04 CUDA Version: 12.8 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3090 Off | 00000000:1A:00.0 Off | N/A | | 36% 30C P0 103W / 350W | 1MiB / 24576MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ (bbdm) zzl@dlteam-X299-WU8:~/BBDM$ nvidia-debugdump -l Found 1 NVIDIA devices Device ID: 0 Device name: NVIDIA GeForce RTX 3090 (*PrimaryCard) GPU internal ID: GPU-c3c236fc-05c9-eeb0-0f86-68106a625ad8 (bbdm) zzl@dlteam-X299-WU8:~/BBDM$ cat /proc/driver/nvidia/version NVRM version: NVIDIA UNIX x86_64 Kernel Module 570.124.04 Tue Feb 25 04:12:12 UTC 2025 GCC version: gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2) (bbdm) zzl@dlteam-X299-WU8:~/BBDM$ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Thu_Jun_11_22:26:38_PDT_2020 Cuda compilation tools, release 11.0, V11.0.194 Build cuda_11.0_bu.TC445_37.28540450_0 (bbdm) zzl@dlteam-X299-WU8:~/BBDM$ python3 main.py --config configs/Template-BBDM-ct2cta.yaml --train --save_top --gpu_ids 0 --resume_model "results/ct2cta(全)/BrownianBridge/checkpoint/latest_model_12.pth" --resume_optim "results/ct2cta(全)/BrownianBridge/checkpoint/latest_optim_sche_12.pth" save training results to results/ct2cta(全)/BrownianBridge/ Traceback (most recent call last): File "main.py", line 130, in <module> main() File "main.py", line 125, in main CPU_singleGPU_launcher(nconfig) File "main.py", line 91, in CPU_singleGPU_launcher runner = get_runner(config.runner, config) File "/home/zzl/BBDM/utils.py", line 45, in get_runner runner = Registers.runners[runner_name](config) File "/home/zzl/BBDM/runners/DiffusionBasedModelRunners/BBDMRunner.py", line 19, in __init__ super().__init__(config) File "/home/zzl/BBDM/runners/DiffusionBasedModelRunners/DiffusionBaseRunner.py", line 11, in __init__ super().__init__(config) File "/home/zzl/BBDM/runners/BaseRunner.py", line 59, in __init__ self.net, self.optimizer, self.scheduler = self.initialize_model_optimizer_scheduler(self.config) File "/home/zzl/BBDM/runners/BaseRunner.py", line 104, in initialize_model_optimizer_scheduler net = self.initialize_model(config) File "/home/zzl/BBDM/runners/DiffusionBasedModelRunners/BBDMRunner.py", line 23, in initialize_model bbdmnet = BrownianBridgeModel(config.model).to(config.training.device[0]) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1145, in to return self._apply(convert) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 820, in _apply param_applied = fn(param) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1143, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) File "/home/zzl/.conda/envs/bbdm/lib/python3.8/site-packages/torch/cuda/__init__.py", line 247, in _lazy_init torch._C._cuda_init() RuntimeError: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero.
最新发布
11-11
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值