MQBench记录

分类任务

PTQ可以作为QAT初始化

model = prepare_qat_fx_by_platform(model, BackendType)	# 可以加入额外参数(函数模块策略)
model.eval()
enable_calibration(model)

开启伪量化后，模型停止校准
通过梯度更新学习scale、zp，或者固定scale调整weight

model.train()
enable_quantization(model)

将模型与量化参数以ONNX形式保存
然后将量化参数从ONNX分离开

conver_deploy(model.eval(), BackendType, input_shape)

yolov5 + MQbench QAT

初始学习率：fp32模型收敛的学习率或者更小
适当关闭一些复杂的数据增强(mosaic、mixup)
warm_up也可以关闭
fakequant：fix、lsq observer：minmax、ClipStd、Quantile

将模型拆分成纯网络和后处理

model = Model(xxx)
model_post = Detect_post().to(device)

量化方法设置

extra_qconfig_dict = {
	"w_observer": "MinMaxObserver",
	"a_observer": "EMAMinMaxObserver",
	"w_fakequantize": "FixedFakeQuantize",
	"a_fakequantize": "FixedFakeQuantize",
}
leaf_module = {frozenBatchNorm2d, Focus}
extra_quantizer_dict = {"additional_module_type": {ConvFreezebn2d, ConvFreezebnReLU2d}}
prepare_custom_config_dict = {
	"extra_qconfig_dict": extra_qconfig_dict,
	"leaf_module": leaf_module,
	"extra_quantizer_dict", extra_quantizer_dict
}

模型trace

model.train()
model = prepare_by_platform(model, BackendType.ONNX_QNN, prepare_custom_config_dict)
model.cuda()

PTQ校准

model.eval()
enable_calibration(model)
for i (imgs, targets, paths, _) in enumerate(train_loader):
	imgs = imgs.to(device, non_blocking=True).float() / 255
	pred = model(imgs)
	if i >= 256:
		break
enable_quantization(model)

QAT训练

for epoch in range(start_epoch, epochs):
	model.train()
	model_post.train()