从零构建:一个高精度车牌识别系统的工程化实践
很多开发者朋友在接触计算机视觉时,都会对车牌识别这个经典项目产生兴趣。它不像人脸识别那样需要庞大的数据集,也不像目标检测那样对硬件要求苛刻,但它却完整地串联了图像处理、模式识别和机器学习等多个核心环节。我记得自己第一次尝试用OpenCV做车牌识别时,被各种预处理步骤搞得晕头转向——为什么别人的代码能准确找到车牌,我的却总是误判?为什么字符分割时总是切得乱七八糟?这些问题背后,其实隐藏着对图像处理流程的系统性理解。
今天,我们不谈那些泛泛而谈的理论,而是从一个工程化的角度,深入探讨如何构建一个真正可用、鲁棒性强的车牌识别系统。这篇文章适合那些已经掌握了Python基础语法,对OpenCV有初步了解,但想要深入理解计算机视觉项目完整流程的开发者。我们将从最基础的图像预处理开始,逐步深入到字符分割、特征提取和识别模型的选择,最终形成一个完整的解决方案。更重要的是,我会分享在实际项目中遇到的坑和解决方案,这些经验往往比代码本身更有价值。
1. 环境搭建与项目架构设计
在开始编码之前,合理的环境配置和项目架构设计能让你事半功倍。很多初学者习惯把所有代码写在一个文件里,当项目复杂度增加时,调试和维护就变得异常困难。
1.1 环境配置的细节考量
首先,我们需要一个稳定且版本匹配的OpenCV环境。虽然可以直接用pip install opencv-python安装,但对于车牌识别这种需要精确控制图像处理流程的项目,我建议安装完整的OpenCV包:
pip install opencv-python==4.8.1
pip install opencv-contrib-python==4.8.1
pip install numpy==1.24.3
pip install scikit-learn==1.3.0
pip install imutils==0.5.4
这里有几个关键点需要注意:
- 版本锁定:不同版本的OpenCV在API和算法实现上可能有细微差异,锁定版本可以确保代码的稳定性
- contrib模块:包含了一些额外的功能,比如更先进的图像处理算法
- imutils:这个库提供了一些非常实用的图像处理辅助函数,能大大简化代码
注意:如果你在Windows系统上遇到编译问题,可以考虑使用Anaconda环境,它已经预编译好了大多数科学计算库,能避免很多环境配置的麻烦。
1.2 项目目录结构设计
一个清晰的项目结构不仅方便代码管理,也便于团队协作。我推荐采用以下目录结构:
license_plate_recognition/
├── config/
│ ├── __init__.py
│ ├── settings.py # 配置文件,存放所有可调参数
│ └── paths.py # 路径配置文件
├── data/
│ ├── raw/ # 原始图像数据
│ ├── processed/ # 预处理后的数据
│ ├── train/ # 训练数据
│ └── test/ # 测试数据
├── src/
│ ├── __init__.py
│ ├── preprocessing.py # 图像预处理模块
│ ├── detection.py # 车牌检测模块
│ ├── segmentation.py # 字符分割模块
│ ├── recognition.py # 字符识别模块
│ └── utils.py # 工具函数
├── models/
│ ├── __init__.py
│ ├── svm_classifier.pkl # 训练好的SVM模型
│ └── cnn_model.h5 # 训练好的CNN模型
├── tests/
│ ├── test_preprocessing.py
│ └── test_detection.py
├── notebooks/
│ └── exploration.ipynb # 探索性数据分析
├── requirements.txt
├── train.py # 训练脚本
└── inference.py # 推理脚本
这种模块化的设计有几个明显优势:
- 代码复用性高:每个功能模块独立,可以在不同项目中复用
- 调试方便:可以单独测试每个模块的功能
- 配置集中管理:所有可调参数都在config目录下,修改时不需要到处找
1.3 配置文件的编写技巧
在config/settings.py中,我们可以定义所有可配置的参数:
class Config:
# 图像预处理参数
IMAGE_SIZE = (640, 480) # 统一输入图像尺寸
GAUSSIAN_BLUR_KERNEL = (5, 5) # 高斯模糊核大小
ADAPTIVE_THRESH_BLOCK_SIZE = 11 # 自适应阈值块大小
ADAPTIVE_THRESH_C = 2 # 自适应阈值常数
# 车牌检测参数
MIN_PLATE_AREA = 2000 # 最小车牌面积
MAX_PLATE_AREA = 30000 # 最大车牌面积
ASPECT_RATIO_RANGE = (2.0, 5.0) # 车牌宽高比范围
# 字符分割参数
CHAR_MIN_HEIGHT_RATIO = 0.4 # 字符最小高度占车牌高度的比例
CHAR_MAX_HEIGHT_RATIO = 0.9 # 字符最大高度占车牌高度的比例
CHAR_MIN_WIDTH_RATIO = 0.05 # 字符最小宽度占车牌宽度的比例
CHAR_MAX_WIDTH_RATIO = 0.25 # 字符最大宽度占车牌宽度的比例
# 模型路径
SVM_MODEL_PATH = "models/svm_classifier.pkl"
CNN_MODEL_PATH = "models/cnn_model.h5"
# 训练参数
TRAIN_TEST_SPLIT = 0.8
RANDOM_STATE = 42
使用配置类的好处是,所有参数都在一个地方管理,修改起来非常方便。在实际项目中,我经常需要根据不同的场景调整这些参数,这种设计让我不需要在代码中到处搜索硬编码的值。
2. 图像预处理:从原始图像到可处理区域
图像预处理是整个车牌识别流程中最关键的一步,也是决定后续步骤成功率的基础。很多人直接跳过预处理或者处理不当,导致后面的检测和识别效果大打折扣。
2.1 多尺度图像增强策略
车牌图像可能在不同光照条件下拍摄,我们需要一套自适应的增强策略。我通常采用多级处理的方式:
import cv2
import numpy as np
from src.utils import timeit
class ImagePreprocessor:
def __init__(self, config):
self.config = config
@timeit
def preprocess(self, image):
"""完整的图像预处理流程"""
# 步骤1:尺寸标准化
resized = self._resize_image(image)
# 步骤2:多策略去噪
denoised = self._denoise_image(resized)
# 步骤3:对比度增强
enhanced = self._enhance_contrast(denoised)
# 步骤4:边缘增强
edged = self._enhance_edges(enhanced)
return edged
def _resize_image(self, image):
"""将图像缩放到统一尺寸"""
height, width = image.shape[:2]
target_width, target_height = self.config.IMAGE_SIZE
# 保持宽高比进行缩放
scale = min(target_width/width, target_height/height)
new_width = int(width * scale)
new_height = int(height * scale)
resized = cv2.resize(image, (new_width, new_height))
# 如果缩放后尺寸不足,进行填充
if new_width < target_width or new_height < target_height:
top = (target_height - new_height) // 2
bottom = target_height - new_height - top
left = (target_width - new_width) // 2
right = target_width - new_width - left
resized = cv2.copyMakeBorder(
resized, top, bottom, left, right,
cv2.BORDER_CONSTANT, value=[0, 0, 0]
)
return resized
def _denoise_image(self, image):
"""多策略去噪"""
# 方法1:中值滤波,对椒盐噪声效果好
median = cv2.medianBlur(image, 3)
# 方法2:高斯滤波,对高斯噪声效果好
gaussian = cv2.GaussianBlur(median, self.config.GAUSSIAN_BLUR_KERNEL, 0)
# 方法3:双边滤波,保留边缘的同时去噪
bilateral = cv2.bilateralFilter(gaussian, 9, 75, 75)
return bilateral
def _enhance_contrast(self, image):
"""自适应对比度增强"""
# 转换为LAB颜色空间,只对L通道进行CLAHE
lab = cv2.cvtColor(image, cv2.COLOR_BGR2LAB)
l, a, b = cv2.split(lab)
clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8, 8))
cl = clahe.apply(l)
# 合并通道并转回BGR
enhanced_lab = cv2.merge((cl, a, b))
enhanced = cv2.cvtColor(enhanced_lab, cv2.COLOR_LAB2BGR)
return enhanced
def _enhance_edges(self, image):
"""边缘增强"""
# 转换为灰度图
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# 使用Sobel算子检测边缘
sobelx = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3)
sobely = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3)
# 计算梯度幅值
gradient_magnitude = np.sqrt(sobelx**2 + sobely**2)
gradient_magnitude = np.uint8(255 * gradient_magnitude / gradient_magnitude.max())
# 与原图融合
enhanced = cv2.addWeighted(gray, 0.7, gradient_magnitude, 0.3, 0)
return enhanced
这个预处理类包含了几个关键技巧:
- 保持宽高比的缩放:避免图像变形
- 多级去噪:针对不同类型的噪声使用不同的滤波方法
- CLAHE对比度增强:在LAB颜色空间处理,避免颜色失真
- 边缘增强:突出车牌区域的边缘特征
2.2 颜色空间分析与车牌定位
车牌通常有特定的颜色特征(蓝底白字、黄底黑字等),我们可以利用这一点进行初步定位:
def detect_color_regions(self, image):
"""基于颜色特征检测候选区域"""
# 转换为HSV颜色空间,对光照变化更鲁棒
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# 定义蓝色车牌的HSV范围(中国车牌)
lower_blue = np.array([100, 50, 50])
upper_blue = np.array([130, 255, 255])
# 定义黄色车牌的HSV范围
lower_yellow = np.array([15, 50, 50])
upper_yellow = np.array([35, 255, 255])
# 创建掩码
blue_mask = cv2.inRange(hsv, lower_blue, upper_blue)
yellow_mask = cv2.inRange(hsv, lower_yellow, upper_yellow)
# 合并掩码
color_mask = cv2.bitwise_or(blue_mask, yellow_mask)
# 形态学操作去除噪声
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
color_mask = cv2.morphologyEx(color_mask, cv2.MORPH_CLOSE, kernel)
color_mask = cv2.morphologyEx(color_mask, cv2.MORPH_OPEN, kernel)
return color_mask
在实际测试中,我发现单纯依赖颜色特征容易受到环境干扰。比如,蓝色的天空或黄色的广告牌都可能被误判为车牌。因此,颜色检测只能作为初步筛选,还需要结合其他特征进行验证。
3. 车牌检测:从候选区域到精确定位
经过预处理后,我们需要从图像中准确找到车牌的位置。这是一个典型的目标检测问题,但与传统目标检测不同,车牌有非常明显的几何特征。
3.1 基于边缘和轮廓的检测算法
传统的车牌检测方法主要基于边缘检测和轮廓分析。虽然现在深度学习的方法更先进,但传统方法在计算资源有限或需要快速部署的场景下仍有优势:
class LicensePlateDetector:
def __init__(self, config):
self.config = config
def detect(self, image):
"""检测图像中的所有车牌"""
# 步骤1:边缘检测
edges = self._detect_edges(image)
# 步骤2:查找轮廓
contours = self._find_contours(edges)
# 步骤3:筛选候选区域
candidates = self._filter_contours(contours)
# 步骤4:精确定位和校正
plates = self._refine_candidates(image, candidates)
return plates
def _detect_edges(self, image):
"""使用Canny算法检测边缘"""
# 转换为灰度图
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# 自适应阈值,避免手动调整参数
v = np.median(gray)
sigma = 0.33
# 根据中值自动计算高低阈值
lower = int(max(0, (1.0 - sigma) * v))
upper = int(min(255, (1.0 + sigma) * v))
edges = cv2.Canny(gray, lower, upper)
# 形态学操作连接断开的边缘
kernel = np.ones((3, 3), np.uint8)
edges = cv2.dilate(edges, kernel, iterations=1)
edges = cv2.erode(edges, kernel, iterations=1)
return edges
def _find_contours(self, edges):
"""查找所有轮廓"""
contours, _ = cv2.findContours(
edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)
return contours
def _filter_contours(self, contours):
"""基于几何特征筛选轮廓"""
candidates = []
for contour in contours:
# 计算轮廓的边界矩形
x, y, w, h = cv2.boundingRect(contour)
# 计算轮廓面积
area = cv2.contourArea(contour)
# 计算宽高比
aspect_ratio = w / h if h > 0 else 0
# 计算矩形度(轮廓面积与边界矩形面积的比值)
rect_area = w * h
rectangularity = area / rect_area if rect_area > 0 else 0
# 筛选条件
conditions = [
area > self.config.MIN_PLATE_AREA,
area < self.config.MAX_PLATE_AREA,
self.config.ASPECT_RATIO_RANGE[0] < aspect_ratio < self.config.ASPECT_RATIO_RANGE[1],
rectangularity > 0.6, # 矩形度足够高
]
if all(conditions):
candidates.append({
'contour': contour,
'bbox': (x, y, w, h),
'area': area,
'aspect_ratio': aspect_ratio,
'rectangularity': rectangularity
})
# 按面积排序,取前N个
candidates.sort(key=lambda x: x['area'], reverse=True)
return candidates[:5] # 最多返回5个候选区域
def _refine_candidates(self, image, candidates):
"""精确定位和透视校正"""
plates = []
for candidate in candidates:
x, y, w, h = candidate['bbox']
# 提取候选区域
roi = image[y:y+h, x:x+w]
# 计算最小外接矩形,用于透视校正
rect = cv2.minAreaRect(candidate['contour'])
box = cv2.boxPoints(rect)
box = np.int0(box)
# 获取透视变换的源点
src_points = self._order_points(box)
# 计算目标点(校正后的矩形)
width = int(rect[1][0])
height = int(rect[1][1])
dst_points = np.array([
[0, height-1],
[0, 0],
[width-1, 0],
[width-1, height-1]
], dtype="float32")
# 计算透视变换矩阵
M = cv2.getPerspectiveTransform(src_points, dst_points)
# 应用透视变换
warped = cv2.warpPerspective(
image, M, (width, height)
)
plates.append({
'original_roi': roi,
'warped_plate': warped,
'bbox': (x, y, w, h),
'contour': candidate['contour']
})
return plates
def _order_points(self, pts):
"""将四个点按左上、右上、右下、左下的顺序排序"""
rect = np.zeros((4, 2), dtype="float32")
# 计算点的中心
center = np.mean(pts, axis=0)
# 根据点与中心的相对位置排序
diff = pts - center
angles = np.arctan2(diff[:, 1], diff[:, 0])
# 按角度排序
sorted_indices = np.argsort(angles)
# 重新排列点
for i, idx in enumerate(sorted_indices):
rect[i] = pts[idx]
return rect
这个检测器包含了几个关键优化:
- 自适应Canny阈值:根据图像中值自动计算高低阈值,避免手动调整
- 多特征筛选:结合面积、宽高比、矩形度等多个特征,提高准确性
- 透视校正:处理倾斜的车牌,确保字符分割的准确性
3.2 深度学习检测方法的集成
虽然传统方法在很多场景下表现不错,但对于复杂背景或极端光照条件,深度学习方法通常更鲁棒。我们可以集成一个轻量级的深度学习检测器:
class DeepPlateDetector:
def __init__(self, model_path="models/plate_detector.onnx"):
# 加载ONNX模型
self.net = cv2.dnn.readNetFromONNX(model_path)
# 设置后端和目标(CPU或GPU)
self.net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)
def detect(self, image, confidence_threshold=0.5):
"""使用深度学习模型检测车牌"""
# 准备输入
blob = cv2.dnn.blobFromImage(
image, 1/255.0, (640, 640), swapRB=True, crop=False
)
# 前向传播
self.net.setInput(blob)
outputs = self.net.forward()
# 解析输出
detections = self._parse_outputs(outputs, image.shape, confidence_threshold)
return detections
def _parse_outputs(self, outputs, image_shape, confidence_threshold):
"""解析模型输出"""
height, width = image_shape[:2]
detections = []
# 假设输出格式为 [batch_size, num_detections, 85]
# 85 = [x, y, w, h, confidence, class_probabilities...]
for detection in outputs[0]:
scores = detection[5:]
class_id = np.argmax(scores)
confidence = scores[class_id]
if confidence > confidence_threshold:
# 将归一化坐标转换为像素坐标
center_x = int(detection[0] * width)
center_y = int(detection[1] * height)
w = int(detection[2] * width)
h = int(detection[3] * height)
# 计算边界框坐标
x = int(center_x - w / 2)
y = int(center_y - h / 2)
detections.append({
'bbox': (x, y, w, h),
'confidence': float(confidence),
'class_id': int(class_id)
})
# 应用非极大值抑制
detections = self._apply_nms(detections)
return detections
def _apply_nms(self, detections, iou_threshold=0.5):
"""应用非极大值抑制"""
if not detections:
return []
boxes = [d['bbox'] for d in detections]
confidences = [d['confidence'] for d in detections]
# 使用OpenCV的NMSBoxes
indices = cv2.dnn.NMSBoxes(
boxes, confidences, 0.5, iou_threshold
)
if len(indices) > 0:
indices = indices.flatten()
return [detections[i] for i in indices]
return []
在实际项目中,我通常采用混合策略:先用传统方法快速检测,如果置信度不够高,再使用深度学习模型进行验证。这种策略在保证速度的同时提高了准确性。
4. 字符分割:将车牌分解为单个字符
字符分割是车牌识别中最具挑战性的环节之一。字符粘连、光照不均、污损等问题都会影响分割效果。
4.1 基于投影法的字符分割
投影法是最经典的字符分割方法,通过分析像素在水平和垂直方向的投影来定位字符边界:
class CharacterSegmenter:
def __init__(self, config):
self.config = config
def segment(self, plate_image):
"""分割车牌图像中的字符"""
# 步骤1:预处理车牌图像
processed = self._preprocess_plate(plate_image)
# 步骤2:垂直投影分割字符
vertical_projection = self._vertical_projection(processed)
char_boundaries = self._find_char_boundaries(vertical_projection)
# 步骤3:水平投影确定字符高度
horizontal_projection = self._horizontal_projection(processed)
char_height_range = self._find_char_height(horizontal_projection)
# 步骤4:提取字符图像
characters = self._extract_characters(
plate_image, char_boundaries, char_height_range
)
# 步骤5:验证和修正
validated_chars = self._validate_characters(characters)
return validated_chars
def _preprocess_plate(self, plate_image):
"""车牌图像预处理"""
# 转换为灰度图
gray = cv2.cvtColor(plate_image, cv2.COLOR_BGR2GRAY)
# 二值化
_, binary = cv2.threshold(
gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU
)
# 去除小噪声
kernel = np.ones((2, 2), np.uint8)
cleaned = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)
return cleaned
def _vertical_projection(self, binary_image):
"""计算垂直投影"""
height, width = binary_image.shape
projection = np.zeros(width, dtype=np.int32)
for col in range(width):
# 统计每列中白色像素的数量
projection[col] = np.sum(binary_image[:, col] == 255)
return projection
def _find_char_boundaries(self, projection):
"""根据垂直投影找到字符边界"""
boundaries = []
in_char = False
start = 0
# 平滑投影曲线,减少噪声影响
smoothed = self._smooth_projection(projection)
# 计算动态阈值
threshold = np.mean(smoothed) * 0.3
for i, value in enumerate(smoothed):
if value > threshold and not in_char:
# 进入字符区域
in_char = True
start = i
elif value <= threshold and in_char:
# 离开字符区域
in_char = False
end = i
# 验证字符宽度是否合理
char_width = end - start
plate_width = len(smoothed)
width_ratio = char_width / plate_width
if (self.config.CHAR_MIN_WIDTH_RATIO < width_ratio <
self.config.CHAR_MAX_WIDTH_RATIO):
boundaries.append((start, end))
# 如果最后一个字符延伸到图像边缘
if in_char:
end = len(smoothed) - 1
char_width = end - start
plate_width = len(smoothed)
width_ratio = char_width / plate_width
if (self.config.CHAR_MIN_WIDTH_RATIO < width_ratio <
self.config.CHAR_MAX_WIDTH_RATIO):
boundaries.append((start, end))
return boundaries
def _horizontal_projection(self, binary_image):
"""计算水平投影"""
height, width = binary_image.shape
projection = np.zeros(height, dtype=np.int32)
for row in range(height):
projection[row] = np.sum(binary_image[row, :] == 255)
return projection
def _find_char_height(self, projection):
"""根据水平投影确定字符高度范围"""
# 平滑投影
smoothed = self._smooth_projection(projection)
# 找到主要字符区域
threshold = np.mean(smoothed) * 0.2
char_rows = np.where(smoothed > threshold)[0]
if len(char_rows) > 0:
start = char_rows[0]
end = char_rows[-1]
return (start, end)
# 如果没有找到,返回整个高度
return (0, len(projection)-1)
def _smooth_projection(self, projection, window_size=3):
"""平滑投影曲线"""
kernel = np.ones(window_size) / window_size
smoothed = np.convolve(projection, kernel, mode='same')
return smoothed.astype(np.int32)
def _extract_characters(self, plate_image, boundaries, height_range):
"""根据边界提取字符图像"""
characters = []
height_start, height_end = height_range
char_height = height_end - height_start + 1
for start, end in boundaries:
# 提取字符区域,留出一些边距
margin = 2
x_start = max(0, start - margin)
x_end = min(plate_image.shape[1], end + margin)
y_start = max(0, height_start - margin)
y_end = min(plate_image.shape[0], height_end + margin)
char_image = plate_image[y_start:y_end, x_start:x_end]
# 标准化字符图像尺寸
standardized = self._standardize_char(char_image)
characters.append({
'image': standardized,
'original_bbox': (x_start, y_start, x_end-x_start, y_end-y_start),
'position': len(characters) # 字符位置
})
return characters
def _standardize_char(self, char_image, target_size=(20, 20)):
"""标准化字符图像尺寸"""
# 转换为灰度图
if len(char_image.shape) == 3:
gray = cv2.cvtColor(char_image, cv2.COLOR_BGR2GRAY)
else:
gray = char_image
# 二值化
_, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# 调整大小,保持宽高比
h, w = binary.shape
scale = min(target_size[0]/h, target_size[1]/w)
new_h, new_w = int(h * scale), int(w * scale)
resized = cv2.resize(binary, (new_w, new_h))
# 创建目标图像并居中放置
result = np.zeros(target_size, dtype=np.uint8)
y_offset = (target_size[0] - new_h) // 2
x_offset = (target_size[1] - new_w) // 2
result[y_offset:y_offset+new_h, x_offset:x_offset+new_w] = resized
return result
def _validate_characters(self, characters):
"""验证分割出的字符"""
if len(characters) < 5: # 车牌至少5个字符
return []
# 检查字符宽度的一致性
widths = [char['image'].shape[1] for char in characters]
mean_width = np.mean(widths)
std_width = np.std(widths)
# 移除宽度异常的字符
valid_chars = []
for char in characters:
width = char['image'].shape[1]
if abs(width - mean_width) < 2 * std_width:
valid_chars.append(char)
return valid_chars
投影法虽然简单,但在实际应用中需要处理很多边界情况。我通过添加平滑处理、动态阈值和验证机制,大大提高了分割的鲁棒性。
4.2 处理粘连字符的进阶技巧
当字符粘连在一起时,投影法会失效。这时需要更高级的分割方法:
def segment_connected_chars(self, char_image):
"""处理粘连字符"""
# 使用连通组件分析
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(
char_image, connectivity=8
)
# 过滤掉太小的组件(噪声)
min_area = char_image.size * 0.01 # 至少占图像面积的1%
components = []
for i in range(1, num_labels): # 跳过背景(标签0)
area = stats[i, cv2.CC_STAT_AREA]
if area > min_area:
x = stats[i, cv2.CC_STAT_LEFT]
y = stats[i, cv2.CC_STAT_TOP]
w = stats[i, cv2.CC_STAT_WIDTH]
h = stats[i, cv2.CC_STAT_HEIGHT]
components.append({
'label': i,
'bbox': (x, y, w, h),
'area': area,
'centroid': centroids[i]
})
# 按x坐标排序
components.sort(key=lambda c: c['bbox'][0])
# 如果组件数量大于1,说明有粘连字符被分开了
if len(components) > 1:
separated_chars = []
for comp in components:
x, y, w, h = comp['bbox']
char_roi = char_image[y:y+h, x:x+w]
standardized = self._standardize_char(char_roi)
separated_chars.append(standardized)
return separated_chars
# 如果只有一个组件,尝试使用滴水算法进一步分割
return self._water_flow_segmentation(char_image)
def _water_flow_segmentation(self, char_image):
"""滴水算法分割粘连字符"""
height, width = char_image.shape
# 找到字符的上下边界
horizontal_proj = np.sum(char_image, axis=1)
char_rows = np.where(horizontal_proj > 0)[0]
if len(char_rows) == 0:
return [char_image]
top = char_rows[0]
bottom = char_rows[-1]
# 在字符中间高度处进行垂直切割
middle_row = (top + bottom) // 2
vertical_line = char_image[middle_row, :]
# 找到切割点(像素值从0到255或从255到0的过渡点)
transitions = []
for i in range(1, len(vertical_line)):
if vertical_line[i-1] == 0 and vertical_line[i] == 255:
transitions.append(('start', i))
elif vertical_line[i-1] == 255 and vertical_line[i] == 0:
transitions.append(('end', i))
# 根据过渡点进行分割
segments = []
if len(transitions) >= 2:
for i in range(0, len(transitions)-1, 2):
if transitions[i][0] == 'start' and transitions[i+1][0] == 'end':
start_col = transitions[i][1]
end_col = transitions[i+1][1]
# 提取字符段
segment = char_image[:, start_col:end_col]
# 验证宽度是否合理
if 5 < segment.shape[1] < width * 0.5:
segments.append(segment)
return segments if segments else [char_image]
粘连字符分割是车牌识别中的难点,我通常结合多种方法:先用连通组件分析,如果不行再用滴水算法,最后还可以考虑使用基于深度学习的分割模型。
5. 字符识别:从图像到文本
字符识别是整个流程的最后一步,也是决定系统准确性的关键。传统方法使用特征提取+分类器,现代方法则倾向于使用深度学习。
5.1 传统方法:特征提取与SVM分类
对于资源受限的环境,传统方法仍然有优势:
class TraditionalCharRecognizer:
def __init__(self, model_path=None):
self.feature_extractor = FeatureExtractor()
self.classifier = self._load_classifier(model_path)
self.characters = '0123456789ABCDEFGHJKLMNPQRSTUVWXYZ' # 去除易混淆字符
def extract_features(self, char_image):
"""提取字符特征"""
features = []
# 1. 网格特征
grid_features = self._extract_grid_features(char_image, grid_size=4)
features.extend(grid_features)
# 2. 投影特征
projection_features = self._extract_projection_features(char_image)
features.extend(projection_features)
# 3. 轮廓特征
contour_features = self._extract_contour_features(char_image)
features.extend(contour_features)
# 4. 矩特征
moment_features = self._extract_moment_features(char_image)
features.extend(moment_features)
return np.array(features)
def _extract_grid_features(self, image, grid_size=4):
"""网格特征:将图像分为多个网格,统计每个网格的像素密度"""
h, w = image.shape
cell_h = h // grid_size
cell_w = w // grid_size
features = []
for i in range(grid_size):
for j in range(grid_size):
cell = image[i*cell_h:(i+1)*cell_h, j*cell_w:(j+1)*cell_w]
density = np.sum(cell == 255) / (cell_h * cell_w)
features.append(density)
return features
def _extract_projection_features(self, image):
"""投影特征:水平和垂直投影的统计特征"""
# 水平投影
horizontal_proj = np.sum(image == 255, axis=1)
# 垂直投影
vertical_proj = np.sum(image == 255, axis=0)
features = []
# 均值
features.append(np.mean(horizontal_proj))
features.append(np.mean(vertical_proj))
# 方差
features.append(np.var(horizontal_proj))
features.append(np.var(vertical_proj))
# 最大值位置
features.append(np.argmax(horizontal_proj) / len(horizontal_proj))
features.append(np.argmax(vertical_proj) / len(vertical_proj))
return features
def _extract_contour_features(self, image):
"""轮廓特征"""
# 查找轮廓
contours, _ = cv2.findContours(
image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)
if not contours:
return [0] * 6
# 取最大的轮廓
contour = max(contours, key=cv2.contourArea)
features = []
# 面积
area = cv2.contourArea(contour)
features.append(area / (image.shape[0] * image.shape[1]))
# 周长
perimeter = cv2.arcLength(contour, True)
features.append(perimeter)
# 矩形度
_, _, w, h = cv2.boundingRect(contour)
rect_area = w * h
features.append(area / rect_area if rect_area > 0 else 0)
# 圆形度
features.append((4 * np.pi * area) / (perimeter ** 2) if perimeter > 0 else 0)
# 凸性
hull = cv2.convexHull(contour)
hull_area = cv2.contourArea(hull)
features.append(area / hull_area if hull_area > 0 else 0)
# 偏心率
if len(contour) >= 5:
ellipse = cv2.fitEllipse(contour)
features.append(ellipse[1][0] / ellipse[1][1] if ellipse[1][1] > 0 else 0)
else:
features.append(0)
return features
def _extract_moment_features(self, image):
"""矩特征"""
moments = cv2.moments(image)
features = []
# 中心矩
if moments['m00'] != 0:
cx = moments['m10'] / moments['m00']
cy = moments['m01'] / moments['m00']
features.extend([cx, cy])
else:
features.extend([0, 0])
# Hu矩(7个不变矩)
hu_moments = cv2.HuMoments(moments)
# 取对数以压缩动态范围
for i in range(7):
if hu_moments[i] != 0:
features.append(-np.sign(hu_moments[i]) * np.log10(abs(hu_moments[i])))
else:
features.append(0)
return features
def recognize(self, char_image):
"""识别单个字符"""
# 提取特征
features = self.extract_features(char_image)
# 使用分类器预测
if self.classifier is not None:
prediction = self.classifier.predict([features])[0]
confidence = np.max(self.classifier.predict_proba([features])[0])
return self.characters[prediction], confidence
return None, 0
def _load_classifier(self, model_path):
"""加载训练好的分类器"""
if model_path and os.path.exists(model_path):
with open(model_path, 'rb') as f:
return pickle.load(f)
return None
传统方法的优势在于特征的可解释性,我们可以清楚地知道每个特征代表什么。但它的缺点是需要手动设计特征,且对字符变形、噪声等情况的鲁棒性不如深度学习方法。
5.2 深度学习方法:CNN字符识别
对于追求高准确率的场景,卷积神经网络(CNN)是更好的选择:
class CNNCharRecognizer:
def __init__(self, model_path=None):
self.characters = '0123456789ABCDEFGHJKLMNPQRSTUVWXYZ'
self.num_classes = len(self.characters)
if model_path and os.path.exists(model_path):
self.model = self._load_model(model_path)
else:
self.model = self._build_model()
def _build_model(self):
"""构建CNN模型"""
model = Sequential([
# 第一卷积层
Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
BatchNormalization(),
MaxPooling2D((2, 2)),
Dropout(0.25),
# 第二卷积层
Conv2D(64, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D((2, 2)),
Dropout(0.25),
# 第三卷积层
Conv2D(128, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D((2, 2)),
Dropout(0.25),
# 全连接层
Flatten(),
Dense(256, activation='relu'),
BatchNormalization(),
Dropout(0.5),
Dense(self.num_classes, activation='softmax')
])
model.compile(
optimizer=Adam(learning_rate=0.001),
loss='categorical_crossentropy',
metrics=['accuracy']
)
return model
def preprocess_for_cnn(self, char_image):
"""预处理字符图像供CNN使用"""
# 确保是28x28的灰度图
if len(char_image.shape) == 3:
gray = cv2.cvtColor(char_image, cv2.COLOR_BGR2GRAY)
else:
gray = char_image
# 调整大小
resized = cv2.resize(gray, (28, 28))
# 归一化
normalized = resized.astype('float32') / 255.0
# 添加通道维度
processed = np.expand_dims(normalized, axis=-1)
return processed
def recognize(self, char_image):
"""使用CNN识别字符"""
# 预处理
processed = self.preprocess_for_cnn(char_image)
# 批量维度
batch = np.expand_dims(processed, axis=0)
# 预测
predictions = self.model.predict(batch, verbose=0)
# 获取最可能的字符和置信度
class_idx = np.argmax(predictions[0])
confidence = predictions[0][class_idx]
return self.characters[class_idx], confidence
def train(self, train_data, train_labels, val_data, val_labels, epochs=50):
"""训练模型"""
# 数据增强
datagen = ImageDataGenerator(
rotation_range=10,
width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=0.1,
shear_range=0.1
)
# 训练
history = self.model.fit(
datagen.flow(train_data, train_labels, batch_size=32),
epochs=epochs,
validation_data=(val_data, val_labels),
callbacks=[
EarlyStopping(patience=10, restore_best_weights=True),
ReduceLROnPlateau(factor=0.5, patience=5)
]
)
return history
CNN模型能够自动学习字符的特征表示,对变形、噪声等有更好的鲁棒性。在实际部署时,我们可以将模型转换为TensorFlow Lite或ONNX格式,以便在移动设备或嵌入式系统上运行。
5.3 后处理与错误纠正
即使单个字符识别准确率很高,整个车牌的识别结果仍可能出现错误。这时需要后处理来纠正常见错误:
class PostProcessor:
def __init__(self):
# 常见易混淆字符对
self.confusing_pairs = {
'0': 'O', 'O': '0',
'1': 'I', 'I': '1',
'2': 'Z', 'Z': '2',
'5': 'S', 'S': '5',
'8': 'B', 'B': '8'
}
# 车牌格式规则(以中国车牌为例)
self.plate_patterns = [
r'^[京津沪渝冀豫云辽黑湘皖鲁新苏浙赣鄂桂甘晋蒙陕吉闽贵粤青藏川宁琼使领][A-Z][A-Z0-9]{5}$', # 普通车牌
r'^[京津沪渝冀豫云辽黑湘皖鲁新苏浙赣鄂桂甘晋蒙陕吉闽贵粤青藏川宁琼使领][A-Z][A-Z0-9]{4}[学警港澳]$', # 特殊车牌
r'^WJ[京津沪渝冀豫云辽黑湘皖鲁新苏浙赣鄂桂甘晋蒙陕吉闽贵粤青藏川宁琼][0-9]{4}[TD]$' # 武警车牌
]
def correct(self, plate_text, char_confidences):
"""纠正识别结果"""
if not plate_text:
return plate_text
# 第一步:基于置信度纠正单个字符
corrected = list(plate_text)
for i, (char, conf) in enumerate(zip(plate_text, char_confidences)):
if conf < 0.8: # 置信度阈值
# 查找易混淆字符
if char in self.confusing_pairs:
corrected[i] = self.confusing_pairs[char]
plate_text = ''.join(corrected)
# 第二步:基于车牌规则纠正
for pattern in self.plate_patterns:
if re.match(pattern, plate_text):
return plate_text # 符合规则,无需纠正
# 第三步:尝试纠正常见错误
# 例如:第二个字符应该是字母,但被识别为数字
if len(plate_text) >= 2:
if plate_text[1].isdigit():
# 尝试用易混淆的字母替换
for digit, letter in [('0', 'O'), ('1', 'I'), ('2', 'Z'), ('5', 'S'), ('8', 'B')]:
if plate_text[1] == digit:
test_text = plate_text[0] + letter + plate_text[2:]
for pattern in self.plate_patterns:
if re.match(pattern, test_text):
return test_text
return plate_text
def validate(self, plate_text):
"""验证车牌格式"""
for pattern in self.plate_patterns:
if re.match(pattern, plate_text):
return True
return False
后处理虽然简单,但能显著提高系统的整体准确率。在实际项目中,我还会收集识别错误的样本,分析错误模式,不断优化后处理规则。
6. 系统集成与性能优化
将各个模块集成到一个完整的系统中,并优化其性能,是项目落地的关键一步。
6.1 完整的处理流水线
class LicensePlateRecognitionSystem:
def __init__(self, config_path='config/settings.py'):
# 加载配置
self.config = self._load_config(config_path)
# 初始化各个模块
self.preprocessor = ImagePreprocessor(self.config)
self.detector = LicensePlateDetector(self.config)
self.segmenter = CharacterSegmenter(self.config)
self.recognizer = CNNCharRecognizer(self.config.CNN_MODEL_PATH)
self.postprocessor = PostProcessor()
# 性能统计
self.stats = {
'total_images': 0,
'successful_detections': 0,
'successful_recognitions': 0,
'processing_times': []
}
def process_image(self, image_path):
"""处理单张图像"""
start_time = time.time()
try:
# 1. 读取图像
image = cv2.imread(image_path)
if image is None:
raise ValueError(f"无法读取图像: {image_path}")
# 2. 图像预处理
processed = self.preprocessor.preprocess(image)
# 3. 车牌检测
plates = self.detector.detect(processed)
if not plates:
return {'success': False, 'error': '未检测到车牌'}
# 4. 对每个检测到的车牌进行处理
results = []
for plate_info in plates:
plate_image = plate_info['warped_plate']
# 5. 字符分割
characters = self.segmenter.segment(plate_image)
if len(characters) < 5: # 至少5个字符
continue
# 6. 字符识别
plate_text = ''
char_confidences = []
for char_info in characters:
char_image = char_info['image']
char, confidence = self.recognizer.recognize(char_image)
plate_text += char
char_confidences.append(confidence)
# 7. 后处理
corrected_text = self.postprocessor.correct(plate_text, char_confidences)
# 8. 验证
is_valid = self.postprocessor.validate(corrected_text)
results.append({
'plate_text': corrected_text,
'confidence': np.mean(char_confidences),
'is_valid': is_valid,
'bbox': plate_info['bbox'],
'characters': len(characters)
})
# 更新统计信息
processing_time = time.time() - start_time
self.stats['total_images'] += 1
self.stats['processing_times'].append(processing_time)
if results:
self.stats['successful_detections'] += 1
valid_results = [r for r in results if r['is_valid']]
if valid_results:
self.stats['successful_recognitions'] += 1
return {
'success': True,
'results': results,
'processing_time': processing_time
}
except Exception as e:
return {
'success': False,
'error': str(e),
'processing_time': time.time() - start_time
}
def process_video(self, video_path, output_path=None, show_preview=False):
"""处理视频文件"""
cap = cv2.VideoCapture(video_path)
if not cap.isOpened():
raise ValueError(f"无法打开视频: {video_path}")
# 获取视频信息
fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# 设置输出视频
if output_path:
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))
frame_count = 0
results = []
while True:
ret, frame = cap.read()
if not ret:
break
frame_count += 1
# 每隔N帧处理一次(提高性能)
if frame_count % 5 == 0:
result = self.process_image_frame(frame)
results.append(result)
# 在帧上绘制结果
if result['success']:
for plate in result['results']:
x, y, w, h = plate['bbox']
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.putText(frame, plate['plate_text'], (x, y-10),
cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 0), 2)
# 显示预览
if show_preview:
cv2.imshow('License Plate Recognition', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# 写入输出视频
if output_path:
out.write(frame)
# 清理
cap.release()
if output_path:
out.release()
if show_preview:
cv2.destroyAllWindows()
return results
def get_performance_stats(self):
"""获取性能统计"""
if not self.stats['processing_times']:
return self.stats
self.stats['avg_processing_time'] = np.mean(self.stats['processing_times'])
self.stats['min_processing_time'] = np.min(self.stats['processing_times'])
self.stats['max_processing_time'] = np.max(self.stats['processing_times'])
self.stats['detection_rate'] = (
self.stats['successful_detections'] / self.stats['total_images']
if self.stats['total_images'] > 0 else 0
)
self.stats['recognition_rate'] = (
self.stats['successful_recognitions'] / self.stats['total_images']
if self.stats['total_images'] > 0 else 0
)
return self.stats
def _load_config(self, config_path):
"""加载配置文件"""
if os.path.exists(config_path):
spec = importlib.util.spec_from_file_location("config", config_path)
config_module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(config_module)
return config_module.Config()
else:
# 使用默认配置
from config.settings import Config
return Config()
这个完整的系统集成了所有模块,并提供了图像和视频处理的功能。在实际使用中,我发现有几个关键点需要注意:
- 异常处理:每个步骤都可能失败,需要有完善的异常处理机制
- 性能监控:记录处理时间和成功率,便于优化
- 资源管理:及时释放不再需要的资源,避免内存泄漏
6.2 性能优化技巧
在实际部署中,性能往往是关键考量因素。以下是我在实践中总结的一些优化技巧:
class OptimizedLPRSystem(LicensePlateRecognitionSystem):
def __init__(self, config_path='config/settings.py', use_gpu=False):
super().__init__(config_path)
self.use_gpu = use_gpu
self._init_optimizations()
def _init_optimizations(self):
"""初始化优化设置"""
# 设置OpenCV使用优化
cv2.setUseOptimized(True)
# 如果使用GPU,设置相关参数
if self.use_gpu:
cv2.cuda.setDevice(0)
# 尝试使用CUDA加速
try:
self.detector.net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
self.detector.net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
except:
print("CUDA不可用,回退到CPU")
# 预分配内存
self._preallocate_buffers()
def _preallocate_buffers(self):
"""预分配常用缓冲区"""
self.image_buffer = None
self.gray_buffer = None
self.edge_buffer = None
def process_image_optimized(self, image):
"""优化版的图像处理"""
# 重用缓冲区
if self.image_buffer is None or self.image_buffer.shape != image.shape:
self.image_buffer = np.empty_like(image)
# 使用in-place操作减少内存分配
cv2.resize(image, self.config.IMAGE_SIZE, dst=self.image_buffer)
# 批量处理多个ROI
plates = self.detector.detect_batch([self.image_buffer])
# 并行处理多个车牌
with ThreadPoolExecutor(max_workers=4) as executor:
futures = []
for plate in plates:
future = executor.submit(self._process_single_plate, plate)
futures.append(future)
results = [f.result() for f in futures]
return results
def _process_single_plate(self, plate_image):
"""处理单个车牌(用于并行处理)"""
# 这里可以使用更轻量级的处理流程
characters = self.segmenter.segment_fast(plate_image)
if len(characters) >= 5:
# 批量识别字符
char_images = [char['image'] for char in characters]
recognitions = self.recognizer.recognize_batch(char_images)
plate_text = ''.join([r[0] for r in recognitions])
confidences = [r[1] for r in recognitions]
corrected = self.postprocessor.correct(plate_text, confidences)
return {
'plate_text': corrected,
'confidence': np.mean(confidences),
'char_count': len(characters)
}
return None
优化技巧包括:
- 内存重用:预分配缓冲区,避免频繁的内存分配
- 并行处理:使用多线程处理多个车牌
- 批量处理:批量识别字符,减少函数调用开销
- 算法选择:根据场景选择速度或精度优先的算法
6.3 实际部署考虑
在将系统部署到生产环境时,还需要考虑以下因素:
class ProductionLPRSystem(OptimizedLPRSystem):
def __init__(self, config_path='config/settings.py',
model_dir='models/', log_dir='logs/'):
super().__init__(config_path)
# 创建必要的目录
os.makedirs(model_dir, exist_ok=True)
os.makedirs(log_dir, exist_ok=True)
# 初始化日志系统
self.logger = self._init_logger(log_dir)
# 加载或训练模型
self._ensure_models(model_dir)
# 初始化监控
self.monitor = SystemMonitor()
def _init_logger(self, log_dir):
"""初始化日志系统"""
logger = logging.getLogger('LPRSystem')
logger.setLevel(logging.INFO)
# 文件处理器
file_handler = logging.FileHandler(
os.path.join(log_dir, 'lpr_system.log')
)
file_handler.setLevel(logging.INFO)
# 控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
# 格式化
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
logger.addHandler(file_handler)
logger.addHandler(console_handler)
return logger
def _ensure_models(self, model_dir):
"""确保模型文件存在,如果不存在则训练"""
model_path = os.path.join(model_dir, 'cnn_model.h5')
if not os.path.exists(model_path):
self.logger.warning("模型文件不存在,开始训练...")
self._train_model(model_dir)
else:
self.logger.info("加载已训练的模型")
def _train_model(self, model_dir):
"""训练模型"""
# 这里应该从数据库或文件加载训练数据
# 为简化示例,我们假设已经有训练数据
pass
def process_with_fallback(self, image):
"""带降级策略的处理"""
try:
# 首先尝试使用深度学习检测器
result = self.process_image_optimized(image)
if result['success'] and result['results']:
return result
# 如果深度学习检测失败,回退到传统方法
self.logger.info("深度学习检测失败,尝试传统方法")
# 使用传统检测器
plates = self.detector.detect_traditional(image)
if plates:
# 处理检测到的车牌
results = []
for plate in plates:
plate_result = self._process_plate_traditional(plate)
if plate_result:
results.append(plate_result)
if results:
return {
'success': True,
'results': results,
'method': 'traditional'
}
# 如果都失败,返回错误
return {
'success': False,
'error': '无法识别车牌',
'method': 'both_failed'
}
except Exception as e:
self.logger.error(f"处理失败: {str(e)}")
return {
'success': False,
'error': str(e),
'method': 'error'
}
def _process_plate_traditional(self, plate_image):
"""使用传统方法处理单个车牌"""
# 这里可以实现传统的处理流程
pass
class SystemMonitor:
"""系统监控"""
def __init__(self):
self.metrics = {
'requests': 0,
'successes': 0,
'failures': 0,
'avg_response_time': 0,
'last_alert': None
}
self.alert_threshold = 0.9 # 失败率阈值
def record_request(self, success, response_time):
"""记录请求"""
self.metrics['requests'] += 1
if success:
self.metrics['successes'] += 1
else:
self.metrics['failures'] += 1
# 更新平均响应时间
old_avg = self.metrics['avg_response_time']
n = self.metrics['requests']
self.metrics['avg_response_time'] = (
old_avg * (n-1) + response_time
) / n if n > 0 else response_time
# 检查是否需要报警
self._check_alerts()
def _check_alerts(self):
"""检查系统状态,必要时报警"""
if self.metrics['requests'] > 100:
failure_rate = self.metrics['failures'] / self.metrics['requests']
if failure_rate > self.alert_threshold:
current_time = time.time()
# 避免频繁报警
if (self.metrics['last_alert'] is None or
current_time - self.metrics['last_alert'] > 3600): # 1小时
self.metrics['last_alert'] = current_time
self._send_alert(failure_rate)
def _send_alert(self, failure_rate):
"""发送报警"""
# 这里可以实现邮件、短信等报警方式
print(f"警告:系统失败率过高 ({failure_rate:.2%})")
def get_report(self):
"""获取监控报告"""
total = self.metrics['requests']
success_rate = (
self.metrics['successes'] / total if total > 0 else 0
)
return {
'total_requests': total,
'success_rate': f"{success_rate:.2%}",
'avg_response_time': f"{self.metrics['avg_response_time']:.3f}s",
'failure_rate': f"{self.metrics['failures']/total:.2%}" if total > 0 else "0%"
}
生产环境中的系统需要考虑:
- 健壮性:有降级策略,当一种方法失败时能自动切换到另一种
- 可监控:记录系统运行状态,及时发现问题
- 可维护性:良好的日志和错误处理
- 可扩展性:能够处理高并发请求
7. 测试与评估
任何计算机视觉系统都需要经过严格的测试和评估。我通常从多个维度评估系统性能:
7.1 创建测试数据集
class TestDataset:
def __init__(self, data_dir):
self.data_dir = data_dir
self.images = []
self.annotations = {}
self._load_dataset()
def _load_dataset(self):
"""加载测试数据集"""
# 支持多种格式的标注文件
annotation_files = [
os.path.join(self.data_dir, 'annotations.json'),
os.path.join(self.data_dir, 'annotations.xml'),
os.path.join(self.data_dir, 'annotations.txt')
]
for ann_file in annotation_files:
if os.path.exists(ann_file):
if ann_file.endswith('.json'):
self._load_json_annotations(ann_file)
elif ann_file.endswith('.xml'):
self._load_xml_annotations(ann_file)
elif ann_file.endswith('.txt'):
self._load_txt_annotations(ann_file)
break
# 如果没有标注文件,尝试从目录结构推断
if not self.annotations:
self._infer_from_directory()
def _load_json_annotations(self, json_file):
"""加载JSON格式的标注"""
with open(json_file, 'r', encoding='utf-8') as f:
data = json.load(f)
for item in data:
image_path = os.path.join(self.data_dir, item['image'])
if os.path.exists(image_path):
self.images.append(image_path)
self.annotations[image_path] = {
'plates': item.get('plates', []),
'difficulty': item.get('difficulty', 'medium')
}
def evaluate_system(self, system, output_dir=None):
"""评估系统性能"""
results = {
'total': len(self.images),
'detected': 0,
'recognized': 0,
'correct': 0,
'per_image': [],
'per_difficulty': {
'easy': {'total': 0, 'correct': 0},
'medium': {'total': 0, 'correct': 0},
'hard': {'total': 0, 'correct': 0}
},
'confusion_matrix': np.zeros((36, 36)), # 36个字符类别
'timing': []
}
if output_dir:
os.makedirs(output_dir, exist_ok=True)
for image_path in tqdm(self.images, desc="评估中"):
# 加载图像
image = cv2.imread(image_path)
if image is None:
continue
# 获取真实标注
ground_truth = self.annotations.get(image_path, {})
true_plates = ground_truth.get('plates', [])
difficulty = ground_truth.get('difficulty', 'medium')
# 记录难度统计
results['per_difficulty'][difficulty]['total'] += 1
# 处理图像
start_time = time.time()
result = system.process_image(image)
processing_time = time.time() - start_time
results['timing'].append(processing_time)
# 评估结果
if result['success']:
results['detected'] += 1
# 比较识别结果与真实标注
for plate_result in result['results']:
recognized_text = plate_result['plate_text']
# 查找匹配的真实车牌
matched = False
for true_plate in true_plates:
if self._compare_plates(recognized_text, true_plate['text']):
matched = True
results['correct'] += 1
results['per_difficulty'][difficulty]['correct'] += 1
break
if matched:
results['recognized'] += 1
# 记录每张图像的结果
image_result = {
'image': os.path.basename(image_path),
'detected': len(result['results']) > 0,
'recognized': any(r['is_valid'] for r in result['results']),
'correct': matched,
'processing_time': processing_time,
'ground_truth': true_plates,
'predictions': result['results']
}
results['per_image'].append(image_result)
# 保存可视化结果
if output_dir and result['success']:
self._save_visualization(
image, result, ground_truth,
os.path.join(output_dir, os.path.basename(image_path))
)
# 计算指标
results['detection_rate'] = results['detected'] / results['total']
results['recognition_rate'] = results['recognized'] / results['total']
results['accuracy'] = results['correct'] / results['total']
results['avg_processing_time'] = np.mean(results['timing'])
results['std_processing_time'] = np.std(results['timing'])
# 计算每个难度的准确率
for difficulty in results['per_difficulty']:
total = results['per_difficulty'][difficulty]['total']
correct = results['per_difficulty'][difficulty]['correct']
if total > 0:
results['per_difficulty'][difficulty]['accuracy'] = correct / total
else:
results['per_difficulty'][difficulty]['accuracy'] = 0
return results
def _compare_plates(self, pred_text, true_text):
"""比较识别结果与真实标注"""
# 简单的字符串比较
if pred_text == true_text:
return True
# 允许一些常见错误
confusing_map = {'0': 'O', 'O': '0', '1': 'I', 'I': '1'}
if len(pred_text) == len(true_text):
# 逐个字符比较,允许易混淆字符
mismatches = 0
for p, t in zip(pred_text, true_text):
if p != t:
if p in confusing_map and confusing_map[p] == t:
continue
elif t in confusing_map and confusing_map[t] == p:
continue
else:
mismatches += 1
# 允许最多1个非易混淆字符错误
return mismatches <= 1
return False
def _save_visualization(self, image, result, ground_truth, output_path):
"""保存可视化结果"""
vis_image = image.copy()
# 绘制检测框
for plate in result['results']:
x, y, w, h = plate['bbox']
color = (0, 255, 0) if plate['is_valid'] else (0, 0, 255)
cv2.rectangle(vis_image, (x, y), (x+w, y+h), color, 2)
# 绘制识别文本
text = f"{plate['plate_text']} ({plate['confidence']:.2f})"
cv2.putText(vis_image, text, (x, y-10),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, color, 2)
# 绘制真实标注(如果存在)
for true_plate in ground_truth.get('plates', []):
x, y, w, h = true_plate['bbox']
cv2.rectangle(vis_image, (x, y), (x+w, y+h), (255, 0, 0), 2)
cv2.putText(vis_image, true_plate['text'], (x, y+h+20),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 0, 0), 2)
# 保存图像
cv2.imwrite(output_path, vis_image)
7.2 性能指标分析
评估一个车牌识别系统时,
5万+



被折叠的 条评论
为什么被折叠?



