自定义ava数据集及训练与测试 crowded human visible body-优快云博客

本文链接：https://blog.youkuaiyun.com/WhiffeYF/article/details/124686012

1 安装Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset

apt-get update
apt-get install zip
apt-get install unzip

conda install x264 ffmpeg -c conda-forge -y

cd /home
git clone https://gitee.com/YFwinston/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset.git

2 视频裁剪

在视频裁剪前，需要准备相对应的视频，我的视频放在了/user-data/studentVideo/videos下，执行下面命令，让复制视频到指定位置：

cp -r /user-data/studentVideo/videos/* /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/videos/

视频放在/home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/videos中

视频取名：10001，前2位数（10～99）代表视频大类，后3位数（001～999）代表该大类视频下收集的各个视频。

裁剪的视频取名：1000101，前2位数（10～99）代表视频大类，中间3位数（001～999）代表该大类视频下收集的各个视频，最后2位数（01～99）代表某一视频裁剪的编号。

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
bash cutVideos.sh ../Dataset/videos ../Dataset/video_crop ./cutVideos.txt

3 视频抽帧

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection
bash cut_frames.sh ../Dataset/video_crop ../Dataset/frames

4 选帧

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection
python chooseVideoFrame.py --seconds 15 --frames_dir ../Dataset/frames --chooseFrame_dir ./chooseVideoFrame/

5 yoloV5检测

首先安装yoloV5

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
pip install -r requirements.txt
pip install opencv-python-headless==4.1.2.30

mkdir -p /root/.config/Ultralytics/
cp /user-data/yolov5File/crowdhuman_vbody_yolov5m.pt /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/crowdhuman_vbody_yolov5m.pt 
cp /user-data/yolov5File/Arial.ttf /root/.config/Ultralytics/Arial.ttf

检测前先删除以前的内容

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection/chooseVideoFrameYolov5
rm -r *

开始检测

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source ../imageDetection/chooseVideoFrame/ --save-txt --save-conf  --weights ./yolov5/crowdhuman_vbody_yolov5m.pt --hide-labels --line-thickness 2 --project ../imageDetection/chooseVideoFrameYolov5

6 yoloV5转via

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
python yolo2via.py --yoloLabel_dir ./chooseVideoFrameYolov5/exp/labels --image_dir ./chooseVideoFrame

压缩检测结果：

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
rm imageDetection.zip 
zip -r imageDetection.zip  ./chooseVideoFrame

7 via对生成内容微调

微调后的文件名：detection2.json
存放在：/home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection/chooseVideoFrame下面

8 via转yoloV5训练格式

参考：手把手教你使用YOLOV5训练自己的目标检测模型

创建yoloV5所使用的训练文件夹

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
mkdir -p ./chooseVideoFrameYolov5/yolov5CusDataset/labels/{train,test,val}
mkdir -p ./chooseVideoFrameYolov5/yolov5CusDataset/images/{train,test,val}

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
python via2yoloTrain.py --anno_dir ./chooseVideoFrame/detection2.json --img_dir ./chooseVideoFrame/ --newLabels_dir ./chooseVideoFrameYolov5/yolov5CusDataset/labels/ --newImages_dir ./chooseVideoFrameYolov5/yolov5CusDataset/images/

9 制作预训练模型

参考资料：
1，制作自己的Detectron2预训练模型：https://zhuanlan.zhihu.com/p/147336249
2，ModuleNotFoundError: No module named ‘models‘解决torch.load问题: https://blog.youkuaiyun.com/weixin_42815846/article/details/115289861

使用的训练模型：crowdhuman_vbody_yolov5m.pt
我将该模型放在了：/user-data/yolov5File/crowdhuman_vbody_yolov5m.pt

创建文件夹：

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
mkdir pretainModel

删除之前的训练文件

rm -r /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/train/*

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5
python makePretrainModel.py --weight_dir /user-data/yolov5File/crowdhuman_vbody_yolov5m.pt --preWeight_dir ../../imageDetection/pretainModel/crowdhuman_vbody_yolov5m_pretrained.pt

10 训练

cd  /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5
python train.py --data ./data/student.yaml --cfg ./models/yolov5m.yaml --weights ../../imageDetection/pretainModel/crowdhuman_vbody_yolov5m_pretrained.pt --batch-size 16 --epochs 200

11 采用重训练模型检测

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
rm -r ./yolov5/runs/detect/*
python ./yolov5/detect.py --source ../imageDetection/chooseVideoFrame --save-txt --save-conf  --weights ./yolov5/runs/train/exp/weights/best.pt --hide-labels --line-thickness 2

12 检测结果导入via

13 传统筛选

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
mkdir -p ./chooseVideoFrameYolov5/newExp/
mkdir -p ./visualize
rm -r  ./chooseVideoFrameYolov5/newExp/*
python filter.py --label_dir ./chooseVideoFrameYolov5/exp/labels --image_dir ./chooseVideoFrame/ --newExp_dir ./chooseVideoFrameYolov5/newExp/

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
mkdir -p ./chooseVideoFrameYolov5/newExp/
mkdir -p ./visualize
rm -r  ./chooseVideoFrameYolov5/newExp/*
rm -r  ./visualize/*
python filter2.py --label_dir ./chooseVideoFrameYolov5/exp/labels --image_dir ./chooseVideoFrame/ --newExp_dir ./chooseVideoFrameYolov5/newExp/

rm x.zip
zip -r x.zip ./visualize/

14 筛选后转via

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDetection
python yolo2via.py --yoloLabel_dir ./chooseVideoFrameYolov5/newExp/ --image_dir ./chooseVideoFrame --labelName_dir detections.json

安装

首先准备好数据集：
/user-data/studentVideo/clips下放着裁剪好的所有视频：10.avi到99.avi
/user-data/yolov5File/下准备：Arial.ttf crowdhuman_vbody_yolov5m.pt

apt-get update
apt-get install zip
apt-get install unzip

conda install x264 ffmpeg -c conda-forge -y

cd /home
git clone https://gitee.com/YFwinston/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset.git

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
bash cut_frames_s.sh

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
python choose_frames_all.py 15 0
python choose_frames.py 15 0

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
pip install -r requirements.txt
pip install opencv-python-headless==4.1.2.30

mkdir -p /root/.config/Ultralytics/
cp /user-data/yolov5File/crowdhuman_vbody_yolov5m.pt /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/crowdhuman_vbody_yolov5m.pt 
cp /user-data/yolov5File/Arial.ttf /root/.config/Ultralytics/Arial.ttf

rm -r /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/detect

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source ../Dataset/choose_frames_all/ --save-txt --save-conf  --weights ./yolov5/crowdhuman_vbody_yolov5m.pt --hide-labels --line-thickness 2

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/mywork
python dense_proposals_train_s.py ../yolov5/runs/detect/exp/labels ./dense_proposals_train.pkl show

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/
python choose_frames_middle.py

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/mywork/
python dense_proposals_train_to_via_s.py ./dense_proposals_train.pkl ../../Dataset/choose_frames_middle/

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
python chang_via_json_s.py 

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
zip -r choose_frames_middle.zip choose_frames_middle

选取

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection
python chooseVideoFrame.py 15 0

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source ../imageDatasetDetection/chooseVideoFrame/ --save-txt --save-conf  --weights ./yolov5/crowdhuman_vbody_yolov5m.pt --hide-labels --line-thickness 2 --project ../imageDatasetDetection/chooseVideoFrameYolov5

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection
python SingleDisplayDetectionBox.py

zip -r SingleDisplayDetectionBox.zip ./SingleDisplayDetectionBox

mkdir -p /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection/chooseVideoFrameYolov5/newExp

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source ../Dataset/choose_frames_all/ --save-txt --save-conf  --weights ./yolov5/crowdhuman_vbody_yolov5m.pt --hide-labels --line-thickness 2

筛选

python filter.py

重训练

参考资料

制作自己的Detectron2预训练模型：https://zhuanlan.zhihu.com/p/147336249
ModuleNotFoundError: No module named ‘models‘解决torch.load问题: https://blog.youkuaiyun.com/weixin_42815846/article/details/115289861

/user-data/yolov5File/crowdhuman_vbody_yolov5m.pt

via2yolo

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection/
mkdir -p ./chooseVideoFrameYolov5/yolov5CusDataset/images/{test,train,val}
mkdir -p ./chooseVideoFrameYolov5/yolov5CusDataset/labels/{test,train,val}

制作预训练模型

文件缺失，后期加到cus中去
在这里插入图片描述

cd  /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5
python train.py --data ./data/crowdhuman.yaml --cfg ./models/yolov5m.yaml --weights /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection/pretainModel/crowdhuman_vbody_yolov5m_pretrained.pt --batch-size 16 --epochs 200

python train.py --data ./data/crowdhuman.yaml --cfg ./models/yolov5m.yaml  --weights ./pretrained/yolov5m.pt  --batch-size 16 --epochs 200

采用重训练模型检测

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/imageDatasetDetection/chooseVideoFrame --save-txt --save-conf  --weights /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/train/exp15/weights/best.pt --hide-labels --line-thickness 2

检测结果导入via

mkdir -p /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/detect/newExp

采用jupyer notebook

import sys
import os
import json
import pickle
import cv2  
import numpy as np
import codecs

import matplotlib.pyplot as plt

font = cv2.FONT_HERSHEY_COMPLEX_SMALL

#坐标格式转化 xywh代表：中心点与宽长，xyxy代表左上角点与右下角点
def xywhToxyxy(box):
    temp = box
    temp[0] = float(box[0]) - float(box[2]) / 2  # top left x
    temp[1] = float(box[1]) - float(box[3]) / 2  # top left y
    temp[2] = float(box[0]) + float(box[2])  # bottom right x
    temp[3] = float(box[1]) + float(box[3])  # bottom right y
    return

#可视化
def VisualizeBoxPlt(box1,box2,name1,name2,title, imgTag=False, path = ''):
    
    # 设置plt大小
    plt.rcParams['figure.figsize'] = (16.0, 16.0)
    
    #设置画布的大小
    width = 1920
    height = 1080
    if imgTag:
        img = cv2.imread(path)
        
    else:
        img = np.zeros((height, width, 3), np.uint8)
    
    image = cv2.rectangle(img, (int(box1[0]*width),int(box1[1]*height)), (int(box1[2]*width),int(box1[3]*height)), (0,0,255), 4) 
    image = cv2.putText(img, name1, (int(box1[0]*width),int(box1[1]*height)+15), font, 2, (0, 0, 255), 2)
    image = cv2.rectangle(img, (int(box2[0]*width),int(box2[1]*height)), (int(box2[2]*width),int(box2[3]*height)), (0,255,0), 4) 
    image = cv2.putText(img, name2, (int(box2[2]*width),int(box2[3]*height)), font, 2, (0,255,0), 2)
    image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    plt.title(title)
    plt.imshow(image)
    plt.show()
    plt.close()
    
    
def compareArea(box1,box2):
    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
    if box1_area <= box2_area:
        return 1
    else:
        return 0

#筛选出可能异常的检测框 ，通过算法检测后，确定为异常框，返回1
def filterAbnormalBox(headAr, abnormalBox):
    for i,lineHead in enumerate(headAr):
        headBox = [float(lineHead[1]),float(lineHead[2]),float(lineHead[3]),float(lineHead[4])]
        xywhToxyxy(headBox)

        # 下面四个if判断是去除没有交集的box
        if abs(headBox[0] - abnormalBox[0]) > max(headBox[2]-headBox[0],abnormalBox[2]-abnormalBox[0]) :
            continue
        if abs(headBox[1] - abnormalBox[1]) > max(headBox[3]-headBox[1],abnormalBox[3]-abnormalBox[1]) :
            continue
        if abs(headBox[2] - abnormalBox[2]) > max(headBox[2]-headBox[0],abnormalBox[2]-abnormalBox[0]) :    
            continue
        if abs(headBox[3] - abnormalBox[3]) >  max(headBox[3]-headBox[1],abnormalBox[3]-abnormalBox[1]) :
            continue

        xi1 = max(headBox[0], abnormalBox[0])
        yi1 = max(headBox[1], abnormalBox[1])
        xi2 = min(headBox[2], abnormalBox[2])
        yi2 = min(headBox[3], abnormalBox[3])

        inter_area = (yi2 - yi1) * (xi2 - xi1)

        if inter_area <= 0:
            continue

        headBox_area = (headBox[2] - headBox[0]) * (headBox[3] - headBox[1])
        abnormalBox_area = (abnormalBox[2] - abnormalBox[0]) * (abnormalBox[3] - abnormalBox[1])

        min_box_area = min(headBox_area,abnormalBox_area)

        r_area = inter_area/min_box_area
                
        #可视化
        VisualizeBoxPlt(headBox,abnormalBox,'headBox','abnormalBox','2 The headBox around the abnormalBox')
        
        if r_area > 0.4:
            
            r_area2 = headBox_area/abnormalBox_area

            if r_area2 > 0.5 and r_area2 < 1.5:
                
                #可视化
                VisualizeBoxPlt(headBox,abnormalBox,'headBox','abnormalBox','3 Determine the filter abnormalBox')
                
                return 1
            
    return 0
            

def r_filter(box1, box2, headAr):
    
    # 下面四个if判断是去除没有交集的box
    if abs(box1[0] - box2[0]) > max(box1[2]-box1[0],box2[2]-box2[0]) :
        return 0
    if abs(box1[1] - box2[1]) > max(box1[3]-box1[1],box2[3]-box2[1]) :
        return 0
    if abs(box1[2] - box2[2]) > max(box1[2]-box1[0],box2[2]-box2[0]) :    
        return 0
    if abs(box1[3] - box2[3]) >  max(box1[3]-box1[1],box2[3]-box2[1]) :
        return 0
    
    # 只留下有包含关系的box中：其中一个box位于另一个box的顶部!，顶部！
    #if abs(box1[1] - box2[1]) > max(box1[3]-box1[1],box2[3]-box2[1])*0.4:
    #    return 0
    
    xi1 = max(box1[0], box2[0])
    yi1 = max(box1[1], box2[1])
    xi2 = min(box1[2], box2[2])
    yi2 = min(box1[3], box2[3])
    
    inter_area = (yi2 - yi1) * (xi2 - xi1)
    
    
    if inter_area <= 0:
        return 0
    
    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
    
    min_box_area = min(box1_area,box2_area)
    
    if min_box_area <= inter_area:
        return 0
    
    r_area = inter_area/min_box_area
    
    boolHead = 0
    
    
    
    
    if r_area>0.7:      
        
        #可视化
        VisualizeBoxPlt(box1,box2,'box1','box2','1 Find boxes that may be abnormal')
        
        if box1_area < box2_area:
            boolHead = filterAbnormalBox(headAr, box1)
        else:
            boolHead = filterAbnormalBox(headAr, box2)
        
    if boolHead == 0:
        return 0
    
    return r_area

# 需要筛选的检测标签数据位置
labelPath = '../yolov5/runs/detect/exp/labels'
print(labelPath)
for root, dirs, files in os.walk(labelPath):
    if root == labelPath:
        
        #排序，防止10排在1的前面
        files.sort(key=lambda arr: (int(arr[:-7]), int(arr[3:-4])))
        for filename in files:
            print(filename)
            #读取txt中的信息
            temp_txt=open(os.path.join(root, filename))
            temp_data_txt = temp_txt.readlines() 
            
            # 存放身体坐标与头部信息
            vbodyAr = []
            headAr = []
            
            #通过循环本次txt文件的坐标，取出头部与身体坐标
            for lineData in temp_data_txt:
                
                # 只要人的信息
                eLineData = lineData.split(' ')
                if eLineData[0]=='0':
                    headAr.append(eLineData)
                    
                # 只要头的信息
                if eLineData[0]=='1':
                    vbodyAr.append(eLineData)
                    
            # 存放新txt文件
            new_data_txt = []
            
            #获取vbodyAr的长度
            lenB = len(vbodyAr)
            s_count=0
            
            # 再次循环，通过多层算法筛选，去掉异常检测框
            for i,lineData in enumerate(temp_data_txt):
                
                #delteE代表是否删除检测框，1代表删除，0代表不删除
                delteE = 0
                
                # 当前的检测框逐个对比其它检测框
                for j in range(i+1, lenB, 1):
                    
                    box1 = []
                    box2 = []
                    eLineData = lineData.split(' ')
                    
                    # 只进行身体检测框的筛选
                    if eLineData[0]=='1':
                        
                        #提取出两个检测框的坐标
                        box1 = [float(eLineData[1]),float(eLineData[2]),float(eLineData[3]),float(eLineData[4])]
                        box2 = [float(vbodyAr[j][1]),float(vbodyAr[j][2]),float(vbodyAr[j][3]),float(vbodyAr[j][4])]
                        
                        #坐标格式转化
                        xywhToxyxy(box1)
                        xywhToxyxy(box2)
                        
                        filter = r_filter(box1,box2,headAr)
                        if filter > 0.7:
                            s_count = s_count+1
                            
                            
                            dirImage = '/home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/choose_frames/'
                            dirFile = filename.split('_')[0]
                            dirFileName = filename.split('.')[0]+'.jpg'
                            path = dirImage + dirFile + '/' + dirFileName
                            
                            
                            
                            
                            
                            # 当box1的面积小于等于box2时，该坐标就该删除了
                            if compareArea(box1,box2) == 1:
                                delteE = 1
                                # 可视化
                                VisualizeBoxPlt(box1,box2,'box1','box2','4 Visualize the results of a successful filter', imgTag=True, path = path)
                                break
                                                        

                if delteE == 0:
                    new_data_txt.append(lineData) 
            print("new:",len(new_data_txt))
            print("old:",len(temp_data_txt))
            
            newExpDir = "/home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/detect/newExp/" + filename
            
            f = codecs.open(newExpDir,'w')

            for i in new_data_txt:
                f.write(str(i)) 
            f.close()
            
            #with open(newExpDir,"w") as f:
            #    f.write(str(new_data_txt))  # 自带文件关闭功能，不需要再写f.close()

import sys
import os
import json
import pickle
import cv2  
import numpy as np

import matplotlib.pyplot as plt

font = cv2.FONT_HERSHEY_COMPLEX_SMALL

def xywhToxyxy(box):
    temp = box
    temp[0] = float(box[0]) - float(box[2]) / 2  # top left x
    temp[1] = float(box[1]) - float(box[3]) / 2  # top left y
    temp[2] = float(box[0]) + float(box[2])  # bottom right x
    temp[3] = float(box[1]) + float(box[3])  # bottom right y
    return

def headBody(headAr, box):
    for i,lineHead in enumerate(headAr):
        headBox = [float(lineHead[1]),float(lineHead[2]),float(lineHead[3]),float(lineHead[4])]
        xywhToxyxy(headBox)

        # 下面四个if判断是去除没有交集的box
        if abs(headBox[0] - box[0]) > max(headBox[2]-headBox[0],box[2]-box[0]) :
            continue
        if abs(headBox[1] - box[1]) > max(headBox[3]-headBox[1],box[3]-box[1]) :
            continue
        if abs(headBox[2] - box[2]) > max(headBox[2]-headBox[0],box[2]-box[0]) :    
            continue
        if abs(headBox[3] - box[3]) >  max(headBox[3]-headBox[1],box[3]-box[1]) :
            continue

        xi1 = max(headBox[0], box[0])
        yi1 = max(headBox[1], box[1])
        xi2 = min(headBox[2], box[2])
        yi2 = min(headBox[3], box[3])

        inter_area = (yi2 - yi1) * (xi2 - xi1)

        if inter_area <= 0:
            continue

        headBox_area = (headBox[2] - headBox[0]) * (headBox[3] - headBox[1])
        box_area = (box[2] - box[0]) * (box[3] - box[1])

        min_box_area = min(headBox_area,box_area)

        #if min_box_area <= inter_area:
        #    print("xxxxxx")
        #    print(inter_area/min_box_area)
        #    continue

        r_area = inter_area/min_box_area
        
        
        #可视化
        plt.rcParams['figure.figsize'] = (16.0, 16.0)
        img = np.zeros((1080, 1920, 3), np.uint8)
        image = cv2.rectangle(img, (int(headBox[0]*1920),int(headBox[1]*1080)), (int(headBox[2]*1920),int(headBox[3]*1080)), (255,0,0), 4) 
        image = cv2.putText(img, '1 headBox', (int(headBox[2]*1920),int(headBox[3]*1080)), font, 2, (255, 0, 0), 2)
        image = cv2.rectangle(img, (int(box[0]*1920),int(box[1]*1080)), (int(box[2]*1920),int(box[3]*1080)), (255,250,205), 4) 
        image = cv2.putText(img, '1 box', (int(box[2]*1920),int(box[3]*1080)), font, 2, (255,250,205), 2)
        plt.imshow(image)
        plt.show()
        
        if r_area > 0.5:
            r_area2 = headBox_area/box_area
            print("r_area2:",r_area2)
            if r_area2 > 0.6 and r_area2 < 1.4:
                #可视化
                plt.rcParams['figure.figsize'] = (16.0, 16.0)
                img = np.zeros((1080, 1920, 3), np.uint8)
                image = cv2.rectangle(img, (int(headBox[0]*1920),int(headBox[1]*1080)), (int(headBox[2]*1920),int(headBox[3]*1080)), (255, 0, 0), 2) 
                image = cv2.putText(img, '2 headBox', (int(headBox[2]*1920),int(headBox[3]*1080)), font, 2, (255, 0, 0), 2)
                image = cv2.rectangle(img, (int(box[0]*1920),int(box[1]*1080)), (int(box[2]*1920),int(box[3]*1080)), (255,250,205), 2) 
                image = cv2.putText(img, '2 box', (int(box[2]*1920),int(box[3]*1080)), font, 2, (255,250,205), 2)
                plt.imshow(image)
                plt.show()
                print("r_area2:",r_area2)
                return 1
    return 0
            

def inter_area(box1, box2, headAr):
    
    # 下面四个if判断是去除没有交集的box
    if abs(box1[0] - box2[0]) > max(box1[2]-box1[0],box2[2]-box2[0]) :
        return 0
    if abs(box1[1] - box2[1]) > max(box1[3]-box1[1],box2[3]-box2[1]) :
        return 0
    if abs(box1[2] - box2[2]) > max(box1[2]-box1[0],box2[2]-box2[0]) :    
        return 0
    if abs(box1[3] - box2[3]) >  max(box1[3]-box1[1],box2[3]-box2[1]) :
        return 0
    
    # 只留下有包含关系的box中：其中一个box位于另一个box的顶部!，顶部！
    if abs(box1[1] - box2[1]) > 0.01:
        return 0
    
    xi1 = max(box1[0], box2[0])
    yi1 = max(box1[1], box2[1])
    xi2 = min(box1[2], box2[2])
    yi2 = min(box1[3], box2[3])
    
    inter_area = (yi2 - yi1) * (xi2 - xi1)
    
    
    if inter_area <= 0:
        return 0
    
    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
    
    min_box_area = min(box1_area,box2_area)
    
    if min_box_area <= inter_area:
        return 0
    
    r_area = inter_area/min_box_area
    
    boolHead = 0
    
    
    
    
    if r_area>0.95:
    
        #可视化
        plt.rcParams['figure.figsize'] = (16.0, 16.0)
        img = np.zeros((1080, 1920, 3), np.uint8)
        image = cv2.rectangle(img, (int(box1[0]*1920),int(box1[1]*1080)), (int(box1[2]*1920),int(box1[3]*1080)), (255,250,205), 4) 
        image = cv2.putText(img, 'box1', (int(box1[2]*1920),int(box1[3]*1080)), font, 2, (255,250,205), 2)
        image = cv2.rectangle(img, (int(box2[0]*1920),int(box2[1]*1080)), (int(box2[2]*1920),int(box2[3]*1080)), (0,255,255), 4) 
        image = cv2.putText(img, 'box2', (int(box2[2]*1920),int(box2[3]*1080)), font, 2, (0,255,255), 2)
        
        plt.imshow(image)
        plt.show()
        
        
        if box1_area < box2_area:
            boolHead = headBody(headAr, box1)
        else:
            boolHead = headBody(headAr, box2)
        
    if boolHead == 0:
        return 0
    
    return r_area
    
labelPath = '../yolov5/runs/detect/exp/labels'

for root, dirs, files in os.walk(labelPath):
    if root == labelPath:
        #排序，防止10排在1的前面
        files.sort(key=lambda arr: (int(arr[:-7]), int(arr[3:-4])))
        for filename in files:
            print(filename)
            #读取yolov5中的信息
            temp_txt=open(os.path.join(root, filename))
            temp_data_txt = temp_txt.readlines() 
            
            # 存放身体坐标与头部信息
            vbodyAr = []
            headAr = []
            for lineData in temp_data_txt:
                # 只要人的信息
                eLineData = lineData.split(' ')
                if eLineData[0]=='0':
                    headAr.append(eLineData)
                if eLineData[0]=='1':
                    vbodyAr.append(eLineData)
                    
            
            # type(results[0][1]) <class 'str'>
            lenB = len(vbodyAr)
            s_count=0
            for i,lineData in enumerate(temp_data_txt):
                for j in range(i+1, lenB, 1):
                    box1 = []
                    box2 = []
                    eLineData = lineData.split(' ')
                    if eLineData[0]=='1':
                        box1 = [float(eLineData[1]),float(eLineData[2]),float(eLineData[3]),float(eLineData[4])]
                        box2 = [float(vbodyAr[j][1]),float(vbodyAr[j][2]),float(vbodyAr[j][3]),float(vbodyAr[j][4])]
                        

                        xywhToxyxy(box1)
                        xywhToxyxy(box2)
                        

                        r_area = inter_area(box1,box2,headAr)
                        if r_area > 0.95:
                            s_count = s_count+1
                            #print(box1[0]*1920,box1[1]*1080,box1[2]*1920,box1[3]*1080)
                            #print(box2[0]*1920,box2[1]*1080,box2[2]*1920,box2[3]*1080)
                            

                            # 可视化
                            dirImage = '/home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/choose_frames/'
                            dirFile = filename.split('_')[0]
                            dirFileName = filename.split('.')[0]+'.jpg'
                            path = dirImage + dirFile + '/' + dirFileName
                            image = cv2.imread(path)
                            #print(path)

                            image = cv2.rectangle(image, (int(box1[0]*1920),int(box1[1]*1080)), (int(box1[2]*1920),int(box1[3]*1080)), (255, 0, 255), 2) 
                            image = cv2.rectangle(image, (int(box2[0]*1920),int(box2[1]*1080)), (int(box2[2]*1920),int(box2[3]*1080)), (255, 0, 0), 2) 

                            OutPath = './temp/'+filename.split('.')[0] +'_' + str(s_count) +'.jpg'
                            cv2.imwrite(OutPath, image)

apt-get update
apt-get install zip
apt-get install unzip

cd /home
git clone https://gitee.com/YFwinston/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset.git

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
bash cut_frames_s.sh


python choose_frames_all.py 15 0
python choose_frames.py 15 0

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
pip install -r requirements.txt
pip install opencv-python-headless==4.1.2.30

mkdir -p /root/.config/Ultralytics/
cp /user-data/yolov5File/crowdhuman_vbody_yolov5m.pt /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/crowdhuman_vbody_yolov5m.pt 
cp /user-data/yolov5File/Arial.ttf /root/.config/Ultralytics/Arial.ttf

rm -r /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/yolov5/runs/detect

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort
python ./yolov5/detect.py --source ../Dataset/choose_frames_all/ --save-txt --save-conf  --weights ./yolov5/crowdhuman_vbody_yolov5m.pt --hide-labels --line-thickness 2


xxxxxxxxx


cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/mywork
python dense_proposals_train_s.py ../yolov5/runs/detect/newExp ./dense_proposals_train.pkl show

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset/
python choose_frames_middle.py

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/yolovDeepsort/mywork/
python dense_proposals_train_to_via_s.py ./dense_proposals_train.pkl ../../Dataset/choose_frames_middle/

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
python chang_via_json_s.py 

cd /home/Custom-ava-dataset_Custom-Spatio-Temporally-Action-Video-Dataset/Dataset
zip -r choose_frames_middle.zip choose_frames_middle