999999_id %= 999999-优快云博客

</svg>

<a href="https://so.youkuaiyun.com/so/search?q=%E5%AE%9E%E4%BE%8B%E5%88%86%E5%89%B2&spm=1001.2101.3001.7020" target="_blank" class="hl hl-1" data-report-click="{"spm":"1001.2101.3001.7020","dest":"https://so.youkuaiyun.com/so/search?q=%E5%AE%9E%E4%BE%8B%E5%88%86%E5%89%B2&spm=1001.2101.3001.7020","extra":"{\"searchword\":\"实例分割\"}"}" data-tit="实例分割" data-pretit="实例分割">实例分割</a>可以分为两类，一类为two-stage方法，即先检测后分割；另一类为one-stage方法，将检测与分割作为并行任务同时进行，其中包括anchor-based方法与anchor-free方法。yolact属于one-stage方法中的anchor-based方法。该方法主要采用两个并行的分支实现one-stage检测与分割。两个分支分别为：

(1)Prediction Head分支生成各个<a href="https://so.youkuaiyun.com/so/search?q=anchor&spm=1001.2101.3001.7020" target="_blank" class="hl hl-1" data-report-click="{"spm":"1001.2101.3001.7020","dest":"https://so.youkuaiyun.com/so/search?q=anchor&spm=1001.2101.3001.7020","extra":"{\"searchword\":\"anchor\"}"}" data-tit="anchor" data-pretit="anchor">anchor</a>的类别置信度、位置回归参数以及mask的掩码系数； (2)Protonet分支生成一组原型mask。然后将原型mask和mask的掩码系数相乘，从而得到图片中每一个目标物体的mask。

</blockquote>

<h1><a name="t1"></a><a id="_5"></a>一、开发环境配置</h1>

系统：Windows10；开发语言：Python3.8; 训练框架：Pytorch; 开发IDE: Pycharm

</blockquote>

<h1><a name="t2"></a><a id="yolact_9"></a>二、yolact实例分割效果</h1>

下图为实例分割分割效果，通过两幅图可以发现，yolact不仅将两幅图像中的目标进行检测，同时输出物体的分割Mask. <img src="https://img-blog.csdnimg.cn/20210504164932391.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="实例分割测试1"> <img src="https://img-blog.csdnimg.cn/20210504165617143.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="实例分割测试2">

<h1><a name="t3"></a><a id="yolact_13"></a>三、yolact实例分割测试</h1>

yolact paper提供了<a href="" target="_blank">https://github.com/dbolya/yolact">源码</a>,对其结果进行测试。

1、解压缩后如下图所示： <img src="https://img-blog.csdnimg.cn/20210504170414819.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 2、下载预训练权重<a href="yolact_base_54_800000.pth
" target="_blank">https://ucdavis365-my.sharepoint.com/personal/yongjaelee_ucdavis_edu/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fyongjaelee%5Fucdavis%5Fedu%2FDocuments%2Fyolact%5Fbase%5F54%5F800000%2Epth&parent=%2Fpersonal%2Fyongjaelee%5Fucdavis%5Fedu%2FDocuments&originalPath=aHR0cHM6Ly91Y2RhdmlzMzY1LW15LnNoYXJlcG9pbnQuY29tLzp1Oi9nL3BlcnNvbmFsL3lvbmdqYWVsZWVfdWNkYXZpc19lZHUvRVlSV3hCRW9LVTlEaWJscld4Mk04OU1CR0ZrVlZCX2RybFJkX3Y1c2RUM0hnZz9ydGltZT1Xd01OdTl3TzJVZw">yolact_base_54_800000.pth</a> <img src="https://img-blog.csdnimg.cn/20210504170554937.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 3.image、video算法测试

<ul><li>image测试</li></ul>

<pre data-index="0" class="prettyprint"><code class="prism language-python has-numbering" οnclick="mdcp.copyCode(event)" style="position: unset;">python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --image=my_image.png

其中train_model为预训练权重；score_threshold为置信度阈值，对置信度小于0.15的物体bounding box进行滤除。top_k为保留每个类别置信度排名前15个score。image为需要测试的图片。

测试结果如下： <img src="https://img-blog.csdnimg.cn/20210504171521734.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述">

<ul><li>video测试</li></ul>

<pre data-index="1" class="prettyprint"><code class="prism language-python has-numbering" οnclick="mdcp.copyCode(event)" style="position: unset;">python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --video_multiframe=1 --video=my_video.mp4

其中trained_model为训练权重，video_multiframe为并行输出的视频路数。video为需要测试的视频。

测试结果如下。 <img src="https://img-blog.csdnimg.cn/20210504195817452.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70#pic_center" alt="在这里插入图片描述">

<h1><a name="t4"></a><a id="_39"></a>四、训练自己的数据集</h1>

1、数据集制作 使用labelme进行实例分割数据标注，labelme在终端中输入如下命令进行labelme安装

<pre data-index="2" class="prettyprint"><code class="prism language-python has-numbering" οnclick="mdcp.copyCode(event)" style="position: unset;">pip install labelme

在终端中输入<code>labelme</code>启动labelme标注工具。 <img src="https://img-blog.csdnimg.cn/20210504203753569.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70#pic_center" alt="在这里插入图片描述"> 该图中包含四个实例，label均为trafficcone.新建labelme文件夹并将标注好的json文件与image图片一起放入labelme文件夹中。 <img src="https://img-blog.csdnimg.cn/20210504204011215.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> <img src="https://img-blog.csdnimg.cn/20210504204049497.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 在yolact-master文件夹下新建<code>labelme2coco.py</code> <img src="https://img-blog.csdnimg.cn/20210504204327324.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 其中<code>labelme2coco.py</code>代码如下：

<pre data-index="3" class="set-code-hide prettyprint"><code class="prism language-python has-numbering" οnclick="mdcp.copyCode(event)" style="position: unset;">import os

import json

import numpy as np

import glob

import shutil

from sklearn.model_selection import train_test_split

np.random.seed(41)

#0为背景，此处根据你数据集的类别来修改key

classname_to_id = {"trafficone": 1}

class Lableme2CoCo:

def __init__(self):

self.images = []

self.annotations = []

self.categories = []

self.img_id = 0

self.ann_id = 0

def save_coco_json(self, instance, save_path):

json.dump(instance, open(save_path, 'w', encoding='utf-8'), ensure_ascii=False, indent=1) # indent=2 更加美观显示

# 由json文件构建COCO

def to_coco(self, json_path_list):

self._init_categories()

for json_path in json_path_list:

obj = self.read_jsonfile(json_path)

self.images.append(self._image(obj, json_path))

shapes = obj['shapes']

for shape in shapes:

annotation = self._annotation(shape)

self.annotations.append(annotation)

self.ann_id += 1

self.img_id += 1

instance = {}

instance['info'] = 'spytensor created'

instance['license'] = ['license']

instance['images'] = self.images

instance['annotations'] = self.annotations

instance['categories'] = self.categories

return instance

# 构建类别

def _init_categories(self):

for k, v in classname_to_id.items():

category = {}

category['id'] = v

category['name'] = k

self.categories.append(category)

# 构建COCO的image字段

def _image(self, obj, path):

image = {}

from labelme import utils

img_x = utils.img_b64_to_arr(obj['imageData'])

h, w = img_x.shape[:-1]

image['height'] = h

image['width'] = w

image['id'] = self.img_id

image['file_name'] = os.path.basename(path).replace(".json", ".jpg")

return image

# 构建COCO的annotation字段

def _annotation(self, shape):

label = shape['label']

points = shape['points']

annotation = {}

annotation['id'] = self.ann_id

annotation['image_id'] = self.img_id

annotation['category_id'] = int(classname_to_id[label])

annotation['segmentation'] = [np.asarray(points).flatten().tolist()]

annotation['bbox'] = self._get_box(points)

annotation['iscrowd'] = 0

annotation['area'] = 1.0

return annotation

# 读取json文件，返回一个json对象

def read_jsonfile(self, path):

with open(path, "r", encoding='utf-8') as f:

return json.load(f)

# COCO的格式： [x1,y1,w,h] 对应COCO的bbox格式

def _get_box(self, points):

min_x = min_y = np.inf

max_x = max_y = 0

for x, y in points:

min_x = min(min_x, x)

min_y = min(min_y, y)

max_x = max(max_x, x)

max_y = max(max_y, y)

return [min_x, min_y, max_x - min_x, max_y - min_y]

if __name__ == '__main__':

labelme_path = "labelme/" # 此处根据你的数据集地址来修改

saved_coco_path = "./"

# 创建文件

if not os.path.exists("%scoco/annotations/"%saved_coco_path):

os.makedirs("%scoco/annotations/"%saved_coco_path)

if not os.path.exists("%scoco/images/train2017/"%saved_coco_path):

os.makedirs("%scoco/images/train2017"%saved_coco_path)

if not os.path.exists("%scoco/images/val2017/"%saved_coco_path):

os.makedirs("%scoco/images/val2017"%saved_coco_path)

# 获取images目录下所有的joson文件列表

json_list_path = glob.glob(labelme_path + "/*.json")

# 数据划分,这里没有区分val2017和tran2017目录，所有图片都放在images目录下

train_path, val_path = train_test_split(json_list_path, test_size=0.12)

print("train_n:", len(train_path), 'val_n:', len(val_path))

# 把训练集转化为COCO的json格式

l2c_train = Lableme2CoCo()

train_instance = l2c_train.to_coco(train_path)

l2c_train.save_coco_json(train_instance, '%scoco/annotations/instances_train2017.json'%saved_coco_path)

for file in train_path:

shutil.copy(file.replace("json","jpg"),"%scoco/images/train2017/"%saved_coco_path)

for file in val_path:

shutil.copy(file.replace("json","jpg"),"%scoco/images/val2017/"%saved_coco_path)

# 把验证集转化为COCO的json格式

l2c_val = Lableme2CoCo()

val_instance = l2c_val.to_coco(val_path)

l2c_val.save_coco_json(val_instance, '%scoco/annotations/instances_val2017.json'%saved_coco_path)

<img src="https://img-blog.csdnimg.cn/20210504204859314.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 运行完成<code>labelme2coco.py</code>后生成coco文件夹，里面包含annotations以及images。其中annotations包含instances_train2017.json以及instances_val2017.json; images中包含train2017以及val2017用于存放训练与验证数据集。

2、参数配置

修改<code>config.py</code>中参数，主要有四处。 ① coco_classes与coco_label_map <img src="https://img-blog.csdnimg.cn/20210504205405566.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 该数据集有一类名称为<code>‘trafficcone’</code>逗号不要忘记。 ② <code>dataset_base</code>地址 <img src="https://img-blog.csdnimg.cn/20210504205622906.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> ③<code>coco_base_config</code> <img src="https://img-blog.csdnimg.cn/20210504210025743.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> ④<code>yolact_base_config</code> <img src="https://img-blog.csdnimg.cn/20210504210110693.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 3、训练与测试 运行<code>python train.py --config=yolact_base_config </code>进行训练。 如果在google colab上在运行<code>!python train.py --config=yolact_base_config </code> <img src="https://img-blog.csdnimg.cn/20210504210509273.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述"> 按Ctr+C终止训练，并保存权重。

执行：

<pre data-index="4" class="prettyprint"><code class="prism language-python has-numbering" οnclick="mdcp.copyCode(event)" style="position: unset;">python eval.py --trained_model=weights/yolact_base_169_169_interrupt.pth --score_threshold=0.15 --top_k=15 --image=1.jpg

测试结果如下： <img src="https://img-blog.csdnimg.cn/20210504210754889.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM0MjI1NDY5,size_16,color_FFFFFF,t_70" alt="在这里插入图片描述">

</div>