Caffe2教程实例，加载预训练模型

最新推荐文章于 2020-09-24 04:09:11 发布

浪Q拜

最新推荐文章于 2020-09-24 04:09:11 发布

阅读量2.5k

点赞数 1

分类专栏： caffe2

caffe2 专栏收录该内容

4 篇文章

订阅专栏

Caffe2教程实例，加载预训练模型

概述

本教程使用模型库中的预训练模型squeezenet 里分类我们自己的图片。我们需要提供要分类图片的路径或者URL信息作为输入。了解ImageNet对象代码可以帮助我们来验证我们的结果。对象代码只是训练过程中的整型数字标签来标识不同的分类，比如985标识daisy分类。注意，我们在这里使用squeezenet ，本教程也对运行预训练模型提供通用方法。

如果是从图像预处理教程进来的，你会发现我们使用了rescale和crop方法来对图像进行预处理,将图像格式化为CHW,BGR类型，最后格式化为NCHW。我们还通过使用从提供的npy文件中计算出的平均值，或者静态删除128作为占位符平均值，来纠正图像平均值。

你会发现加载预训练模型非常简单，并且语法简洁。从全局来看，下面是使用预训练模型接口需要的三个步骤：

读取预训练模型的初始化和预测protobuf 文件

with open("init_net.pb", "rb") as f:
     init_net = f.read()
 with open("predict_net.pb", "rb") as f:
     predict_net = f.read()

初始化一个预测器

 p = workspace.Predictor(init_net, predict_net)

在测试数据上运行神经网络，并得到最符合的结果

 results = p.run({'data': img})

注意，假设网络的最后一层是softmax层，返回的结果是一个多维概率数组，其长度等于模型所训练的类的数量。概率可以由目标代码(整数类型)进行索引，因此如果知道目标代码，可以在该索引处索引结果数组，以查看网络对输入图像是否属于该类的可能性。

模型下载操作

尽管此处使用squeezenet ，也可以从预训练库中检出不同的预训练模型，或者使用Caffe2的caffe2.python.models.download模型来从模型库轻松获取预训练模型。

我们在这里使用 models.download模型来下载squeezenet 并导入到 /caffe2/python/models本地Caffe2文件夹

python -m caffe2.python.models.download -i squeezenet

如果上述操作下载完成，将会在你的/caffe2/python/models文件夹下生成一个squeezenet 文件夹，其中包含了
init_net.pb和predict_net.pb。注意，如果不使用-i标签，模型库会下载到你的当前工作目录。此外，如果想下载所有的模型，可以直接克隆版本库

git clone https://github.com/caffe2/models

代码示例

依赖导入

在开始前，需要导入需要的依赖库或者方法

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals


from caffe2.proto import caffe2_pb2
import numpy as np
import skimage.io
import skimage.transform
from matplotlib import pyplot
import os
from caffe2.python import core, workspace, models
import urllib2
import operator
print("Required modules imported.")

运行结果：

WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
Required modules imported.

输入
此处，指定输入，包括输入的图像，模型路径，均值文件，图像要求大小，标签映射文件


# Configuration --- Change to your setup and preferences!
# This directory should contain the models downloaded from the model zoo. To run this 
#   tutorial, make sure there is a 'squeezenet' directory at this location that 
#   contains both the 'init_net.pb' and 'predict_net.pb'
CAFFE_MODELS = "~/caffe2/caffe2/python/models"

# Some sample images you can try, or use any URL to a regular image.
# IMAGE_LOCATION = "https://upload.wikimedia.org/wikipedia/commons/thumb/f/f8/Whole-Lemon.jpg/1235px-Whole-Lemon.jpg"
# IMAGE_LOCATION = "https://upload.wikimedia.org/wikipedia/commons/7/7b/Orange-Whole-%26-Split.jpg"
# IMAGE_LOCATION = "https://upload.wikimedia.org/wikipedia/commons/a/ac/Pretzel.jpg"
# IMAGE_LOCATION = "https://cdn.pixabay.com/photo/2015/02/10/21/28/flower-631765_1280.jpg"
IMAGE_LOCATION = "images/flower.jpg"

# What model are we using?
#    Format below is the model's: <folder, INIT_NET, predict_net, mean, input image size>
#    You can switch 'squeezenet' out with 'bvlc_alexnet', 'bvlc_googlenet' or others that you have downloaded
MODEL = 'squeezenet', 'init_net.pb', 'predict_net.pb', 'ilsvrc_2012_mean.npy', 227

# codes - these help decypher the output and source from a list from ImageNet's object codes 
#    to provide an result like "tabby cat" or "lemon" depending on what's in the picture 
#   you submit to the CNN.
codes =  "https://gist.githubusercontent.com/aaronmarkham/cd3a6b6ac071eca6f7b4a6e40e6038aa/raw/9edb4038a37da6b5a44c3b5bc52e448ff09bfe5b/alexnet_codes"
print("Config set!")

运行结果：Config set!

设置路径

完成配置后，需要加载均值文件（如果存在的话），就像预测文件和初始化网络文件一样。

# set paths and variables from model choice and prep image
CAFFE_MODELS = os.path.expanduser(CAFFE_MODELS)

# mean can be 128 or custom based on the model
# gives better results to remove the colors found in all of the training images
MEAN_FILE = os.path.join(CAFFE_MODELS, MODEL[0], MODEL[3])
if not os.path.exists(MEAN_FILE):
    print("No mean file found!")
    mean = 128
else:
    print ("Mean file found!")
    mean = np.load(MEAN_FILE).mean(1).mean(1)
    mean = mean[:, np.newaxis, np.newaxis]
print("mean was set to: ", mean)

# some models were trained with different image sizes, this helps you calibrate your image
INPUT_IMAGE_SIZE = MODEL[4]

# make sure all of the files are around...
INIT_NET = os.path.join(CAFFE_MODELS, MODEL[0], MODEL[1])
PREDICT_NET = os.path.join(CAFFE_MODELS, MODEL[0], MODEL[2])

# Check to see if the files exist
if not os.path.exists(INIT_NET):
    print("WARNING: " + INIT_NET + " not found!")
else:
    if not os.path.exists(PREDICT_NET):
        print("WARNING: " + PREDICT_NET + " not found!")
    else:
        print("All needed files found!")

运行结果：
No mean file found!
mean was set to: 128
All needed files found!

图像处理
现在我们已经指定了输入并验证了输入网络的存在，我们可以将该图像加载并对该图像进行预处理，以便摄入到一个Caffe2卷积神经网络中!这是一个非常重要的步骤，因为经过训练的CNN需要特定大小的输入图像，其值来自特定的分布.

# Function to crop the center cropX x cropY pixels from the input image
def crop_center(img,cropx,cropy):
    y,x,c = img.shape
    startx = x//2-(cropx//2)
    starty = y//2-(cropy//2)
    return img[starty:starty+cropy,startx:startx+cropx]

# Function to rescale the input image to the desired height and/or width. This function will preserve
#   the aspect ratio of the original image while making the image the correct scale so we can retrieve
#   a good center crop. This function is best used with center crop to resize any size input images into
#   specific sized images that our model can use.
def rescale(img, input_height, input_width):
    # Get original aspect ratio
    aspect = img.shape[1]/float(img.shape[0])
    if(aspect>1):
        # landscape orientation - wide image
        res = int(aspect * input_height)
        imgScaled = skimage.transform.resize(img, (input_width, res),mode='constant',anti_aliasing=False)
    if(aspect<1):
        # portrait orientation - tall image
        res = int(input_width/aspect)
        imgScaled = skimage.transform.resize(img, (res, input_height),mode='constant',anti_aliasing=False)
    if(aspect == 1):
        imgScaled = skimage.transform.resize(img, (input_width, input_height),mode='constant',anti_aliasing=False)
    return imgScaled

# Load the image as a 32-bit float
#    Note: skimage.io.imread returns a HWC ordered RGB image of some size
img = skimage.img_as_float(skimage.io.imread(IMAGE_LOCATION)).astype(np.float32)
print("Original Image Shape: " , img.shape)

# Rescale the image to comply with our desired input size. This will not make the image 227x227
#    but it will make either the height or width 227 so we can get the ideal center crop.
img = rescale(img, INPUT_IMAGE_SIZE, INPUT_IMAGE_SIZE)
print("Image Shape after rescaling: " , img.shape)
pyplot.figure()
pyplot.imshow(img)
pyplot.title('Rescaled image')

pyplot.show()

# Crop the center 227x227 pixels of the image so we can feed it to our model
img = crop_center(img, INPUT_IMAGE_SIZE, INPUT_IMAGE_SIZE)
print("Image Shape after cropping: " , img.shape)
pyplot.figure()
pyplot.imshow(img)
pyplot.title('Center Cropped')
pyplot.show()

# switch to CHW (HWC --> CHW)
img = img.swapaxes(1, 2).swapaxes(0, 1)
print("CHW Image Shape: " , img.shape)

pyplot.figure()
for i in range(3):
    # For some reason, pyplot subplot follows Matlab's indexing
    # convention (starting with 1). Well, we'll just follow it...
    pyplot.subplot(1, 3, i+1)
    pyplot.imshow(img[i])
    pyplot.axis('off')
    pyplot.title('RGB channel %d' % (i+1))
    pyplot.show()

# switch to BGR (RGB --> BGR)
img = img[(2, 1, 0), :, :]

# remove mean for better results
img = img * 255 - mean

# add batch size axis which completes the formation of the NCHW shaped input that we want
img = img[np.newaxis, :, :, :].astype(np.float32)

print("NCHW image (ready to be used as input): ", img.shape)

运行结果：

Original Image Shape: (1200, 1087, 3)
Image Shape after rescaling: (250, 227, 3)
Image Shape after cropping: (227, 227, 3)
CHW Image Shape: (3, 227, 227)
NCHW image (ready to be used as input): (1, 3, 227, 227)

在这里插入图片描述

准备CNN运行神经网络

现在图像已经准备好被CNN接收，让我们打开protobufs，将它们加载到工作区中，并运行网络。

# Read the contents of the input protobufs into local variables
with open(INIT_NET, "rb") as f:
    init_net = f.read()
with open(PREDICT_NET, "rb") as f:
    predict_net = f.read()

# Initialize the predictor from the input protobufs
p = workspace.Predictor(init_net, predict_net)

# Run the net and return prediction
results = p.run({'data': img})

# Turn it into something we can play with and examine which is in a multi-dimensional array
results = np.asarray(results)
print("results shape: ", results.shape)

# Quick way to get the top-1 prediction result
# Squeeze out the unnecessary axis. This returns a 1-D array of length 1000
preds = np.squeeze(results)
# Get the prediction and the confidence by finding the maximum value and index of maximum value in preds array
curr_pred, curr_conf = max(enumerate(preds), key=operator.itemgetter(1))
print("Prediction: ", curr_pred)
print("Confidence: ", curr_conf)

运行结果：

results shape: (1, 1, 1000, 1, 1)
Prediction: 309
Confidence: 0.6333014

处理结果

回忆一下ImageNet是一个1000类数据集，可以看到结果的第三个轴是长度为1000，这不是巧合。这个轴表示预训练模型中每个类别的概率。因此，当您查看特定索引处的结果数组时，该数字可以解释为输入属于与该索引对应的类的概率。现在我们已经运行了预测器并收集了结果，我们可以通过将它们与相应的英语标签进行匹配来解释它们。

# the rest of this is digging through the results 
results = np.delete(results, 1)
index = 0
highest = 0
arr = np.empty((0,2), dtype=object)
arr[:,0] = int(10)
arr[:,1:] = float(10)
for i, r in enumerate(results):
    # imagenet index begins with 1!
    i=i+1
    arr = np.append(arr, np.array([[i,r]]), axis=0)
    if (r > highest):
        highest = r
        index = i 

# top N results
N = 5
topN = sorted(arr, key=lambda x: x[1], reverse=True)[:N]
print("Raw top {} results: {}".format(N,topN))

# Isolate the indexes of the top-N most likely classes
topN_inds = [int(x[0]) for x in topN]
print("Top {} classes in order: {}".format(N,topN_inds))

# Now we can grab the code list and create a class Look Up Table
response = urllib2.urlopen(codes)
class_LUT = []
for line in response:
    code, result = line.partition(":")[::2]
    code = code.strip()
    result = result.replace("'", "")
    if code.isdigit():
        class_LUT.append(result.split(",")[0][1:])
        
# For each of the top-N results, associate the integer result with an actual class
for n in topN:
    print("Model predicts '{}' with {}% confidence".format(class_LUT[int(n[0])],float("{0:.2f}".format(n[1]*100))))

运行结果：
Raw top 5 results: [array([309.0, 0.6333013772964478], dtype=object), array([312.0, 0.18357813358306885], dtype=object), array([311.0, 0.07437768578529358], dtype=object), array([113.0, 0.05832172930240631], dtype=object), array([78.0, 0.028850432485342026], dtype=object)]
Top 5 classes in order: [309, 312, 311, 113, 78]
Model predicts ‘bee’ with 63.33% confidence
Model predicts ‘cricket’ with 18.36% confidence
Model predicts ‘grasshopper’ with 7.44% confidence
Model predicts ‘snail’ with 5.83% confidence
Model predicts ‘tick’ with 2.89% confidence

批量处理

上面的例子演示了如何一次提供一个图像。如果我们在一个批处理中同时提供多个图像，我们可以获得更高的吞吐量。回想一下，输入分类器的数据是‘NCHW’顺序，因此为了输入多个图像，我们将展开‘N’轴。

# List of input images to be fed
images = ["images/cowboy-hat.jpg",
            "images/cell-tower.jpg",
            "images/Ducreux.jpg",
            "images/pretzel.jpg",
            "images/orangutan.jpg",
            "images/aircraft-carrier.jpg",
            "images/cat.jpg"]

# Allocate space for the batch of formatted images
NCHW_batch = np.zeros((len(images),3,227,227))
print ("Batch Shape: ",NCHW_batch.shape)

# For each of the images in the list, format it and place it in the batch
for i,curr_img in enumerate(images):
    img = skimage.img_as_float(skimage.io.imread(curr_img)).astype(np.float32)
    img = rescale(img, 227, 227)
    img = crop_center(img, 227, 227)
    img = img.swapaxes(1, 2).swapaxes(0, 1)
    img = img[(2, 1, 0), :, :]
    img = img * 255 - mean
    NCHW_batch[i] = img

print("NCHW image (ready to be used as input): ", NCHW_batch.shape)

# Run the net on the batch
results = p.run([NCHW_batch.astype(np.float32)])

# Turn it into something we can play with and examine which is in a multi-dimensional array
results = np.asarray(results)

# Squeeze out the unnecessary axis
preds = np.squeeze(results)
print("Squeezed Predictions Shape, with batch size {}: {}".format(len(images),preds.shape))

# Describe the results
for i,pred in enumerate(preds):
    print("Results for: '{}'".format(images[i]))
    # Get the prediction and the confidence by finding the maximum value 
    #   and index of maximum value in preds array
    curr_pred, curr_conf = max(enumerate(pred), key=operator.itemgetter(1))
    print("\tPrediction: ", curr_pred)
    print("\tClass Name: ", class_LUT[int(curr_pred)])
    print("\tConfidence: ", curr_conf)

输出结果示例：
Batch Shape: (7, 3, 227, 227)
NCHW image (ready to be used as input): (7, 3, 227, 227)
Squeezed Predictions Shape, with batch size 7: (7, 1000)
Results for: ‘images/cowboy-hat.jpg’
Prediction: 515
Class Name: cowboy hat
Confidence: 0.85009265
Results for: ‘images/cell-tower.jpg’
Prediction: 645
Class Name: maypole
Confidence: 0.18584323
Results for: ‘images/Ducreux.jpg’
Prediction: 568
Class Name: fur coat
Confidence: 0.1025313
Results for: ‘images/pretzel.jpg’
Prediction: 932
Class Name: pretzel
Confidence: 0.999622
Results for: ‘images/orangutan.jpg’
Prediction: 365
Class Name: orangutan
Confidence: 0.99200517
Results for: ‘images/aircraft-carrier.jpg’
Prediction: 403
Class Name: aircraft carrier
Confidence: 0.9998779
Results for: ‘images/cat.jpg’
Prediction: 281
Class Name: tabby
Confidence: 0.51331687