这一章来记录一下yolov5通过onnx->ncnn然后部署到Android端手机APP上运行。移植文件的.param和.bin文件的转换请看上一篇文章。
一. 下载工程项目
项目参考知乎的nihui大佬的文章。我们可以下载大佬GitHub上开源工程，然后这个工程需要去下载ncnn，然后将ncnn复制到jni文件夹下面，然后编译运行就可以了。
文章链接：详细记录u版YOLOv5目标检测ncnn实现 - 知乎 (zhihu.com)
项目链接：ncnn-android-yolov5
ncnn：ncnn-android-vulkan
二. 新建Android工程移植
1、新建Android工程
Android studio的环境搭建请参考网上教程，这里就不过多赘述了。我们打开Android studio
新建工程后可以先运行一次，保证工程创建没有问题（避免后续出现问题很难找到的情况）。
2、移植文件
我们需要将GitHub中下载的项目中的一些文件复制到我们新建工程中来。
a. 将ncnn-android-yolov5-master\ncnn-android-yolov5-master\app\src\main 中的assets 文件夹和 jni 文件夹复制到我们的新工程中来。
jni目录下
b. 将.cxx文件夹复制到新建工程的 app 目录下。
d. 将java中的YoloV5Ncnn.java文件复制到新建工程中。
3、修改参数
1、打开新建工程
切换成Android视图界面。
2. 打开build.gradle(:app) 文件，修改和添加参数，这里的最小sdk版本为24，不能低于24，否则可能会编译不通过，然后添加步骤1和步骤2
代码：
ndk {
moduleName "ncnn"
abiFilters "armeabi-v7a", "arm64-v8a"
}externalNativeBuild {
cmake {
version "3.10.2"
path file('src/main/jni/CMakeLists.txt')
}
}3、 修改模型初始化和检测方法，打开YoloV5Ncnn.java文件这个Init和Detect方法会报错，原因是我们复制过来的项目导致这两个方法在C++文件中的路径变化了，所以我们点击这个红色灯泡重新生成一个方法接口到C++文件中就可以了。
点击红色灯泡这里
会在这个yolov5ncnn_jni.cpp文件中生成接口方法。
在这个文件上面也有一个初始化的接口，我们把它替换掉。首先复制这个新创建的接口名字。剩余的删掉。
将其复制到308行这个之前的初始化方法这里。
另外那个Detect也是一样的处理方法。
修改完成之后就是这样的。
4、在这个初始化方法中，我们需要修改初始化模型文件的路径。
复制这个java文件的路径
替换掉，注意下面这个也一起复制覆盖
4、添加UI文件
a. 在activity_main.xml中添加界面代码。
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<LinearLayout
android:layout_width="match_parent"
android:orientation="vertical"
android:layout_height="wrap_content">
<LinearLayout
android:layout_width="match_parent"
android:padding="5dp"
android:layout_height="wrap_content">
<TextView
android:layout_width="wrap_content"
android:text="@string/model"
android:textColor="@color/black"
android:textSize="18dp"
android:textStyle="bold"
android:layout_height="wrap_content"/>
<Spinner
android:id="@+id/Spinner_class"
android:layout_width="wrap_content"
android:entries="@array/spinnerclass"
android:layout_height="wrap_content"/>
<Button
android:id="@+id/yolo_detect"
android:layout_width="wrap_content"
android:text="@string/detect"
android:layout_height="wrap_content"/>
</LinearLayout>
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content">
<Button
android:layout_width="wrap_content"
android:id="@+id/photo_btn"
android:text="@string/photo"
android:layout_height="wrap_content"/>
</LinearLayout>
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content">
<ImageView
android:id="@+id/dis_detect_bitmap"
android:layout_width="wrap_content"
android:layout_height="wrap_content"/>
</LinearLayout>
</LinearLayout>
</LinearLayout>b. 在values目录下添加array.xml的下拉列表选项。
array代码：
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string-array name="spinnerclass">
<item>交通灯识别模型</item>
<item>左右转识别模型</item>
<item>暂未添加模型</item>
<item>暂未添加模型</item>
<item>暂未添加模型</item>
</string-array>
</resources>c. 添加java功能代码，这里直接添加在了maniactivity中了。
5、添加功能代码
java代码：
package com.example.android_yolov5_test;
import static android.widget.Toast.LENGTH_SHORT;
import androidx.appcompat.app.AppCompatActivity;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Matrix;
import android.graphics.Paint;
import android.media.ExifInterface;
import android.media.Image;
import android.net.Uri;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.AdapterView;
import android.widget.ArrayAdapter;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.Spinner;
import android.widget.Toast;
import java.io.FileNotFoundException;
import java.io.IOException;
public class MainActivity extends AppCompatActivity {
private static final int SELECT_IMAGE = 1;
private YoloV5Ncnn yolov5ncnn = new YoloV5Ncnn();
private String models_detect_name = "";
private ImageView dis_detect_bitmap;
private Bitmap Bitmapbm;
private Button photo_btn;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Spinner spinnerItems = (Spinner) findViewById(R.id.Spinner_class);
Button yolo_detect = findViewById(R.id.yolo_detect);
dis_detect_bitmap = findViewById(R.id.dis_detect_bitmap);
photo_btn = findViewById(R.id.photo_btn);
Bitmapbm = BitmapFactory.decodeResource(getResources(),R.drawable.img);
dis_detect_bitmap.setImageBitmap(Bitmapbm);
spinnerItems.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
@Override
public void onItemSelected(AdapterView<?> parent, View view, int pos, long id) {
String[] languages = getResources().getStringArray(R.array.spinnerclass);
//Toast.makeText(MainActivity.this, "你点击的是:"+languages[pos], LENGTH_SHORT).show();
Models_Init(languages, pos);
}
@Override
public void onNothingSelected(AdapterView<?> parent) {
// Another interface callback
}
});
yolo_detect.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
Models_detect(models_detect_name);
}
});
photo_btn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
Intent i = new Intent(Intent.ACTION_PICK);
i.setType("image/*");
startActivityForResult(i, SELECT_IMAGE);
}
});
}
private void Models_detect(String models_detect_name) {
switch (models_detect_name){
case "交通灯识别模型":
YoloV5Ncnn.Obj[] objects = yolov5ncnn.Detect(Bitmapbm, false);
showObjects(objects);
break;
case "左右转识别模型":
Toast.makeText(MainActivity.this, "暂未投放模型...", Toast.LENGTH_LONG).show();
break;
default:
break;
}
}
public void Models_Init(String[] models, int pos){
boolean ret_init = false;
models_detect_name = models[pos];
switch (models[pos]){
case "交通灯识别模型":
ret_init = yolov5ncnn.Init(getAssets());
if (!ret_init)
{
Log.e("MainActivity", "yolov5ncnn Init failed");
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化失败", Toast.LENGTH_LONG).show();
}else{
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化成功", Toast.LENGTH_LONG).show();
}
break;
case "左右转识别模型":
Toast.makeText(MainActivity.this, "暂未投放模型...", Toast.LENGTH_LONG).show();
break;
default:
break;
}
}
public void showObjects(YoloV5Ncnn.Obj[] objects)
{
if (objects == null)
{
dis_detect_bitmap.setImageBitmap(Bitmapbm);
return;
}
// draw objects on bitmap
Bitmap rgba = Bitmapbm.copy(Bitmap.Config.ARGB_8888, true);
final int[] colors = new int[] {
Color.rgb( 54, 67, 244),
Color.rgb( 99, 30, 233),
Color.rgb(176, 39, 156),
Color.rgb(183, 58, 103),
Color.rgb(181, 81, 63),
Color.rgb(243, 150, 33),
Color.rgb(244, 169, 3),
Color.rgb(212, 188, 0),
Color.rgb(136, 150, 0),
Color.rgb( 80, 175, 76),
Color.rgb( 74, 195, 139),
Color.rgb( 57, 220, 205),
Color.rgb( 59, 235, 255),
Color.rgb( 7, 193, 255),
Color.rgb( 0, 152, 255),
Color.rgb( 34, 87, 255),
Color.rgb( 72, 85, 121),
Color.rgb(158, 158, 158),
Color.rgb(139, 125, 96)
};
Canvas canvas = new Canvas(rgba);
Paint paint = new Paint();
paint.setStyle(Paint.Style.STROKE);
paint.setStrokeWidth(4);
Paint textbgpaint = new Paint();
textbgpaint.setColor(Color.WHITE);
textbgpaint.setStyle(Paint.Style.FILL);
Paint textpaint = new Paint();
textpaint.setColor(Color.BLACK);
textpaint.setTextSize(26);
textpaint.setTextAlign(Paint.Align.LEFT);
for (int i = 0; i < objects.length; i++)
{
paint.setColor(colors[i % 19]);
canvas.drawRect(objects[i].x, objects[i].y, objects[i].x + objects[i].w, objects[i].y + objects[i].h, paint);
// draw filled text inside image
{
String text = objects[i].label + " = " + String.format("%.1f", objects[i].prob * 100) + "%";
float text_width = textpaint.measureText(text);
float text_height = - textpaint.ascent() + textpaint.descent();
float x = objects[i].x;
float y = objects[i].y - text_height;
if (y < 0)
y = 0;
if (x + text_width > rgba.getWidth())
x = rgba.getWidth() - text_width;
canvas.drawRect(x, y, x + text_width, y + text_height, textbgpaint);
canvas.drawText(text, x, y - textpaint.ascent(), textpaint);
}
}
dis_detect_bitmap.setImageBitmap(rgba);
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data)
{
super.onActivityResult(requestCode, resultCode, data);
if (resultCode == RESULT_OK && null != data) {
Uri selectedImage = data.getData();
try
{
if (requestCode == SELECT_IMAGE) {
Bitmap bitmap = decodeUri(selectedImage);
Bitmapbm = bitmap.copy(Bitmap.Config.ARGB_8888, true);
dis_detect_bitmap.setImageBitmap(bitmap);
}
}
catch (FileNotFoundException e)
{
Log.e("MainActivity", "FileNotFoundException");
return;
}
}
}
private Bitmap decodeUri(Uri selectedImage) throws FileNotFoundException
{
// Decode image size
BitmapFactory.Options o = new BitmapFactory.Options();
o.inJustDecodeBounds = true;
BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o);
// The new size we want to scale to
final int REQUIRED_SIZE = 640;
// Find the correct scale value. It should be the power of 2.
int width_tmp = o.outWidth, height_tmp = o.outHeight;
int scale = 1;
while (true) {
if (width_tmp / 2 < REQUIRED_SIZE
|| height_tmp / 2 < REQUIRED_SIZE) {
break;
}
width_tmp /= 2;
height_tmp /= 2;
scale *= 2;
}
// Decode with inSampleSize
BitmapFactory.Options o2 = new BitmapFactory.Options();
o2.inSampleSize = scale;
Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o2);
// Rotate according to EXIF
int rotate = 0;
try
{
ExifInterface exif = new ExifInterface(getContentResolver().openInputStream(selectedImage));
int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
switch (orientation) {
case ExifInterface.ORIENTATION_ROTATE_270:
rotate = 270;
break;
case ExifInterface.ORIENTATION_ROTATE_180:
rotate = 180;
break;
case ExifInterface.ORIENTATION_ROTATE_90:
rotate = 90;
break;
}
}
catch (IOException e)
{
Log.e("MainActivity", "ExifInterface IOException");
}
Matrix matrix = new Matrix();
matrix.postRotate(rotate);
return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
}
}三. 项目运行
接下来就可以直接运行该项目了。
四. 添加自己的yolo模型
这里我们直接在后面添加我们自己的模型文件，不需要修改该项目原本的模型参数，这样就可以切换模型使用了。
1、首先在assets中添加我们自己的模型文件（文件的转换看上一篇文章）。
2、在YoloV5Ncnn.java 文件中添加模型初始化和检测的方法。
这个方法可以通过红色小灯泡创建与C++代码连接的接口
点击创建
检测接口也是一样的生成步骤
然后我们将上面的初始化代码和检测代码copy到这两个新建的接口方法中来。
复制到新创建的接口中
检测接口也是一样的操作
copy完成我们需要修改这两个新建接口中的参数，将初始化接口中的两个模型文件替换成我们自己训练转换的模型文件。
修改后
检测接口方法中需要修改这两个接口数据名
首先找到该模型的.param文件，将数据依次修改为以下两个数据。
然后修改检测类名
将class_name中的类名参数替换成自己模型的类名
还需要修改.param文件中的这三个参数改为-1
那么模型参数就修改配置好了。
3、接下来我们在java功能代码中进行初始化和检测的调用就可以了。
首先添加功能代码，这里我将整个mainactivity中的代码贴出来。
package com.example.android_yolov5_test;
import static android.widget.Toast.LENGTH_SHORT;
import androidx.appcompat.app.AppCompatActivity;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Matrix;
import android.graphics.Paint;
import android.media.ExifInterface;
import android.media.Image;
import android.net.Uri;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.AdapterView;
import android.widget.ArrayAdapter;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.Spinner;
import android.widget.Toast;
import java.io.FileNotFoundException;
import java.io.IOException;
public class MainActivity extends AppCompatActivity {
private static final int SELECT_IMAGE = 1;
private YoloV5Ncnn yolov5ncnn = new YoloV5Ncnn();
private String models_detect_name = "";
private ImageView dis_detect_bitmap;
private Bitmap Bitmapbm;
private Button photo_btn;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Spinner spinnerItems = (Spinner) findViewById(R.id.Spinner_class);
Button yolo_detect = findViewById(R.id.yolo_detect);
dis_detect_bitmap = findViewById(R.id.dis_detect_bitmap);
photo_btn = findViewById(R.id.photo_btn);
Bitmapbm = BitmapFactory.decodeResource(getResources(),R.drawable.img);
dis_detect_bitmap.setImageBitmap(Bitmapbm);
spinnerItems.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
@Override
public void onItemSelected(AdapterView<?> parent, View view, int pos, long id) {
String[] languages = getResources().getStringArray(R.array.spinnerclass);
//Toast.makeText(MainActivity.this, "你点击的是:"+languages[pos], LENGTH_SHORT).show();
Models_Init(languages, pos);
}
@Override
public void onNothingSelected(AdapterView<?> parent) {
// Another interface callback
}
});
yolo_detect.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
Models_detect(models_detect_name);
}
});
photo_btn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
Intent i = new Intent(Intent.ACTION_PICK);
i.setType("image/*");
startActivityForResult(i, SELECT_IMAGE);
}
});
}
private void Models_detect(String models_detect_name) {
YoloV5Ncnn.Obj[] objects;
switch (models_detect_name){
case "交通灯识别模型":
objects = yolov5ncnn.Detect(Bitmapbm, false);
showObjects(objects);
break;
case "左右转识别模型":
objects = yolov5ncnn.detectCustomLayer_traffic(Bitmapbm, false, 0.4, 0.25);
showObjects(objects);
break;
case "暂未添加模型":
Toast.makeText(MainActivity.this, "暂未投放模型...", Toast.LENGTH_LONG).show();
break;
default:
break;
}
}
public void Models_Init(String[] models, int pos){
boolean ret_init = false;
models_detect_name = models[pos];
switch (models[pos]){
case "交通灯识别模型":
ret_init = yolov5ncnn.Init(getAssets());
if (!ret_init)
{
Log.e("MainActivity", "yolov5ncnn Init failed");
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化失败", LENGTH_SHORT).show();
}else{
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化成功", Toast.LENGTH_SHORT).show();
}
break;
case "左右转识别模型":
ret_init = yolov5ncnn.initCustomLayer_traffic(getAssets());
if (!ret_init)
{
Log.e("MainActivity", "yolov5ncnn Init failed");
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化失败", Toast.LENGTH_SHORT).show();
}else{
Toast.makeText(MainActivity.this, "[" + models[pos] + "] 初始化成功", Toast.LENGTH_SHORT).show();
}
break;
case "暂未添加模型":
Toast.makeText(MainActivity.this, "暂未投放模型...", Toast.LENGTH_LONG).show();
default:
break;
}
}
public void showObjects(YoloV5Ncnn.Obj[] objects)
{
if (objects == null)
{
dis_detect_bitmap.setImageBitmap(Bitmapbm);
return;
}
// draw objects on bitmap
Bitmap rgba = Bitmapbm.copy(Bitmap.Config.ARGB_8888, true);
final int[] colors = new int[] {
Color.rgb( 54, 67, 244),
Color.rgb( 99, 30, 233),
Color.rgb(176, 39, 156),
Color.rgb(183, 58, 103),
Color.rgb(181, 81, 63),
Color.rgb(243, 150, 33),
Color.rgb(244, 169, 3),
Color.rgb(212, 188, 0),
Color.rgb(136, 150, 0),
Color.rgb( 80, 175, 76),
Color.rgb( 74, 195, 139),
Color.rgb( 57, 220, 205),
Color.rgb( 59, 235, 255),
Color.rgb( 7, 193, 255),
Color.rgb( 0, 152, 255),
Color.rgb( 34, 87, 255),
Color.rgb( 72, 85, 121),
Color.rgb(158, 158, 158),
Color.rgb(139, 125, 96)
};
Canvas canvas = new Canvas(rgba);
Paint paint = new Paint();
paint.setStyle(Paint.Style.STROKE);
paint.setStrokeWidth(4);
Paint textbgpaint = new Paint();
textbgpaint.setColor(Color.WHITE);
textbgpaint.setStyle(Paint.Style.FILL);
Paint textpaint = new Paint();
textpaint.setColor(Color.BLACK);
textpaint.setTextSize(26);
textpaint.setTextAlign(Paint.Align.LEFT);
for (int i = 0; i < objects.length; i++)
{
paint.setColor(colors[i % 19]);
canvas.drawRect(objects[i].x, objects[i].y, objects[i].x + objects[i].w, objects[i].y + objects[i].h, paint);
// draw filled text inside image
{
String text = objects[i].label + " = " + String.format("%.1f", objects[i].prob * 100) + "%";
float text_width = textpaint.measureText(text);
float text_height = - textpaint.ascent() + textpaint.descent();
float x = objects[i].x;
float y = objects[i].y - text_height;
if (y < 0)
y = 0;
if (x + text_width > rgba.getWidth())
x = rgba.getWidth() - text_width;
canvas.drawRect(x, y, x + text_width, y + text_height, textbgpaint);
canvas.drawText(text, x, y - textpaint.ascent(), textpaint);
}
}
dis_detect_bitmap.setImageBitmap(rgba);
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data)
{
super.onActivityResult(requestCode, resultCode, data);
if (resultCode == RESULT_OK && null != data) {
Uri selectedImage = data.getData();
try
{
if (requestCode == SELECT_IMAGE) {
Bitmap bitmap = decodeUri(selectedImage);
Bitmapbm = bitmap.copy(Bitmap.Config.ARGB_8888, true);
dis_detect_bitmap.setImageBitmap(bitmap);
}
}
catch (FileNotFoundException e)
{
Log.e("MainActivity", "FileNotFoundException");
return;
}
}
}
private Bitmap decodeUri(Uri selectedImage) throws FileNotFoundException
{
// Decode image size
BitmapFactory.Options o = new BitmapFactory.Options();
o.inJustDecodeBounds = true;
BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o);
// The new size we want to scale to
final int REQUIRED_SIZE = 640;
// Find the correct scale value. It should be the power of 2.
int width_tmp = o.outWidth, height_tmp = o.outHeight;
int scale = 1;
while (true) {
if (width_tmp / 2 < REQUIRED_SIZE
|| height_tmp / 2 < REQUIRED_SIZE) {
break;
}
width_tmp /= 2;
height_tmp /= 2;
scale *= 2;
}
// Decode with inSampleSize
BitmapFactory.Options o2 = new BitmapFactory.Options();
o2.inSampleSize = scale;
Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o2);
// Rotate according to EXIF
int rotate = 0;
try
{
ExifInterface exif = new ExifInterface(getContentResolver().openInputStream(selectedImage));
int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
switch (orientation) {
case ExifInterface.ORIENTATION_ROTATE_270:
rotate = 270;
break;
case ExifInterface.ORIENTATION_ROTATE_180:
rotate = 180;
break;
case ExifInterface.ORIENTATION_ROTATE_90:
rotate = 90;
break;
}
}
catch (IOException e)
{
Log.e("MainActivity", "ExifInterface IOException");
}
Matrix matrix = new Matrix();
matrix.postRotate(rotate);
return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
}
}运行效果展示：
好了，到这里就全部结束了，按照流程来就不会出错的，项目后面也可以添加更多的模型来进行切换识别，或者添加视频检测等待功能... 移植过程中有问题的可以在评论区留言
本文中示例工程
链接：https://pan.baidu.com/s/1qO6-OWOjgXYd0cWHSMFY6Q
提取码：d2a0
扫一扫
1242
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
