一. 图像编辑器 Monica
Monica 是一款跨平台的桌面图像编辑软件(早期主要是个人为了验证一些算法而产生的)。
其技术栈如下：
Kotlin 编写 UI(Kotlin Compose Desktop 作为 UI 框架)
基于 mvvm 模式，依赖注入使用 koin，编译使用 JDK 17。
部分算法使用 Kotlin 实现。
其余的算法使用 OpenCV C++ 来实现，Kotlin 通过 jni 来调用。
Monica 所使用的模型，主要使用 ONNXRuntime 进行部署和推理。
其余少部分模型使用 OpenCV DNN 进行部署和推理。
本地的算法库使用 C++ 17 编译。
Monica 目前还处于开发阶段，当前版本的可以参见 github 地址：https://github.com/fengzhizi715/Monica
在这个月里，我完成了 Monica 比较重要的两个功能：图形绘制、图像调色。
二. 图形绘制
Monica 支持在图像上的任意位置绘制线段、圆、三角形、矩形、任意多边形，在任意位置添加文字，以及对这些绘制的图形更改属性。
2.1 形状绘制
下面是展示形状绘制的入口。
以及绘制形状的页面。
Monica 提供了图像上的任意位置绘制各种图形的功能，以及修改图形的属性比如图像的颜色、透明度、是否填充、边框类型。
在图像中绘制形状，主要是调用 Compose 的 Canvas API。在实现绘制功能之前，需要先定义好能够绘制图形的类型。
sealed class Shape {
data class Line(val from: Offset, val to: Offset, val shapeProperties: ShapeProperties): Shape()
data class Circle(val center: Offset, val radius:Float, val shapeProperties: ShapeProperties): Shape()
data class Triangle(val first: Offset, val second: Offset?=null, val third: Offset?=null, val shapeProperties: ShapeProperties): Shape()
data class Rectangle(val tl: Offset, val bl: Offset, val br: Offset, val tr: Offset, val rectFirst: Offset,val shapeProperties: ShapeProperties): Shape()
data class Polygon(val points: List<Offset>, val shapeProperties: ShapeProperties): Shape()
data class Text(val point: Offset, val message:String, val shapeProperties: ShapeProperties): Shape()
}对于不同图形的绘制，需要确定好相关点的坐标。这块的逻辑比较多，可以查看项目的源码。下面主要讲讲如何绘制图形。
class ShapeDrawingViewModel {
fun drawShape(canvasDrawer:CanvasDrawer,
lines: Map<Offset, Line>,
circles: Map<Offset, Circle>,
triangles: Map<Offset, Triangle>,
rectangles: Map<Offset, Rectangle>,
polygons: Map<Offset, Polygon>,
texts: Map<Offset, Text>,
saveFlag: Boolean = false) {
lines.forEach {
val line = it.value
if (line.from != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(line.from, line.shapeProperties.color)
}
if (line.from != Offset.Unspecified && line.to != Offset.Unspecified) {
canvasDrawer.line(line.from,line.to, Style(null, line.shapeProperties.color, line.shapeProperties.border, null, fill = line.shapeProperties.fill, scale = 1f, alpha = line.shapeProperties.alpha, bounded = true))
}
}
circles.forEach {
val circle = it.value
if (circle.center != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(circle.center, circle.shapeProperties.color)
}
canvasDrawer.circle(circle.center, circle.radius, Style(null, circle.shapeProperties.color, circle.shapeProperties.border, null, fill = circle.shapeProperties.fill, scale = 1f, alpha = circle.shapeProperties.alpha, bounded = true))
}
triangles.forEach {
val triangle = it.value
if (triangle.first != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(triangle.first, triangle.shapeProperties.color)
}
if (triangle.second != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(triangle.second!!, triangle.shapeProperties.color)
canvasDrawer.line(triangle.first,triangle.second, Style(null, triangle.shapeProperties.color, triangle.shapeProperties.border, null, fill = triangle.shapeProperties.fill, scale = 1f, alpha = triangle.shapeProperties.alpha, bounded = true))
}
if (triangle.first != Offset.Unspecified && triangle.second != Offset.Unspecified && triangle.third != Offset.Unspecified) {
val list = mutableListOf<Offset>().apply {
add(triangle.first)
add(triangle.second!!)
add(triangle.third!!)
}
canvasDrawer.polygon(list, Style(null, triangle.shapeProperties.color, triangle.shapeProperties.border, null, fill = triangle.shapeProperties.fill, scale = 1f, alpha = triangle.shapeProperties.alpha, bounded = true))
}
}
rectangles.forEach {
val rect = it.value
if (rect.rectFirst!=Offset.Unspecified && !saveFlag) {
canvasDrawer.point(rect.rectFirst, rect.shapeProperties.color)
}
if (rect.tl!=Offset.Unspecified && rect.bl!=Offset.Unspecified && rect.br!=Offset.Unspecified && rect.tr!=Offset.Unspecified) {
val list = mutableListOf<Offset>().apply {
add(rect.tl)
add(rect.bl)
add(rect.br)
add(rect.tr)
}
canvasDrawer.polygon(list, Style(null, rect.shapeProperties.color, rect.shapeProperties.border, null, fill = rect.shapeProperties.fill, scale = 1f, alpha = rect.shapeProperties.alpha, bounded = true))
}
}
polygons.forEach {
val polygon = it.value
if (polygon.points[0]!=null && polygon.points[0] != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(polygon.points[0] , polygon.shapeProperties.color)
}
if (polygon.points.size>1 && polygon.points[1] != Offset.Unspecified && !saveFlag) {
canvasDrawer.point(polygon.points[1] , polygon.shapeProperties.color)
canvasDrawer.line(polygon.points[0], polygon.points[1], Style(null, polygon.shapeProperties.color, Border.Line, null, fill = polygon.shapeProperties.fill, scale = 1f, alpha = polygon.shapeProperties.alpha, bounded = true))
}
canvasDrawer.polygon(polygon.points, Style(null, polygon.shapeProperties.color, polygon.shapeProperties.border, null, fill = polygon.shapeProperties.fill, scale = 1f, alpha = polygon.shapeProperties.alpha, bounded = true))
}
texts.forEach {
val text = it.value
if (text.point!= Offset.Unspecified) {
val list = mutableListOf<String>().apply {
add(text.message)
}
canvasDrawer.text(text.point, list, text.shapeProperties.color, text.shapeProperties.fontSize)
}
}
}
......
}2.2 添加文字
Monica 支持在图像的任意位置添加文字，修改文字的属性比如字体大小、颜色。
在图像中添加文字，需要一个可以拖动的 TextField ，这样才能在图像的任意位置添加文字。因此，DraggableTextField 控件如下：
@OptIn(ExperimentalFoundationApi::class)
@Composable
fun DraggableTextField(
modifier: Modifier = Modifier,
text: String,
bitmapWidth: Int,
bitmapHeight: Int,
density: Density,
onTextChanged: (String) -> Unit,
onDragged: (Offset) -> Unit
) {
var offset by remember { mutableStateOf(Offset.Zero) }
val halfWidth = bitmapWidth/2
val halfHeight = bitmapHeight/2
val halfTextFieldWidth = 125/density.density
val halfTextFieldHeight = 65/density.density
Box(
modifier = modifier
.offset { IntOffset(offset.x.roundToInt(), offset.y.roundToInt()) }
.pointerInput(Unit) {
detectDragGestures { change ->
offset += change
if (abs(offset.x) > halfWidth - halfTextFieldWidth || abs(offset.y) > halfHeight - halfTextFieldHeight) {
offset -= change
return@detectDragGestures
}
}
}
.shadow(8.dp)
.background(Color.White)
.padding(16.dp)
.fillMaxWidth()
.wrapContentHeight(Alignment.Top)
.clip(RoundedCornerShape(8.dp))
) {
Column {
TextField (
value = text,
onValueChange = onTextChanged,
modifier = Modifier.width(220.dp)
)
confirmButton(true, modifier = Modifier.align(Alignment.End).padding(top = 5.dp)) {
onDragged.invoke(offset)
}
}
}
}需要注意的是 DraggableTextField 中的 offset 通过 onDragged 回调给当前图像，但是 offset 要变成图像中的当前的坐标，还需要做一些坐标转换才行。
三. 图像调色
Monica 支持调节图像的对比度、色调、饱和度、亮度、色温等，从而帮助大家调整图像的色彩。
3.1 应用层的设计和调用
该模块功能的实现，最终也是调用了封装 OpenCV 的函数。对于应用层，需要编写好调用 jni 层的代码：
object ImageProcess {
......
/**
* 初始化图像调色模块
*/
external fun initColorCorrection(src: ByteArray): Long
/**
* 图像调色
*/
external fun colorCorrection(src: ByteArray, colorCorrectionSettings: ColorCorrectionSettings, cppObjectPtr:Long):IntArray
/**
* 删除 ColorCorrection
*/
external fun deleteColorCorrection(cppObjectPtr:Long): Long
......
}其中，initColorCorrection() 返回的是 Long 类型，其实是一个指针的地址，表示的是一个 C++ 对象。之所以这么做是为了在处理当前图片时，能够复用该 C++ 对象。
viewModel 的 colorCorrection 会调用 ImageProcess 的 colorCorrection() 函数，然后将结果展示到 UI 上。离着这个界面的时候，会清除所有的状态，以及回收所用到的 C++ 对象。
class ColorCorrectionViewModel {
private val logger: Logger = logger<ColorCorrectionViewModel>()
var contrast by mutableStateOf(255f )
var hue by mutableStateOf(180f )
var saturation by mutableStateOf(255f )
var lightness by mutableStateOf(255f )
var temperature by mutableStateOf(255f )
var highlight by mutableStateOf(255f )
var shadow by mutableStateOf(255f )
var sharpen by mutableStateOf(0f )
var corner by mutableStateOf(0f )
private var cppObjectPtr:Long = 0
private var init:AtomicBoolean = AtomicBoolean(false)
fun colorCorrection(state: ApplicationState, image: BufferedImage, colorCorrectionSettings: ColorCorrectionSettings,
success: CVSuccess) {
logger.info("colorCorrectionSettings = ${GsonUtils.toJson(colorCorrectionSettings)}")
state.scope.launchWithLoading {
if (!init.get()) {
init.set(true)
val byteArray = image.image2ByteArray()
cppObjectPtr = ImageProcess.initColorCorrection(byteArray)
}
OpenCVManager.invokeCV(image,
action = { byteArray -> ImageProcess.colorCorrection(byteArray, colorCorrectionSettings, cppObjectPtr) },
success = { success.invoke(it) },
failure = { e ->
logger.error("colorCorrection is failed", e)
})
}
}
......
fun clearAllStatus() {
init.set(false)
contrast = 255f
hue = 180f
saturation = 255f
lightness = 255f
temperature = 255f
highlight = 255f
shadow = 255f
sharpen = 0f
corner = 0f
colorCorrectionSettings = ColorCorrectionSettings()
ImageProcess.deleteColorCorrection(cppObjectPtr)
cppObjectPtr = 0
}
}3.2 jni 层的编写
对于 jni 层，需要先在头文件里定义好应用层对应的函数
JNIEXPORT jlong JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_initColorCorrection
(JNIEnv* env, jobject,jbyteArray array);
JNIEXPORT jintArray JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_colorCorrection
(JNIEnv* env, jobject,jbyteArray array, jobject jobj, jlong cppObjectPtr);
JNIEXPORT void JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_deleteColorCorrection
(JNIEnv* env, jobject, jlong cppObjectPtr);然后，编写对应的实现。
JNIEXPORT jlong JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_initColorCorrection
(JNIEnv* env, jobject, jbyteArray array) {
Mat image = byteArrayToMat(env, array);
// 创建 C++ 对象并存储指针
ColorCorrection* cppObject = new ColorCorrection(image);
return reinterpret_cast<jlong>(cppObject);
}
JNIEXPORT jintArray JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_colorCorrection
(JNIEnv* env, jobject,jbyteArray array, jobject jobj, jlong cppObjectPtr) {
ColorCorrection* colorCorrection = reinterpret_cast<ColorCorrection*>(cppObjectPtr);
ColorCorrectionSettings colorCorrectionSettings;
Mat image = byteArrayToMat(env, array);
// 获取 jclass 实例
jclass jcls = env->FindClass("cn/netdiscovery/monica/ui/controlpanel/colorcorrection/model/ColorCorrectionSettings");
jfieldID contrastId = env->GetFieldID(jcls, "contrast", "I");
jfieldID hueId = env->GetFieldID(jcls, "hue", "I");
jfieldID saturationId = env->GetFieldID(jcls, "saturation", "I");
jfieldID lightnessId = env->GetFieldID(jcls, "lightness", "I");
jfieldID temperatureId = env->GetFieldID(jcls, "temperature", "I");
jfieldID highlightId = env->GetFieldID(jcls, "highlight", "I");
jfieldID shadowId = env->GetFieldID(jcls, "shadow", "I");
jfieldID sharpenId = env->GetFieldID(jcls, "sharpen", "I");
jfieldID cornerId = env->GetFieldID(jcls, "corner", "I");
jfieldID statusId = env->GetFieldID(jcls, "status", "I");
colorCorrectionSettings.contrast = env->GetIntField(jobj, contrastId);
colorCorrectionSettings.hue = env->GetIntField(jobj, hueId);
colorCorrectionSettings.saturation = env->GetIntField(jobj, saturationId);
colorCorrectionSettings.lightness = env->GetIntField(jobj, lightnessId);
colorCorrectionSettings.temperature = env->GetIntField(jobj, temperatureId);
colorCorrectionSettings.highlight = env->GetIntField(jobj, highlightId);
colorCorrectionSettings.shadow = env->GetIntField(jobj, shadowId);
colorCorrectionSettings.sharpen = env->GetIntField(jobj, sharpenId);
colorCorrectionSettings.corner = env->GetIntField(jobj, cornerId);
colorCorrectionSettings.status = env->GetIntField(jobj, statusId);
Mat dst;
try {
colorCorrection->doColorCorrection(colorCorrectionSettings, dst);
} catch(...) {
}
jthrowable mException = NULL;
mException = env->ExceptionOccurred();
if (mException != NULL) {
env->ExceptionClear();
jclass exceptionClazz = env->FindClass("java/lang/Exception");
env->ThrowNew(exceptionClazz, "jni exception");
env->DeleteLocalRef(exceptionClazz);
return env->NewIntArray(0);
}
env->DeleteLocalRef(jcls); // 手动释放局部引用
return matToIntArray(env, dst);
}
JNIEXPORT void JNICALL Java_cn_netdiscovery_monica_opencv_ImageProcess_deleteColorCorrection
(JNIEnv* env, jobject, jlong cppObjectPtr) {
// 删除 C++对象，防止内存泄漏
ColorCorrection* colorCorrection = reinterpret_cast<ColorCorrection*>(cppObjectPtr);
delete colorCorrection;
}jni 层还需要调用 C++ 对应的 ColorCorrection 类，这块因为篇幅原因暂时省略，感兴趣的话可以直接看项目的源码。
同意还有一个值得注意的是，从应用层传递的 ColorCorrectionSettings 对象，需要通过 jobject 转换到 jclass 然后再获取对应的各个属性。在 jni 层也需要定义好对应的 ColorCorrectionSettings 对象。
typedef struct {
int contrast;
int hue;
int saturation;
int lightness;
int temperature;
int highlight;
int shadow;
int sharpen;
int corner;
int status;
} ColorCorrectionSettings;四. 总结
Monica 支持了图形绘制、图像调色之后，它才算是一款比较基础的图像编辑器。后面还有很长的路要走，Monica 现有的每一个功能都需要打磨一下。
到农历过年前，我对 Monica 的规划是争取完善 CV 算法快速调参的模块和将部分模型部署到云端。如果能完成这些的话，明年可以做更多有意思的功能。
最后，Monica github 地址：https://github.com/fengzhizi715/Monica
【Java与Android技术栈】公众号
关注 Java/Kotlin 服务端、桌面端 、Android 、机器学习、端侧智能
更多精彩内容请关注：
扫一扫
938
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
