自动驾驶语义分割模型

医学影像、自动驾驶、遥感分析:盘点语义分割6大经典模型在不同领域的实战表现 本文深入评测了FCN、U-Net、DeepLab等6大语义分割经典模型在医学影像、自动驾驶和遥感分析三大领域的实战表现。通过对比mIoU、Dice系数等核心指标,分析了各模型在医疗小目标检测、自动驾驶实时性和遥感多尺度处理等场景的优劣势,为不同应用场景的技术选型提供专业建议。特别推荐U-Net在医学影像分割中的卓越表现。 阅读详情

State-of-the-art Semantic Segmentation models need to be tuned for efficient memory consumption and fps output to be used in time-sensitive domains like autonomous vehicles

In a previous post, we studied various open datasets that could be used to train a model for pixel-wise semantic segmentation of urban scenes. Here, we take a look at various deep learning architectures that cater specifically to time-sensitive domains like autonomous vehicles. In recent years, deep learning has surpassed traditional computer vision algorithms by learning a hierarchy of features from the training dataset itself. This eliminates the need for hand-crafted features and thus such techniques are being extensively explored in academia and industry.

Deep Learning Architectures

Prior to deep learning architectures, semantic segmentation models relied on hand-crafted features fed into classifiers like Random Forests, SVM, etc. But after their mettle was proved in image classification tasks, these deep learning architectures started being used by researchers as a backbone for semantic segmentation tasks. Their feature learning capabilities, along with further algorithmic and network design improvements, have then helped produce fine and dense pixel predictions. We introduce one such pioneering work below called Fully Convolutional Network (FCN) on the basis of which all future models are roughly based.

FCN

Figure 1 [Source] : VGG-16 architecture reinterpreted as FCN

  • Contribution
    • This work reinterpreted the final fully connected layers of various LSVRC (Large Scale Visual Recognition Challenge, a.k.a ImageNet) networks such as AlexNet and VGG16 as fully convolutional networks. Using the concept of skip-layer fusion to decode low-resolution feature maps to pixel-wise prediction allowed the network to learn end to end.
  • Architecture
    • As seen in the above image, the upsampled outputs of a particular layer are concatenated with the outputs of the previous layer to improve the accuracy of the output. Thus, appearance (edges) from the shallower layers are combined with coarse and semantic information from the deeper layers. The upsampling operation in the deeper layers’ feature maps is also trainable, unlike conventional upsampling operations that use mathematical interpolations.
  • Drawbacks
    • The authors did not add more decoders since there was no additional accuracy gain and thus, high-resolution features were ignored. Also, using the encoder feature maps during inference time makes the process memory intensive.

Real-Time Semantic Segmentation

Post FCN, various other networks such as DeepLab (introduced atrous convolutions), UNet (introduced encoder-decoder structure), etc., have made pioneering contributions to the field of semantic segmentation. On the basis of the aforementioned networks, various state-of-the-art models like RefineNet, PSPNet,DeepLabv3, etc. have achieved an IoU (Intersection Over Union) > 80% on benchmark datasets like Cityscapes and PASCAL VOC.

Figure 2 [Source] : Accuracy vs Time for various Semantic Segmentation architectures

But real-time domains like autonomous vehicles need to make decisions in the order of milliseconds. As can be seen from Figure 2, the aforementioned networks are quite time-intensive. Table 1 also details the memory requirements of various models. This has encouraged researchers to explore novel designs to achieve output rates of >10fps from a neural network and contain fewer parameters.

Table 1 :

Semantic Segmentation models for autonomous vehicles

ArchitectureParamsOutputmIoU (%)Contribution

FCN-8 (VGG-16)

134M2fps65.3Skip-connections to merge lower-level finer features with higher -level coarse features

SegNet

29.46M16.7fps57Reduces computational time by storing max-poling indices

ENet

0.37M76.9fps58.3Early Downsampling along with Dilated and Asymmetric Convolutions

ICNet

6.68M30.3 fps69.5Creates coarse feature map using low-resolution images and gradually refines using high-resolutions images

SEGNET

Figure 3 [Source] : A) VGG-16 architecture with max-pooling indices from encoder to decoder, B) Sparse Upsampling

  • Contribution
    • This network has much fewer trainable parameters since the decoder layers use max-pooling indices from corresponding encoder layers to perform sparse upsampling. This reduces inference time at the decoder stage since, unlike FCNs, the encoder maps are not involved in the upsampling. Such a technique also eliminates the need to learn parameters for upsampling, unlike in FCNs.
  • Architecture
    • The SegNet architecture adopts the VGG16 network along with an encoder-decoder framework wherein it drops the fully connected layers of the network. The decoder sub-network is a mirror copy of the encoder sub-network, both containing 13 layers. Figure 3(B) shows how SegNet and FCN carry out their feature map decoding.
  • Drawbacks
    • Strong downsampling hurts accuracy.

ENET

Figure 4 [Source]: E-Nets with A) Cityscapes output, B) elementary units, C) network layers with output dimensions

  • Contribution
    • The authors created a light network using an asymmetric encoder-decoder architecture. In addition, they made various other architectural decisions such as early downsampling, dilated and asymmetricconvolutions, not using bias terms, parametric ReLU activations and Spatial DropOut.
  • Architecture
    • Efficient Neural Network (E-net) aims to reduce inference time on images by reducing a large number of floating point operations present in previous architectures. This too is an encoder-decoder based architecture, with the difference that the decoder is much larger than the encoder. Here, the authors take inspiration from ResNet-based architectures – there is a single main branch, with extensions (convolutional filters) that separate from it and merge back using element-wise addition as shown in Figure 4(B)(2).
    • The authors also do not use any bias terms and noticed that it does not lead to any loss of accuracy. They also employ early downsampling – Figure 4(C), which reduces the image dimensions and hence saves on costly computations at the beginning of the network. Using dilated convolutions also makes sure that the network has a wider receptive field and saves them from aggressive downsampling early on in the network.
    • This model has been tested by the authors on the Nvidia Jetson TX1 Embedded Platform and code may be found here.

ICNet

Figure 5 [Source] : ICNet architecture with its three branches for multi-scale inputs

  • Contribution
    • ICNet (Image Cascade Network) cascades feature maps from various resolutions. This is done to exploit the processing efficiency of low-resolution images and high inference quality of high-resolution images. A representation of this logic is shown in Figure 5.
    • Instead of trying intuitive strategies such as downsampling inputs (as in ENets), downsampling feature maps or model compression (removing feature maps entirely), the authors use a CFF (Cascade Feature Fusion)unit to merge feature maps of low resolutions with those of high resolutions.
  • Architecture
    • Here the input image is fed into three different branches with different resolutions of ¼, ½ and 1. Each branch reduces the spatial resolution of its input by ⅛ and hence the outputs of the three branches are ¹/₃₂, ¹/₁₆ and ⅛,  of the original image. The output of branch1 (o/p – ¹/₃₂) is fused with the output of branch2 (o/p – ¹/₁₆) using the previously mentioned CFF unit. Similar operations are performed for branch2 andbranch3, and the final output of the network is an image which is ⅛ of the original size.
    • Since convolutional parameters are shared between the ¼ and ½ resolution branches, the network size is also reduced. Also, during inference, branch1 and branch2 are completely discarded, and onlybranch3 is utilised. This leads to a computation time of a mere 9ms.
    • The branches use the design of PSPNet50 (a 50-layer deep ResNet for semantic segmentation). They contain 50, 17 and 3 layers respectively.

Final Thoughts

Various architectures have made novel improvements in the way 2-dimensional data is processed through data graphs. Although embedded platforms continue to improve with more memory and FLOPS capability, the above architectural and mathematical improvements have led to major leaps in semantic segmentation network outputs. With state-of-the-art networks, we can now achieve an output rate (in fps) that is close enough to image acquisition rates, and with acceptable quality (in mIoU) for autonomous vehicles.

实时语义分割---PIDNet论文笔记 实时语义分割-PIDNet论文笔记 阅读详情

相关推荐

使用paddle框架无人驾驶 —— 语义分割模型检测车道线

前言 准备在欧卡2实现无人驾驶,第一步使用paddlepaddle框架的paddleseg模块对车道线语义分割。 语义分割数据集 本文采用的数据集是tusimple数据集,图森车道线检测数据集。 下载地址:传送门 对数据集做了一些简单的处理,记录原始图像和带标签图像的文件路径关系。 数据集样例: paddleseg 这是百度深度学习框架推出的语义分割模块,集成了很多大佬写好的模型,只需要直接配置就可以使用这些模块了,对于新手上手比较友好。 paddleseg项目地址:国产深度学习框架,点个star支

qq_37668436的博客 4095

实时语义分割

实时语义分割

weixin_45912366的博客 2661

道路场景语义分割算法

道路场景语义分割算法 输入输出接口 Input: (1)左右两个摄像头采集的实时图像视频分辨率(整型int) (2)左右两个摄像头采集的实时图像视频格式 (RGB,YUV,MP4等) (3)摄像头标定参数(中心位置(x,y)和5个畸变 系数(2径向,2切向,1棱向),浮点型float) (4)摄像头初始化参数(摄像头初始位置和三个坐标方向 的旋转角度,车辆宽度高度车速等等,浮点型float) Ou...

吴建明wujianming_110117 4528

实时语义分割调研

文章目录0. 前言1. 语义分割概述2. 一些总结3. 论文浏览3.1 ENet3.2 ICNet3.3 BiSeNet3.4 CGNet3.5 DFANet3.6 Fast-SCNN3.7 BiSeNetV23.8 HyperSeg3.9 Rethinking BiSeNet For Real-time Semantic Segmentation 0. 前言 实时语义分割(real-time semantic segmentation),目标是能实时跑在CPU上。 本文调研的基本都是普通的语义分割方法,

清欢 6006

中洺科技自动驾驶数据标注途径为:

河北中洺教育自动驾驶数据标注是通过使用包围盒并定义其他属性,对图像或视频中的车、人、物进行标注的过程,教会模型识别行人、汽车、交通标志等交通要素,从而帮助ML模型理解和识别车辆中传感器检测到的物体。实现自动驾驶的基础是人工智能,实现人工智能的基础是自动驾驶数据标注。3D点云标注是从激光雷达采集的点云图像中寻找目标物体,以立方体框架的形式进行标注,不易受光照强度和其他物体变化的影响。包括平面图片中车辆的3D标注,主要用于训练自动驾驶仪判断过往或超车车辆的体积,为汽车建立对行人和车辆的完整理解。

zmhee的博客 540

实例分割、语义分割和 SAM(Segment Anything Model)

实例分割、语义分割和都是图像处理中的重要技术,它们的目标是通过分割图像中的不同对象或区域来帮助识别和分析图像,但它们的工作方式和适用场景各有不同。

2301_76846375的博客 6078

计算机视觉——实时语义分割ENet算法原理与模型训练

ENet所提出的这一创新神经网络架构,专为语义分割任务量身打造,其核心设计理念在于充分发挥嵌入式设备上有限资源的潜力。在这一目标的指引下,ENet的工作取得了显著的成效,不仅成功实现了与现有技术相比更为高效的计算性能,而且在某些情况下,还实现了与之相匹配甚至更优的结果,即便这些现有技术对计算能力和内存的需求远高于ENet的设计。

知来者逆的博客 5332

语义分割模型选型指南:医疗影像、自动驾驶、遥感,你的场景该用哪个?

本文深入解析语义分割模型在医疗影像、自动驾驶和遥感三大场景中的选型策略。针对医疗影像的像素级精度需求推荐U-Net架构,自动驾驶场景强调DeepLabv3+的实时处理能力,遥感领域则需结合多光谱数据特性定制解决方案。提供从模型原理到实战优化的完整指南,帮助开发者根据具体业务需求选择最佳语义分割模型。

weixin_30940783的博客 342

轻量化实时语义分割LiteSeg——从算法原理到模型训练与部署

1.论文提出Atrous Spatial Pyramid Pooling module (ASPP)这个创新点,论文作者应用长短残差连接、深度可分离卷积构造了一个更快、更有效的语义分割模型。论文给出了LiteSeg在多个backbone上测试的比较,Darknet19、MobileNet以及ShuffleNet,在准确性及计算成本之间提供一种折衷方案。比如以MobileNetV2为主网络的在Cityscapes数据集上针对640×360分辨率的图像以每秒161帧的速度,达到了67.81%的mIoU精度。

2005

预训练模型在无人驾驶中的应用案例分析

1. 背景介绍 1.1 无人驾驶的发展 无人驾驶汽车是近年来人工智能和自动驾驶技术的重要应用领域。随着计算机视觉、传感器技术、深度学习等相关技术的不断发展,无人驾驶汽车已经从实验室走向了现实生活。各大汽车制造商和科技公司纷纷投入巨资研发无人驾驶技术,希望能够在这个领域占据一席之地。

AI天才研究院 220

Python实现自动驾驶的道路语义分割深度网络

Faster R-CNN主要用于目标检测,输出为对象的类别和边界框。Mask R-CNN基于Faster R-CNN增加了用于生成掩码的分支,因此能够同时进行目标检测和实例分割。语义分割需要的是为每个像素点分配一个类别标签,而Mask R-CNN能够提供更丰富的实例级分割。在下一节中,我们将深入探讨DeepLab系列网络,它们专门针对语义分割任务进行了优化。语义分割是一种计算机视觉技术,旨在对图像中的每个像素进行分类,以识别图像中的不同对象和区域。

weixin_31185473的博客 957

语义分割模型怎么选?医疗影像、自动驾驶、遥感场景下的模型实战对比与选型指南

本文深度解析语义分割模型在医疗影像、自动驾驶和遥感场景下的实战选型策略,对比FCN、U-Net、DeepLabv3+等主流架构的性能差异。针对医疗影像的小样本学习、自动驾驶的实时性要求及遥感图像的多尺度挑战,提供数据增强、模型优化和部署方案,帮助开发者根据业务需求选择最佳模型组合。

weixin_30315723的博客 161

Streamlit(九) widgets-input

st.text_input / st.number_input / st.date_input / st.time_input 源码路径:${python}\Lib\site-packages\streamlit\elements\text_widgets.py(number_input.py, time_widgets.py) st.text_input st.text_input(label, value="", max_chars=None, key=None,\

heianduck的博客 3354

鲁棒性极强!利用SAM实现自动驾驶中的零样本语义分割

【免费送书】????????????本次为大家送出5本《三维实景可视化室内定位导航技术》参与方法:关注下方「3DCV」公众号 在「3DCV」公众号后台,回复送书即可参与开奖时间:9月11日 下午6:000. 这篇文章干了啥?语义分割是自动驾驶中的一项重要任务。它有助于实现基于可视化界面的环境感知、路径规划与决策、交通场景分析、障碍规避与碰撞预防、精确定位以及人机交互。语义分割的性能保证了自动驾驶中...

3D视觉工坊 867

汽车电子开发必看:AUTOSAR MCAL层SPI驱动配置全解析(基于EB tresos)

本文深入解析了AUTOSAR MCAL层SPI驱动的核心配置,基于EB tresos工具详细阐述了通道、外部设备、作业与序列等关键概念。重点剖析了多设备共享总线时的资源管理与实时性调优策略,特别是SpiJobPriority参数的决定性影响,并提供了常见配置陷阱的实战排查指南,旨在帮助汽车电子工程师高效完成SPI驱动配置。

weixin_29188043的博客 293

鲁棒优化入门(一)——工具箱Xprog和RSOME的安装与使用

Xprog是由新加坡国立大学的Peng Xiong于2016年发布的一款matlab工具箱,可以用于求解确定性优化、随机优化、鲁棒优化和分布式鲁棒优化问题。 还有一个针对鲁棒随机优化设计的matlab工具箱——RSOME(Robust Stochastic Optimization Made Easy),也可以用来解决一般的线性规划、随机规划、鲁棒优化和数据驱动的分布式鲁棒优化问题。...............

配电网和matlab的博客 1万+

【OpenMMLab】语义分割与MMSegmentation

任务:按照图像中每个像素的语义类别,将图像分割成不同的区域等价于:对图像中的每个像素进行分类。

DplusC的博客 611

cartographer基于3d地图的纯定位模式

关于纯定位模式呢,本人试过2d和3d,怎么说呢,有条件的话,建议直接上3d,因为博主当初在做定位时针对室内和室外都尝试过,2d只在室内环境下效果会很棒,但是在室外就呵呵啦!所以这里直接讲解3d室外定位(建图篇请自己查看我之前的博客:[3d建图](https://blog.youkuaiyun.com/qq_40216084/article/details/105531983) 看了我之前的博客大概知道配置文件和源文件路径了,我这里就不把路径写出来了,挺麻烦的。直接讲怎么做把,有不懂的留言,看到了我一定知无不言。 **首

qq_40216084的博客 5552

干货 | deeplabv3实现自动驾驶数据语义分割(附源码和中文论文)

本项目使用pytorch实现的deeplabv3+算法训练自动驾驶场景的语义分割模型。该数据集使用labelme工具标注,总共6个类别,分别是:车辆,虚线,右车道,中间车道,坑洼和背景。【关键词】:DeepLabv3+ labelme数据标注 数据解析 中文版deeplabv3+论文。

weixin_41575197的博客 2201

DDR5内存调优实战:手把手教你搞定Write Leveling Training Mode(附时序图解析)

本文深入解析DDR5内存调优中的Write Leveling Training Mode技术,详细介绍了其核心原理、硬件准备、实战步骤及常见问题解决方案。通过JESD79-5标准定义的反馈机制,帮助硬件工程师精确对齐DQS与CK信号,提升系统稳定性和数据可靠性。附有时序图解析和代码示例,适合DDR5内存系统设计与调试参考。

weixin_29256771的博客 552

揭秘AI人工智能中Stable Diffusion的模型微调技巧

Stable Diffusion作为一种强大的文本到图像生成模型,在艺术创作、设计、娱乐等多个领域展现出了巨大的应用潜力。然而,其默认的模型输出可能无法满足特定用户或特定场景的个性化需求。本文章的目的在于深入揭秘Stable Diffusion的模型微调技巧,让读者能够根据自己的需求对模型进行定制化调整,生成符合特定风格、主题或要求的图像。范围涵盖了从基础概念到实际操作,从理论原理到项目实战的全方位内容,旨在为读者提供一个系统且深入的学习指南。

大模型应用工坊 951

基于Mxnet实现语义分割-整体多模型【完整、附部分源码】

本文主要讲解基于mxnet深度学习框架实现目标检测,鉴于之前写chainer的麻烦,本结构代码也类似chainer的目标检测框架,各个模型只需要修改网络结构即可,本次直接一篇博文写完目标检测框架及网络结构的搭建,让志同道合者不需要切换文章。

ctu_sue的博客 1656

044、Edge Impulse的音频分类实战

上周调试一块STM32WBA52开发板,板载PDM麦克风死活采不到有效数据。示波器怼上去,时钟线波形漂亮得像教科书,数据线却像心电图——全是毛刺。折腾半天发现是电源纹波干扰,给麦克风单独加了个LC滤波,世界清净了。这种坑,做音频分类的嵌入式工程师迟早要踩一遍。

2401_88858637的博客 58

【联邦学习】实验思路分析01

本文仅对《FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving》中的实验思路进行简单梳理,供实验设计和实施提供参考,暂不涉及具体的实现算法细节和理论知识总结,学习中。

qq_74225552的博客 1098

Claude Code 实战 从零教你用 Claude Code 干活

Claude Code 实用指南:AI 编程助手的核心功能与使用技巧 Claude Code 是 Anthropic 推出的智能编程助手,可直接在本地环境中执行操作,相比传统 AI 对话工具具有显著优势:能直接读取项目文件、自动执行修改、记忆项目规则,并支持连接外部服务。本文详解其安装配置、三大核心场景(资料搜索、网页抓取、文章撰写)及进阶功能(Skills、MCP协议、斜杠命令)。特别提供了工作模式切换、CLAUDE.md配置文件等实用技巧,并附常见问题解决方案,帮助用户高效利用这一"坐在身边的

2401_84052244的博客 7628

大中型超市收银系统(可直接运行)

光步前台收银系统系统功能:商品销售、商品退货、挂单、收银员权限、会员销售、单品取消、整单取消、商品查询、脱机销售、脱机数据上传,基本资料下载,开钱箱、小票打印、收款结算,支持多币种付款方式(现金、储值卡、礼券、微信等),支持多种促销。运行环境:.net 4.0登陆信息用户名:1001 密码:123456内置商品信息,商品编码可输入00001000 - 00010000,测试效果相关功能键F5 结算 F9挂单 F11商品查询 F8退货 Insert单品取消 Del整单取消 F12开钱箱F2 会员 Ctrl+Z退出系统 Home 锁屏 相关源码待公开

上一篇: 机器视觉中常用开源数据集和免费标注工具
下一篇: 基于实例的像素级标注介绍
AONDATA
博客等级 码龄9年 74粉丝 9原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值