一种结合语义分割模型和图割的街景影像变化检测方法

李文国; 黄亮; 左小清; 王译著

doi:10.12265/j.gnss.2020110401

一种结合语义分割模型和图割的街景影像变化检测方法

doi: 10.12265/j.gnss.2020110401

李文国¹,
黄亮^{1, 2, ,},
左小清¹,
王译著¹

1.
昆明理工大学国土资源工程学院，昆明 650093
2.
云南省高校高原山区空间信息测绘技术应用工程研究中心，昆明 650093

详细信息

作者简介:
李文国：（1993—），男，硕士研究生，主要研究方向为深度学习和遥感影像变化检测

黄亮：（1985—），男，博士，讲师，硕士生导师，主要研究方向为遥感影像变化检测

左小清：（1972—），男，博士，教授，博士生导师，研究方向为三维GIS和激光雷达

王译著：（1995—），男，硕士研究生，研究方向为遥感影像变化检测

通信作者:
黄亮 Email：kmhuangliang@163.com

中图分类号: P232
计量
- 文章访问数: 654
- HTML全文浏览量: 356
- PDF下载量: 30
- 被引次数: 0
出版历程
- 收稿日期: 2020-11-04
- 网络出版日期: 2021-04-06
- 刊出日期: 2021-02-15

A street view image change detection method combining semantic segmentation model and graph cuts

1.
Faculty of Land Resource Engineering, Kunming University of Science and Technology, Kunming 650093, China
2.
Surveying and Mapping Geo-Informatics Technology Research Center on Plateau Mountains of Yunnan Higher Education, Kunming 650093, China

摘要

摘要: 由于街景影像具有地物尺度多样化、地物界限不明确、地物光谱信息复杂等问题，造成应用统计方法、机器学习等方法对复杂度高的街景影像变化检测性能欠佳. 因此提出一种结合语义分割模型和图割（GC）的街景影像变化检测方法. 该方法首先采用Camvid数据集训练DeeplabV3+网络得到的迁移学习模型对两个时期的街景影像进行语义分割；然后采用GC方法实现消除天空和植被等对街景影像的影响；接着采用变化向量分析（CVA）获取差异影像，最后对差异影像进行二值化和精度评价. 研究结果表明，提出的方法总体精度优于大津法（OTSU）、K均值法、Segnet网络迁移学习模型方法和DeeplabV3+网络迁移学习模型方法，是一种可行的街景影像变化检测方法.
- DeeplabV3+网络 /
- 图割（GC） /
- 变化检测 /
- 迁移学习 /
- 变化向量分析（CVA）
Abstract: Due to the problems of the diversity of the scale, the unclear boundary and the complex spectral information of the ground objects, the performance of the statistical method and machine learning method for the change detection of the high complexity street view images is poor. Therefore, a street view images change detection method combining semantic segmentation model and graph cuts (GC) is proposed in this paper. Firstly, the DeeplabV3+ semantic segmentation model combined with migration learning is used to pre-train the Camvid data set to obtain the pre-trained model in this method; Then a small number of annotated samples from the data set of this paper were used to Fine Tune the pre-training model, which was respectively used for semantic segmentation of street view images in two periods. Then GC method is used to remove the sky, roads,vegetation and other factors, which is impacting on the street view. Finally, change vector analysis (CVA) is used to obtain the difference images, and binarization and accuracy evaluation were carried out for the difference images. The results show that the overall accuracy of the proposed method is better than the Otsu method (OTSU), K-means method, Segnet network migration learning method and DeeplabV3+ network migration learning method, it is a feasible method for detecting changes in street view images.
- DeeplabV3+ network /
- graph cuts /
- change detection /
- the migration study /
- CVA

HTML全文

图 1 变化检测流程图

下载: 全尺寸图片幻灯片

图 2 DeeplabV3+网络结构

下载: 全尺寸图片幻灯片

图 3 DeeplabV3+网络与Segnet网络分类结果

下载: 全尺寸图片幻灯片

图 4 G₁和G₂街景影像及参考图

下载: 全尺寸图片幻灯片

图 5 第一组对比实验结果图

下载: 全尺寸图片幻灯片

图 6 第二组对比实验结果图

下载: 全尺寸图片幻灯片

表 1 实验结果精度对比

实验方法	漏检率/%	错检率/%	总体精度/%
OTUS(G₁)	27	65	43
OTUS(G₂)	12	84	33
K均值(G₁)	35	54	50
K均值(G₂)	51	56	45
本文方法(G₁)	28	16	81
本文方法(G₂)	14	29	74

下载: 导出CSV

表 2 实验结果精度对比

实验方法	漏检率/%	错检率/%	总体精度/%
Segnet (G₁)	37	52	51
Segnet (G₂)	39	54	49
DeeplabV3+ 网络(G₁)	24	42	62
DeeplabV3+网络(G₂)	15	47	61
Segnet+GC (G₁)	33	20	77
Segnet+GC (G₂)	38	32	67
本文方法(G₁)	28	16	81
本文方法(G₂)	14	29	74

下载: 导出CSV

参考文献(15)

[1]	WU C, ZHANG L F, ZHANG L P. A scene change detection framework for multi-temporal very high resolution remote sensing images[J]. Signal processing, 2016, 124(sl): 184-197. DOI: 10.1016/j.sigpro.2015.09.020.
[2]	WU C, ZHANG L F, DU B. Kernel slow feature analysis for scene change detection[J]. IEEE transactions on geoscience and remote sensing, 2017, 55(4): 2367-2384. DOI: 10.1109/TGRS.2016.2642125.
[3]	POLLARD T, MUNDY J L. Change detection in a 3-d world[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. DOI: 10.1109/CVPR.2007.383073.
[4]	RADKE R J, ANDAR S, AL-KOFAHI O, et al. Image change detection algorithms: a systematic survey[J]. IEEE transactions on image processing, 2005, 14(3): 294-307. DOI: 10.1109/TIP.2004.838698.
[5]	CRISPELL D, MUNDY J, TAUBIN G. A variable-resolution probabilistic three-dimensional model for change detection[J]. IEEE transactions on geoscience and remote sensing, 2012, 50(2): 489-500. DOI: 10.1109/TGRS.2011.2158439.
[6]	HUERTAS A, NEVATIA R. Detecting changes in aerial views of man-made structures[J]. Image and vision computing, 2000, 18(8): 583-596. DOI: 10.1016/S0262-8856(99)00063-3.
[7]	EDEN I, COOPER D B. Using 3D line segments for robust and efficient change detection from multiple noisy images[J]. Lecture notes in computer science, 2008: 172-185. DOI: 10.1007/978-3-540-88693-8_13.
[8]	TANEJA A, BALLAN L, POLLEFEYS M. Image based detection of geometric changes in urban environments[C]//2011 International Conference on Computer Vision. DOI: 10.1109/ICCV.2011.6126515.
[9]	SAKURADA K, OKATANI T. Change detection from a street image pair using CNN features and superpixel segmentation[C]//Proceedings of the British Machine Vision Conference (BMVC), BMVA Press, 2015. DOI: 10.5244/C.29.61.
[10]	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with arous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision, 2018: 833-851. DOI: 10.1007/978-3-030-01234-2_49.
[11]	BOYKOV Y Y, JOLLY M P. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images[C]//Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001. DOI: 10.1109/ICCV.2001.937505.
[12]	SAKURADA K, OKATANI T, DEGUCHI K. Detecting changes in 3D structure of a scene from multi-view images captured by a vehicle-mounted camera[C]//Proceedings of the 2013 IEEE Computer Vision and Pattern Recognition, IEEE, 2013: 137-144. DOI: 10.1109/CVPR.2013.25.
[13]	马骕, 邓喀中, 庄会富, 等. 中低分辨率合成孔径雷达影像多纹理特征的Otsu变化检测[J]. 激光与光电子学进展, 2017, 54(6): 299-305.
[14]	田淞, 宋建社, 张雄美, 等. KM-SVM法的SAR图像无监督变化检测[J]. 系统工程与电子技术, 2015, 37(5): 1042-1046.
[15]	BOVOLO F, BRUZONE L. A detail-preserving scale-driven approach to change detection in multitemporal SAR images[J]. IEEE transactions on geoscience and remote sensing, 2005, 43(12): 2963-2972. DOI: 10.1109/TGRS.2005.857987.