WANG Dengfei 1,2， SU Hongsheng1， CHEN Dengke3， ZHAO Xiaojuan2
（1. School of Automation and Electrical Engineering， Lanzhou Jiaotong University， Lanzhou 730070， China； 2. Key Laboratory of Plateau Traffic Information Engineering nd Control of Gansu Province， Lanzhou 730070， China； 3. BaotouVehicle Depot，China Railway Hohhot Group Co., Ltd.， Hohhot 010000， China）
Abstract： Missing and broken states of railway fasteners are directly related to driving safety. A deep network image detection method for railway fasteners based on residual network-single shot multibox detector (ResNet-SSD) is proposed to improve detection performance. Firstly, virtual image technology is used to simulate the defect states of fasteners, so as to expand feature information and overcome the imbalance problem of various fastener images. Secondly, ResNet is used to construct feature extraction network, and SSD is used for target detection to form ResNet-SSD network to detect the defect states of fasteners. Experimental results show that the ResNet-SSD has faster convergence speed and mean average precision (mAP) is increased by 3.06% from 95.47% to 98.53% after using the virtual image to expand the field collected fastener set; The detection accuracy of ResNet-SSD can reach 98.5% when the recall is 98.8%, which is higher than that of VGG16-SSD and the same as that of Faster-RCNN when intersection over union(IoU) is 0.5; The detection speed of this method is 6 times that of Faster RCNN and 1.5 times that of VGG16-SSD. Therefore, this method can meet the needs of real-time detection of railway fasteners.
Key words： railway fastener； virtual image； image recognition； residual network(ResNet)； single shot multibox detector(SSD)
References
［1］ MIN Y Z, XIAO B Y, MA H F, et al. Missing rail fastener detection based on machine vision method//International Conference on Electromechanical Control Technology and Transportation, Jan. 19-21, 2018, Chongqing, China. 2018： 119-124.
［2］ YANG J F, TAO W, LIU M H, et al. An efficient direction field-based method for the detection of fasteners on high-speed railways. Sensors, 2011, 11(8)： 7364-7381.
［3］ FENG H, JIANG Z G, XIE F Y, et al. Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Transactions on Instrumentation and Measurement, 2014, 63(4)： 877-888.
［4］ MALAR R, JAYALAKSHMY S. Detection of cracks and missing fasteners in railway lines using structure topic mode. International Journal of Innovative Science, Engineering & Technology, 2015, 2(10)： 369-375.
［5］ OU Y, LUO J Q, LI B L, et al. A classification model of railway fasteners based on computer vision. Neural Computing and Applications. 2019, 31： 9307-9319.
［6］ AI C B, QIU S, XU G Y. An anisotropic heat diffusion model for enhancing the extraction of underground rail track fasteners under extremely low and uneven illumination conditions. The Institution of Mechanical Engineers Part F： Journal of Rail and Rapid Transit, 2018, 233(4)： 410-419.
［7］ LIU X, WANG H B, ZHOU B. Research on detection algorithm for rail fastener based on computer vision. Advances in Engineering Research, 2018, 149： 647- 652.
［8］ LIU J J, YUAN J Y, JIA Y F. A new method for railway fastener detection using the symmetrical image and its EA-HOG feature. International Journal of Pattern Recognition and Artificial Intelligence, 2019, 43(2)： 2055006. doi： 10.1142/ S021800142055006X.
［9］ FAN H, COSMAN P C, HOU Y, et al. High speed railway fastener detection based on a line local binary pattern. IEEE Signal Processing Letters, 2018, 25(6)： 788-792.
［10］ LIU J J, XIONG Y, LI B L, et al. Research on automatic inspection algorithm for railway fastener defects based on computer vision. Journal of the China Railway Society, 2016, 38(8)： 73-80.
［10］ FAN H, WANG Q, LUO Y, et al. Abnormal railway fastener detection using minimal significant regions and local binary patterns. Journal of Optical Technology, 2019, 86(12)： 799-807.
［11］ WEI X K, YANG Z M, LIU Y X, et al. Railway track fastener defect detection based on image processing and deep learning techniques. Engineering Applications of Artificial Intelligence, 2019, (80)： 66-81.
［13］ BAI T B, YANG J W, XU G Y,et al. Research on railway fastener positioning based on Faster RCNN. Journal of Railway Science and Engineering, 2021, 18(2)： 502-508.
［14］ HOU Y，FAN H，XIONG Y，et al. Fastener defect detection algorithm based on cost-sensitive convolutional neural network. China Railway Science, 2021, 42(1)： 26-31.
［12］ YAO D C, SUN Q, YANG J W，et al. Railway fastener fault diagnosis based on generative adversarial network and residual network model. Shock and Vibration, 2020. https://doi.org/ 10.1155 /2020/8823050.
［13］ LIU W, ANGUELOV D, ERHAND, et al. SSD： single shot multiBox detector. 2015. DOI： 10.1007/978-3-319-46448-0_2.
［14］ REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once： unified, real-time object detection//IEEE Conference on Computer Vision and Pattern Recognition, Jun.26-Jul.1, 2016, Las Vegas, USA. New York： IEEE, 2016. https://doi.org/10.48550/arXiv.1506.02640.
［15］ HE K M, ZHANG X Y, REN S Q, et al. Identity mappings in deep residual networks//European Conference on Computer Vision, Oct. 11-14, 2016, Amsterdam, The Netherlands. Berlin： Springer, 2016： 630-645.
［16］ LIU J Y, WANG S S, HOU X W, et al. A deep residual learning serial segmentation network for extracting buildings from remote sensing imagery. International Journal of Remote Sensing, 2020, 41(14)： 5573-5587.
［17］ MA L, SHUAI R J, RAN X M, et al. Combining DC-GAN with ResNet for blood cell image classification, Medical & Biological Engineering & Computing, 2020, 58： 1251-1264.
［18］ SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions//2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun.8-10, 2015, Boston, USA. New York： IEEE, 2015： 1-9
［19］ KRIZHEVSKY A, SLTSKEVER I, HINTON G, et al. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6)： 84-90.
［23］ ZHANG Y Q, LU RZ, QIAO S J, et al. A sampling method of imbalanced data based on sample space. Acta Automatica Sinica, 2022, 48(10)： 2549-2563.
基于ResNet-SSD的铁路扣件缺损检测方法
王登飞1,2， 苏宏升1， 陈登科3， 赵小娟2
（1. 兰州交通大学 自动化与电气工程学院， 甘肃 兰州 730070； 2. 甘肃省高原交通信息工程及控制重点实验室， 甘肃 兰州 730070； 3. 中国铁路呼和浩特局集团公司 包头车辆段， 内蒙古 呼和浩特， 010000）
摘要：铁路轨面扣件的缺失、 损坏状态与行车安全密切关系。 为进一步提高铁路扣件检测性能， 提出基于残差网络-单发多框检测器（Residual network-single shot multibox detector, ResNet-SSD）的铁路扣件缺损检测方法。 首先， 采用虚拟图像技术进行扣件缺损状态图像建模来扩展扣件样本特征信息， 完善样本库以解决各类扣件图像不均衡问题。 其次， 使用ResNet构建特征提取网络， SSD模型进行目标检测， 完成对扣件缺损状态的检测。 实验结果表明， 采用虚拟图像扩展现场采集扣件图像集后， ResNet-SSD网络有更快的收敛速度且mAP提高了3.06%（95.47%-98.53%）； ResNet-SSD在召回率为98.8%时， 检测准确度可达98.5%， 达到了在交并比(Intersection over union, IoU)为0.5时较VGG16-SSD高、 与Faster-RCNN相同的评价指标； 就检测速度而言， 该方法为Faster-RCNN的 6倍， VGG16-SSD的 1.4倍。 该方法可满足铁路扣件实时自动检测的需求。
关键词：铁路扣件； 虚拟图像； 图像识别； 残差网络； 单发多框检测器
引用格式：WANG Dengfei， SU Hongsheng， CHEN Dengke， et al． A method of railway fastener defect detection based on ResNet-SSD． Journal of Measurement Science and Instrumentation， 2023， 14（3）： 360-368. DOI： 10．3969／j．issn．1674-8042．2023．03．013
[full text view]
