TONG Zhixue, CHEN Jiajie, KANG Zhiqiang, QU Dongdong
(School of Mechanical and Electrical Engineering, Xi’an University of Architecture and Technology, Xi’an 710055, China)
Abstract: The positioning of coal-carrying railway carriages is a critical technology for coal sampling robots to achieve autonomous sampling. A rapid positioning method for coal-carrying railway carriages combining binocular stereo vision and improved YOLOv4-tiny is proposed. The left view RGB image of the binocular camera is taken as the input of the improved YOLOv4-tiny. The position information of the coal-carrying railway car in the image is obtained. And its spatial distribution position is calculated based on the binocular positioning principle. The improved algorithm carries out a lightweight design for YOLOv4-tiny, adopts a new activation function, and uses the K-mans clustering algorithm to regenerate the prior anchor box. Compared with the original network, the model size has been reduced by 62.5%, the accuracy rate has been increased by 0.92%, the recall rate has been increased by 0.89%, and the detection speed has been increased by 20%. The average positioning error of the coal-carrying car within 10 m is 4.1%, and the maximum error is 8.3%. The improved model has the advantages of robustness, fast recognition speed, lightweight, etc., and can realize the dynamic recognition of the coal-carrying car. The positioning provides a guarantee for the autonomous sampling of coal sampling robots.
Key words: positioning of coal-carrying railway carriages; deep learning; binocular vision; object detection; YOLOv4-tiny
References
[1]ZHAO P, WANG Y. Research and selection analysis of automatic sampler in coal preparation plant. Coal Technology, 2021, 40(6): 194-196.
[2]ZHAO T. Research on key parameter detection method of mobile sampler based on binocular vision. Xi’an: Xi’an University of Architecture and Technology, 2019.
[3]ZHAO W Y, LI B J. Front vehicle detection and tracking based on fusion of millimeter wave radar and monocular vision. Journal of Wuhan University (Information Science Edition), 2019, 44(12): 1832-1840.
[4]SANG Z, ZHOU Y H. Research on real-time ranging of moving vehicles based on video. Journal of Beijing Jiaotong University, 2020, 44(5): 133-140.
[5]WANG X, WANG S, CAO J. Data-driven based tiny-YOLOv3 method for front vehicle detection inducing SPP-net. IEEE Access, 2020, 8: 110227-110236.
[6]JI H, ZENG X, LI H, et al. Human abnormal behavior detection method based on T-TINY-YOLO//ICMIP 2020: 5th International Conference on Multimedia and Image Processing, Jan. 10-12, 2020, Nanjing, China. New York: Association for Computing Machinery, 2020: 1-5.
[7]ZHAO Z Q, ZHENG P, XU S T, et al. Object detection with deep learning: A review. Computer Vision and Pattern Recognition, 2018, arXiv: 1807.05511.
[8]GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation//IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23-28, 2014, Columbus, OH, USA. New York: IEEE, 2014: 580-587.
[9]GIRSHICK R. Fast R-CNN//IEEE International Conference on Computer Vision (ICCV), Dec. 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 1440-1448.
[10]REN S P, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[11]REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 779-788.
[12]LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector//Computer Vision-ECCV 2016, Oct. 11-14, 2016, Amsterdam, Netherlands. Berlin: Springer, 2016: 21-37.
[13]REDMON J, FARHADI A. YOLO9000: Better, faster,stronger. (2016-12-25)[2022-05-13]. https://arxiv. org/abs/1612.08242.
[14]REDMON J, FARHADI A. Yolov3: An incremental improvement. Computer Vision and Pattern Recognition, 2018, arXiv: 1804.02767.
[15]WU Z, SHEN C, HENGEL A V D.Wider or deeper: revisiting the resnet model for visual recognition. Pattern Recognition, 2016, 90: 119-133.
[16]BOCHKOVSKIY A, WANG C Y, LIAO H. YOLOv4: Optimal speed and accuracy of object detection. Computer Vision and Pattern Recognition, 2020, arXiv: 2004.10934.
[17]WANG C Y, BOCHKOVSKIY A, LIAO H. Scaled-YOLOv4: Scaling cross stage partial network. Computer Vision and Pattern Recognition, 2020: arXiv: 2011.08036.
[18]ZHOU C. Research on image processing in car positioning. Computer System Applications, 2011, 20(8): 219-221.
[19]LU J N, SHAN D X. Study on position detection of train carriage based on semantic segmentation. Journal of Engineering Design, 2020, 27(5): 568-576.
基于双目视觉的载煤火车厢动态识别与定位
同志学, 陈佳杰, 康智强, 屈东东
(西安建筑科技大学 机电工程学院, 陕西 西安 710055)
摘要:载煤火车厢定位是煤炭采样机器人实现自主采样的关键技术之一。 针对载煤火车厢的动态识别与定位问题, 本文提出了一种双目立体视觉与改进YOLOv4-tiny结合的快速定位方法。 该方法将双目相机左视图RGB图像作为改进YOLOv4-tiny的输入, 得到载煤车厢在图像中的位置信息, 再结合双目定位原理计算出其空间位置。 改进后的算法主要对YOLOv4-tiny进行轻量化设计, 采用了新的激活函数, 并利用K-mans聚类算法重新生成样本先验锚框。 与原网络相比, 模型大小减少了62.5%, 准确率提高了0.92%, 召回率提高了0.89%, 检测速度提高了20%。 载煤火车厢在10 m以内的平均定位误差为4.1%, 最大误差为8.3%。 改进后的模型具有鲁棒性强、 识别速度快、 轻量化等优点, 能够实现载煤火车厢的动态识别及定位, 为煤炭采样机器人的自主采样提供了保证。
关键词:载煤火车厢定位; 深度学习; 双目视觉; 目标检测; YOLOv4-tiny
引用格式:TONG Zhixue, CHEN Jiajie, KANG Zhiqiang, et al. Dynamic recognition and positioning of coal-carrying railway carriages based on binocular vision. Journal of Measurement Science and Instrumentation, 2023, 14(2): 148-155. DOI: 10.3969/j.issn.1674-8042.2023.02.003
[full text view]