HAO Zhengqing1， WANG Ying1,2， CHEN Xiaoqiang1,2， XIONG Ye1
（1. School of Automation & Electrical Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China； 2. Key Lab of Opt-Electronic Technology and Intelligent Control, Ministry of Education, Lanzhou Jiaotong Universtiy, Lanzhou 730070, China）
Abstract： In order to improve the accuracy of train driver fatigue detection, a method of train driver fatigue detection based on facial multi-information fusion is proposed. Firstly, low-light enhancement is used for image preprocessing, and human faces are detected by local binary patterns (LBP) feature. Secondly, the driver’s facial feature points are obtained by ensemble of regression trees (ERT) algorithm, afnd face model matching is used to obtain the driver’s head posture angle. Finally, according to the special driving environment of the train driver, adaptive threshold correction and eye gaze correction are carried out for the eye characteristic quantities that best show fatigue. The fuzzy inference system is used as a fusion tool, and the features of eyes, mouth and head are used as the input of the fuzzy inference system, and the driver’s fatigue value is used as the detection results. Experiment results show that the detection method can distinguish driver fatigue levels with accuracy rates of 95% in normal environments and 86.8% in low-light environments.
Key words： train driver; fatigue detection; feature point detection; head posture; facial multi-information fusion
References
［1］YANG H, CHEN X Q, LEI J Y, et al. A fatigue detection method for train drivers based on inverse projection correction and eye gaze correction. Journal of the China Railway Society, 2018, 40(4): 83-89.
［2］WANG F, WU S C, PING J Y, et al. EEG driving fatigue detection with PDC-based brain functional network. IEEE Sensors Journal, 2021, 21(9): 10811-10823.
［3］CAI S X, DU C K, ZHOU S Y, et al. Fatigue driving state dection based on vehicle running data. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(4): 77-82.
［4］LI K N, GONG Y B, REN Z L. A fatigue driving detection algorithm based on facial multi-feature fusion. IEEE Access, 2020(8): 101244-101259.
［5］FANG B, XU S, FENG X F. Self-calibration driver fatigue detection for advanced RISC machine(ARM) platform. Journal Automotive Safety and Energy, 2020, 11(1): 71-78.
［6］LIU Y Y, ZHU Y, SUN J B, et al. Haar-like feature dual-threshold Adaboost face detection. Chinese Journal of Graphics, 2020, 25(8): 1618-1626.
［7］SONG Y, LU L Y. A fatigue warning alogorithm based on spatiotemporal ferture extraction of facial motion. Computer Engineering & Science, 2019, 41(10) : 1763-1770.
［8］JI Y Y, WANG S G, ZHAO Y, et al. Fatigue state detection based on multi-index fusion and state recognition network. IEEE Access, 2019(7): 64136-64147.
［9］LI X, HONG L, WANG J C, et al. Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning. IET Intelligent Transport Systems, 2019, 13(9): 1401-1409.
［10］ZHAO X P, MENG C N, Mingkui FENG M K. Fatigue detection based on cascade convolutional neural network. Journal of Optoelectronics Laser, 2017, 28(5): 497-502.
［11］PAN Z G, LIU R F, ZHANG M M. Research on fatigue detection algorithm based on fuzzy comprehensive evaluation. Journal of Software, 2019, 30(10): 2954 -2963.
［12］SIMA Y, YI J Z, CHEN A B, et al. Localization and recognition of expression-complete frames in dynamic face image sequences. Journal of Applied Sciences, 2021, 39(3): 357-366.
［13］WU M J, MU P G, ZHANG C Y. Driver fatigue detection algorithm based on the states of eyes and mouth. Computer Application and Software, 2013, 30(3): 25-27, 54.
［14］QIANG L. Research on driving fatigue detection of train drivers based on PERCLOS. Beijing: Beijing Jiaotong University, 2014: 61-68.
［15］ZHENG J W, LIU X M, YIN J L. Pain expression recognition based on LBP and SVM. Computer System Applications, 2021, 30(4): 111-117.
［16］KAZEMI V, SULLIVAN J. One millisecond face alignment with an ensemble of regression trees//2014 IEEE Conference on Computer Vision and Pattern Recognition, Jun.24-29, 2014, Columbus, USA. New York: IEEE, 2014: 1867-1874.
［17］SOUKUPOV T. Real-time eye blink detection using facial landmarks. Czech: Czech Technical University, 2016.
［18］WU K L. Fatigue driving detection based on facial features and head posture. Shanghai: Donghua University, 2020.
基于面部多信息融合的列车司机疲劳检测
郝正清1， 王英1,2， 陈小强1,2， 熊烨1
（1. 兰州交通大学 自动化与电气工程学院， 甘肃 兰州 730070； 2. 兰州交通大学 光电技术与智能控制教育部重点实验室， 甘肃 兰州 730070）
摘要：为了提高列车司机行车疲劳检测的准确性， 提出一种面部多信息融合的列车司机疲劳检测方法。 首先， 采用低光增强进行图像预处理， 以基于局部二值模式（LBP）特征检测人脸。 其次， 利用回归树集合（ERT）算法获取司机人脸特征点， 并通过人脸模型匹配得到司机头部姿态角。 最后， 根据列车司机的特殊驾驶环境， 对最能表现疲劳的眼睛特征量进行自适应阈值修正和眼睛凝视修正， 以模糊推理系统为融合工具， 以眼睛、 嘴巴、 头部姿态特征量作为模糊推理系统输入， 得到司机的疲劳值作为检测结果。 实验结果表明， 该检测方法可以区分司机疲劳等级， 在正常环境下准确率为95%， 在低光照环境下的准确率为86.8%。
关键词：列车司机； 疲劳检测； 特征点检测； 头部姿态； 面部多信息融合
引用格式：HAO Zhengqing， WANG Ying， CHEN Xiaoqiang， et al． Train driver fatigue detection based on facial multi-information fusion． Journal of Measurement Science and Instrumentation， 2023， 14（4）： 493-500. DOI： 10．3969／j．issn．1674-8042．2023．04．012
[full text view]
