A New Transfer Learning Method with Residual Attention and Its Applications on Rolling Bearing Fault Diagnosis

doi:10.3969/j.issn.1004-132X.2023.03.010

Abstract

Abstract: A transfer learning algorithm was proposed based on class-specific residual attention convolutional neural networks(CSRA-CNN) to improve the fault diagnosis accuracy of rolling bearings. Residual attention mechanism was added to the convolutional neural network model, which made the model pay more attention to fault feature extraction in the training, and also improves the migration accuracy effectively. To evaluate the performance of the proposed method, the results were compared with traditional convolutional neural network under different transfer learning strategies. The proposed algorithm was verified by fault diagnosis integrated test bench of power transmission system and high-speed train comprehensive test bench. The results show that the proposed method may complete the transfer learning of different health states of bearings under variable speed and variable speed and load, and the transfer effect is superior to that of the traditional convolutional neural network.

Key words: transfer learning, bearing fault diagnosis, residual attention, feature extraction

摘要： 提出了一种基于残差注意力卷积神经网络(CSRA-CNN)的迁移学习算法，用于提高滚动轴承的故障诊断精度。在卷积神经网络模型中加入残差注意力机制，使模型在训练过程中更加注重故障特征的提取，从而有效提高迁移准确率。为了测评基于残差注意力卷积神经网络的性能，将其与传统卷积神经网络在不同迁移学习策略下的结果进行对比。用动力传动故障诊断综合实验台和高速列车综合实验台对所提算法进行了验证，该方法可以完成变转速以及变转速变载荷下轴承不同健康状态的迁移学习，且迁移效果均优于传统的卷积神经网络。

关键词: 迁移学习, 轴承故障诊断, 残差注意力, 特征提取

CLC Number:

TP182

ZHAO Jing, YANG Shaopu, LI Qiang, LIU Yongqiang, . A New Transfer Learning Method with Residual Attention and Its Applications on Rolling Bearing Fault Diagnosis[J]. China Mechanical Engineering, 2023, 34(03): 332-343.

赵靖, 杨绍普, 李强, 刘永强, . 一种残差注意力迁移学习方法及其在滚动轴承故障诊断中的应用[J]. 中国机械工程, 2023, 34(03): 332-343.

References

［1］雷亚国, 贾峰, 孔德同, 等. 大数据下机械智能故障诊断的机遇与挑战［J］. 机械工程学报, 2018, 54(5):94-104.
LEI Yaguo, JIA Feng, KONG Detong, et al. Opportunities and Challenges of Machinery Intelligent Fault Diagnosis in Big Data Era［J］. Journal of Mechanical Engineering, 2018, 54(5):94-104.
［2］ZHAO Rui, YAN Ruqiang, CHEN Zhenghua, et al. Deep Learning and Its Applications to Machine Health Monitoring［J］. Mechanical Systems and Signal Processing, 2019, 115:213-237.
［3］LEI Yaguo,YANG Bin, JIANG Xinwei, et al. Applications of Machine Learning to Machine Fault Diagnosis:a Review and Roadmap［J］. Mechanical Systems and Signal Processing, 2020, 138:1-39.
［4］AZADEH A, SABERI M, KAZEM A, et al. A Flexible Algorithm for Fault Diagnosis in a Centrifugal Pump with Corrupted Data and Noise Based on ANN and Support Vector Machine with Hyper-parameters Optimization［J］. Applied Soft Computing, 2013, 13(3):1478-1485.
［5］WANG Dong. K-nearest Neighbors Based Methods for Identification of Different Gear Crack Levels under Different Motor Speeds and Loads:Revisited, Mechanical Systems and Signal Processing［J］. 2016, 70/71:201-208.
［6］CHEN Zhuyun, GRYLLIAS K, LI Weihua. MechanicalFault Diagnosis Using Convolutional Neural Networks and Extreme Learning Machine［J］. Mechanical Systems and Signal Processing, 2019, 133:1-21.
［7］ZHAO Jing, YANG Shaopu, LI Qiang, et al. A New Bearing Fault Diagnosis Method Based on Signal-to-image Mapping and Convolutional Neural Network［J］. Measurement, 2021, 176:1-15.
［8］XIE Jiaqi, DU Guifu, SHEN Changqing, et al. An End-to-end Model Based on Improved Adaptive Deep Belief Network and Its Application to Bearing Fault Diagnosis［J］. IEEE Access, 2018, 6:63584-63596.
［9］赵志宏, 李乐豪, 杨绍普, 等. 一种频域特征提取自编码器及其在故障诊断中的应用研究［J］.中国机械工程, 2021, 32(20):2468-2474.
ZHAO Zhihong, LI Lehao, YANG Shaopu, et al. A Frequency Domain Feature Extraction Auto-encoder and Its Applications on Fault Diagnosis［J］. China Mechanical Engineering, 2021, 32(20):2468-2474.
［10］雷亚国, 杨彬, 杜兆钧, 等. 大数据下机械装备故障的深度迁移诊断方法［J］. 机械工程学报, 2019, 55(7):1-8.
LEI Yaguo, YANG Bin, DU Zhaojun, et al. Deep Transfer Diagnosis Method for Machinery in Big Data Era［J］. Journal of Mechanical Engineering, 2019, 55(7):1-8.
［11］LONG Mingshen, CAO Yue, WANG Jianmin, et al. Learning Transferable Features with Deep Adaptation Networks［C］∥International Conference on Machine Learning. Lille, 2015:97-105.
［12］SUN B, SAENKO K. Deep CORAL:Correlation Alignment for Deep Domain Adaptation［C］∥European Conference on Computer Vision. Amsterdam, 2016:443-450.
［13］LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Deep Transfer Learning with Joint Adaptation Networks［C］∥International Conference on Machine Learning. Sydney, 2017:3470-3479.
［14］GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial Training of Neural Networks［J］. Journal of Machine Learning Research, 2016, 17(1):2096-2030.
［15］CHEN Z Y, GRYLLIAS K, LI W H. Intelligent Fault Diagnosis for Rotary Machinery Using Transferable Convolutional Neural Network［J］. IEEE Transactions on Industrial Informatics, 2020, 16(1):339-349.
［16］LI X Q, JIANG H K, WANG R X, et al. Rolling Bearing Fault Diagnosis Using Optimal Ensemble Deep Transfer Network［J］. Knowledge-Based Systems, 2021, 213:1-10.
［17］揭震国，王细洋，龚廷恺. 基于深度学习与子域适配的齿轮故障诊断［J］.中国机械工程, 2021, 32(22):2716-2723.
JIE Zhenguo, WANG Xiyang, GONG Tingkai. Gear Fault Diagnosis Based on Deep Learning and Subdomain Adaptation［J］. China Mechanical Engineering, 2021, 32(22):2716-2723.
［18］HAN Te, LIU Chao, YANG Wenguang, et al. ANovel Adversarial Learning Framework in Deep Convolutional Neural Network for Intelligent Diagnosis of Mechanical Faults［J］. Knowledge-Based Systems, 2019, 165:474-487.
［19］王晋东，陈益强. 迁移学习导论［M］. 北京：电子工业出版社，2021:21-28, 99-105.
WANG Jindong, CHEN Yiqiang. Introduction to Transfer Learning［M］. Beijing:Publishing House of Electronics Industry, 2021:21-28, 99-105.
［20］KRIZHEVSKY A, SUTSKEVER I, HINTON G. Imagenet Classification with Deep Convolutional Neural Networks［J］. Communications of the ACM, 2017, 60(6):84-90.
［21］邱锡鹏. 神经网络与深度学习［M］. 北京：机械工业出版社, 2019:113-125.
QIU Xipeng.Neural Networks and Deep Learning［M］. Beijing:China Machine Press, 2019:113-125.
［22］ZHU Ke, WU Jianxin. Residual Attention:a Simple But Effective Method for Multi-label Recognition［C］∥International Conference on Computer Vision. Montreal, 2021:184-193.
［23］SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization［J］. International Journal of Computer Vision, 2020, 128(2):336-359.

[1]	LI Ke, , YAN Han, GU Jiefei, , SU Lei, SU Wensheng, XUE Zhigang. Research on Multi-source Transfer Learning Bearing Fault Diagnosis Based on Shapelets Time Series [J]. China Mechanical Engineering, 2022, 33(24): 2990-2996,3006.
[2]	ZHANG Kang, TIAN Zeyu, CHEN Xiangmin, LIAO Lida, WU Jiateng. Application of MMD in Gear Fault Feature Extraction under Variable Rotating Speed Working Conditions [J]. China Mechanical Engineering, 2022, 33(20): 2483-2491.
[3]	WEN Jiangtao, ZHANG Pengcheng, SUN Jiedi, LEI Ming. Unsupervised Transfer Learning with Residual Convolutional Autoencoder Networks for Bearing Fault Diagnosis [J]. China Mechanical Engineering, 2022, 33(14): 1707-1716.
[4]	CHEN Dongmei, ZHAO Siheng, WEI Chengyin, CHEN Yajie. Research and Applications of Condition Monitoring and Predictive Maintenance of Marine Diesel Engines [J]. China Mechanical Engineering, 2022, 33(10): 1162-1168.
[5]	JIANG Jiawei, HU Yihuai, FANG Yunhu, ZHANG Chen, RUI Xiaosong, WANG Meng. Fault Diagnosis Method of Marine Fans Based on MTAD and Fuzzy Entropy [J]. China Mechanical Engineering, 2022, 33(10): 1178-1188.
[6]	LIN Jingliang, HUANG Yunbao, LI Haiyan, ZHOU Sheng, HUANG Zeying. Design Optimization for Hydraulic Systems of Forklift Boom Based on Deep Surrogate Model [J]. China Mechanical Engineering, 2022, 33(03): 290-298.
[7]	ZHANG Long, CAI Binghuan, XIONG Guoliang, WANG Chaobing, HU Junfeng. Multi-stage Noise Reduction Method with ESMK for Adaptive Feature Extraction of Rolling Bearings [J]. China Mechanical Engineering, 2021, 32(24): 2950-2959.
[8]	ZHAO Zhihong, LI Lehao, YANG Shaopu, LI Qing. A Frequency Domain Feature Extraction Auto-encoder and Its Applications on Fault Diagnosis [J]. China Mechanical Engineering, 2021, 32(20): 2468-2474.
[9]	LI Congbo, WANG Rui, ZHANG You, JIANG Lijun , SUN Hao. A Novel Fault Early Warning Method for Centrifugal Blowers Based on Transfer Learning [J]. China Mechanical Engineering, 2021, 32(17): 2090-2099,2107.
[10]	HE Qiang, TANG Xianghong, LI Chuanjiang, LU Jianguang, CHEN Jiadui. Bearing Fault Diagnosis Method Based on Small Sample Data under Unbalanced Loads#br# [J]. China Mechanical Engineering, 2021, 32(10): 1164-1171,1180.
[11]	XU Peng, FANG Zhou, WANG Ping, GENG Ming, XU Yong. Quantitative Evaluation Method of Defects Based on Transient Characteristics of PMFL [J]. China Mechanical Engineering, 2021, 32(07): 860-866,881.
[12]	JIANG Zhinong1;WANG Zijia1;ZHANG Jinjie2;Huang Yifei3;MAO Zhiwei2. Fault Diagnosis Method of EDG in Nuclear Power Plants Based on Energy Operator Gradient Neighborhood Feature Extraction [J]. China Mechanical Engineering, 2021, 32(05): 617-623.
[13]	TANG Guiji;LI Nannan;WANG Xiaolong. Fault Feature Extraction Method for Gears Based on ISSD and SVD [J]. China Mechanical Engineering, 2020, 31(24): 2988-2996.
[14]	WAN Shuting;PENG Bo;WANG Xiaolong. Early Fault Diagnosis Method of Rolling Bearings Based on DTD Transform-SCS Method [J]. China Mechanical Engineering, 2020, 31(23): 2829-2836.
[15]	ZHANG Chaoyang;JI Weixi;PENG Wei. Decision-making Method for Energy-saving Control of CNC Machine Tools Based on Transfer Learning [J]. China Mechanical Engineering, 2020, 31(23): 2855-2863.