浅层卷积神经网络融合Transformer的金属缺陷图像识别方法

doi:10.3969/j.issn.1004-132X.2022.19.003

摘要/Abstract

摘要： 针对金属缺陷识别领域中传统深度学习方法存在参数量多、计算量大的问题，提出了一种浅层卷积神经网络融合Transformer模型的金属缺陷识别方法。利用浅层卷积神经网络学习图像局部信息与位置信息，通过Transformer学习图像全局信息，同时引入通道注意力模块SE关注重要特征通道，实现缺陷图像识别。通过引入公开缺陷数据集验证该方法的有效性，同时利用自建缺陷超声数据集验证所提方法的通用性。实验结果表明，在中小规模数据集上，该方法通用性较强，能够对金属缺陷图像进行有效识别。

关键词: 金属缺陷识别, 深度学习, 卷积神经网络, Transformer模型, 多头注意力

Abstract: Aiming at the problems of large amount of parameters and calculation in the field of metal defect recognition， a metal defect recognition method was proposed based on shallow CNN and Transformer model. The shallow CNN was used to learn the local information and position information of the images， and the Transformer was used to learn the global information of the images. At the same time， the channel attention module SE was introduced to pay attention to the important feature channels to realize the defect image recognition. The effectiveness of this method was verified by introducing the open defect data set， and the universality of this method was verified by using the self-built defect ultrasonic data set. The experimental results show that the proposed method has strong universality and may effectively identify metal defect images on small and medium-sized data sets.

Key words: metal defect identification, deep learning, convolutional neural network（CNN）, Transformer model, multi-head attention

中图分类号:

TP391

唐东林, 杨洲, 程衡, 刘铭璇, 周立, 丁超. 浅层卷积神经网络融合Transformer的金属缺陷图像识别方法[J]. 中国机械工程, 2022, 33(19): 2298-2305，2316.

TANG Donglin, YANG Zhou, CHENG Heng, LIU Mingxuan, ZHOU Li, DING Chao. Metal Defect Image Recognition Method Based on Shallow CNN Fusion Transformer[J]. China Mechanical Engineering, 2022, 33(19): 2298-2305，2316.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.cmemo.org.cn/CN/10.3969/j.issn.1004-132X.2022.19.003

https://www.cmemo.org.cn/CN/Y2022/V33/I19/2298

参考文献

［1］LYU Xiaoming， DUAN Fajie， JIANG Jiajia， et al. Deep Metallic Surface Defect Detection：the New Benchmark and Detection Network［J］. Sensors， 2020， 20（6）：1562.
［2］LIN H I， WIBOWO F S. Image Data Assessment Approach for Deep Learning-based Metal Surface Defect-detection Systems［J］. IEEE Access， 2021， 9：47621-47638.
［3］唐东林，周立，吴续龙，等. CNN融合PCA-DT模型的金属缺陷识别研究［J/OL］. 机械科学与技术：1-8［2021-11-12］. https：∥doi. org/10. 13433/j. cnki. 1003-8728. 20200478.
TANG Donglin， ZHOU Li， WU Xulong， et al. Research on Metal Defect Recognition of CNN Fusion PCA-DT Model［J/OL］. Mechanical Science and Technology for Aerospace Engineering：1-8［2021-11-12］. https：∥doi. org/10. 13433/j. cnki. 1003-8728. 20200478.
［4］GRZEGORZ P. Multi-Sensor Data Integration Using Deep Learning for Characterization of Defects in Steel Elements［J］. Sensors， 2018， 18（2）：292.
［5］WANG J， FU P， GAO R X. Machine Vision Intelligence for Product Defect Inspection Based on Deep Learning and Hough Transform［J］. Journal of Manufacturing Systems， 2019， 51：52-60.
［6］WEI X， YANG Z， LIU Y， et al. Railway Track Fastener Defect Detection Based on Image Processing and Deep Learning Techniques：a comparative study［J］. Engineering Applications of Artificial Intelligence， 2019， 80：66-81.
［7］GU J， WANG Z， KUEN J， et al. Recent Advances in Convolutional Neural Networks［J］. Pattern Recognition：the Journal of the Pattern Recognition Society， 2018， 77：354-377.
［8］TAO X， ZHANG D， MA W， et al. Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks［J］. Applied Sciences， 2018， 8（9）：1575.
［9］GUANS， LEI M， LU H. A Steel Surface Defect Recognition Algorithm Based on Improved Deep Learning Network Model Using Feature Visualization and Quality Evaluation［J］. IEEE Access， 2020， 8：49585-49895.
［10］WANG S， XIA X， YE L， et al. Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks［J］. Metals -open Access Metallurgy Journal， 2021， 11（3）：388.
［11］姜洪权，贺帅，高建民，等. 一种改进卷积神经网络模型的焊缝缺陷识别方法［J］. 机械工程学报， 2020， 56（8）：235-242.
JIANG Hongquan， HE Shuai， GAO Jianmin， et al. An Improved Convolutional Neural Network for Weld Defect Recognition［J］. Journal of Mechanical Engineering， 2020， 56（8）：235-242.
［12］HE D， XU K， WANG D. Design of Multi-scale Receptive Field Convolutional Neural Network for Surface Inspection of Hot Rolled Steels［J］. Imageand Vision Computing， 2019， 89：12-20.
［13］刘涵，郭润元. 基于X射线图像和卷积神经网络的石油钢管焊缝缺陷检测与识别［J］. 仪器仪表学报， 2018， 39（4）：247-256.
LIU Han， GUO Runyuan. Detection and Identification of SAWH Pipe Weld Defects Based on X-ray Image and CNN［J］. Chinese Journal of Scientific Instrument， 2018， 39（4）：247-256.
［14］樊丁，胡桉得，黄健康，等. 基于改进卷积神经网络的管焊缝X射线图像缺陷识别方法［J］. 焊接学报， 2020， 41（1）：7-11，97.
FAN Ding， HU Ande， HUANG Jiankang， et al. X-ray Image Defect Recognition Method for Pipe Weld Based on Improved Convolutional Neural Network［J］. Transactions of the China Welding Institution， 2020， 41（1）：7-11，97.
［15］YUN J P， SHIN W C， KOO G， et al. Automated Defect Inspection System for Metal Surfaces Based on Deep Learning and Data Augmentation［J］. Journal of Manufacturing Systems， 2020， 55：317-324.
［16］VASWANI A， SHAZEER N， PARMAR N， et al. Attention is All You Need［C］∥Advances in Neural Information Processing Systems. Long Beach， CA， 2017：6000-6010.
［17］DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An Image is Worth 16×16 Words：Transformers for Image Recognition at Scale［J］. arXiv Preprint， arXiv：2010. 11929， 2020.
［18］TOUVRON H, CORD M, DOUZE M, et al. Training Data-efficient Image Transformers &Distillation through Attention［C］∥Proceedings of the International Conference on Machine Learning (ICML). ［Online Confenerce］, 2021, 139:10347-10357.
［19］ZHOU D， KANG B， JIN X， et al. DeepViT：towards Deeper Vision Transformer［J］. arXiv Preprint， arXiv：2013. 11886， 2021.
［20］TOUVRON H, CORDM, SABLAYROLLES A, et al. Going Deeper with Image Transformers［C］∥Proceedings of the 18th IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada, 2021:32-42.
［21］HU J， SHEN L， SUN G. Squeeze-and-excitation Networks［C］∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Salt Lake City， UT, 2018：7132-7141.
［22］HE Y， SONG K， MENG Q， et al. An End-to-end Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features［J］. IEEE Transactions on Instrumentation and Measurement， 2019， 69（4）：1493-1504.
［23］SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-cam：Visual Explanations from Deep Networks via Gradient-based Localization［C］∥Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy, 2017：618-626.
［24］SIMONYAN K， ZISSERMAN A. Very Deep Convolutional Networks for Large-scale Image Recognition［J］. arXiv Preprint, arXiv:1409. 1556, 2014.
［25］SZEGEDY C, LIU W, JIA Y Q, et al. Going Deeper with Convolutions［C］∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston, MA, 2015:1-9.
［26］HE K， ZHANG X， REN S， et al. Deep Residual Learning for Image Recognition［C］∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, WA, 2016：770-778.
［27］SANDLER M， HOWARD A， ZHU M， et al. MobileNetv2：Inverted Residuals and Linear Bottlenecks［C］∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, 2018：4510-4520.
［28］MA N， ZHANG X， ZHENG H T， et al. ShuffleNet v2：Practical Guidelines for Efficient CNN Architecture Design［C］∥Proceedings of the European Conference on Computer Vision（ECCV）. Munich, Germany, 2018：116-131.

[1]	王浩, 罗浩东, 施亚中, 王立文, 张威, 王忠. 基于端到端深度学习模型TOPO-U型网的结构拓扑优化方法[J]. 中国机械工程, 2026, 37(1): 174-183.
[2]	肖御风, 张超勇, 赛希亚拉图null, 孟一帆, 朱传军. 基于YOLOv11-Seg与Transformer模型的刀具磨损多步向前实时预测方法[J]. 中国机械工程, 2025, 36(12): 2944-2951.
[3]	张雄, 董乐聪, 王文强, 渠伟瀅, 万书亭. 融合二维卷积与门控循环神经网络的滚动轴承多故障诊断方法[J]. 中国机械工程, 2025, 36(12): 2978-2985.
[4]	闫祖龙, 庞启龙, 熊建龙. 基于深度学习的KDP晶体三维已加工表面形貌预测[J]. 中国机械工程, 2025, 36(10): 2329-2334.
[5]	王坚坚, 廖与禾, 杨磊, 薛久涛. 基于多维复向特征融合与CNN-GRU的转子不平衡量识别方法[J]. 中国机械工程, 2025, 36(09): 1905-1915.
[6]	岳子桐, 李艳婷, 赵宇. 基于神经网络和稳健估计的风电机组状态监测[J]. 中国机械工程, 2025, 36(08): 1842-1852.
[7]	赵昀杰, 贺岩松, 张志飞, 徐中明. 基于生成模型的三维波束形成图像压缩方法[J]. 中国机械工程, 2025, 36(07): 1520-1529.
[8]	杨赫然1, 2, 张培杰1, 2, 孙兴伟1, 2, 潘飞1, 2, 刘寅1, 2. 利用改进卷积神经网络的螺杆砂带磨削表面粗糙度预测[J]. 中国机械工程, 2025, 36(02): 325-332.
[9]	曾浩, 曹华军, 董俭雄. 基于ISABO-IBiLSTM模型的刀具磨损预测方法[J]. 中国机械工程, 2024, 35(11): 1995-2006.
[10]	李悦1, 2, 谢恒1, 周公博1, 2, 周坪1, 2, 李猛钢1, 2. 基于半监督贝叶斯Transformer的刀具磨损软测量及不确定性分析方法[J]. 中国机械工程, 2024, 35(11): 2015-2025.
[11]	郭海宇1, 邹圣公1, 张晓光2, 3, 4, 陆凡凡2, 陈洋2, 王涵2, 徐新志2. 基于多源小波变换神经网络的旋转机械轴承故障诊断[J]. 中国机械工程, 2024, 35(11): 2026-2034.
[12]	阴艳超, 施成娟, 邹朝普, 刘孝保. 基于深度时间卷积神经网络与迁移学习的流程制造工艺过程质量时序关联预测[J]. 中国机械工程, 2023, 34(14): 1659-1671.
[13]	贺青川, 刘慧, 潘骏, 陈文华. 基于电机运行状态的电静压伺服机构内泄漏检测方法[J]. 中国机械工程, 2023, 34(12): 1407-1414.
[14]	聂昕, 谭天, 申丹凤. 基于深度学习的汽车梁类件冲压回弹研究[J]. 中国机械工程, 2023, 34(07): 838-846.
[15]	曾俊玮, 季元进, 任利惠, 葛方顺, 孙泽良, 黄章行. 融合一维卷积神经网络和双向门控循环单元的APM车辆轮胎径向载荷识别方法[J]. 中国机械工程, 2023, 34(03): 359-368.