Human Factor Engineering for Human-Cyber-Physical System Collaboration in Intelligent Manufacturing

doi:10.3969/j.issn.1004-132X.2023.14.008

Abstract

Abstract: The theoretical system of intelligent manufacturing for HCPS confirmed the central position of human in the intelligent manufacturing system. Starting from the demand of human-machine collaboration in the intelligent manufacturing system, the emphases of human factors in HCIM were discussed from three levels such as behavior, intention, and cognition, based on the theory of gulf. Focusing on virtual-real fusion scenarios, multimodal human-machine interaction, cognitive quantification and other methods, the importance of human factor engineering in promoting the integration of human-computer intelligence was expounded. Finally, research direction and development suggestions of human-centered intelligent manufacturing from the implementation of HCPS intelligent manufacturing systems were put forward.

Key words: intelligent manufacturing, human-cyber-physical system(HCPS), human factor engineering, gulf of human-computer interaction, human-centered intelligent manufacturing(HCIM), virtual-real fusion

摘要： 人信息物理系统(HCPS)的智能制造理论体系明确了人在智能制造系统的中心地位。从智能制造人机协同的需求出发，基于人机交互鸿沟理论，在行为、意图、认知三个层面讨论了人因工程在“人本智造”中的研究重点。围绕虚实融合场景、多模态人机交互及认知量化等方法，阐述了人因工程在促进HCPS智能融合的重要作用。最后，从实现HCPS集成的智能制造系统出发，提出了“人本智造”的研究方向和学科发展建议。

关键词: 智能制造, 人信息物理系统, 人因工程, 人机交互鸿沟, 人本智造, 虚实融合

CLC Number:

TP16
TB18

YANG Xiaonan, FANG Haonan, LI Jianguo, XUE Qing. Human Factor Engineering for Human-Cyber-Physical System Collaboration in Intelligent Manufacturing[J]. China Mechanical Engineering, 2023, 34(14): 1710-1722,1740.

杨晓楠, 房浩楠, 李建国, 薛庆. 智能制造中的人-信息-物理系统协同的人因工程[J]. 中国机械工程, 2023, 34(14): 1710-1722,1740.

References

［1］范文健，韦震海，曹锐锋.模块化汽车总装生产线布局研究［J］. 科技创新与应用，2021，11(25)：71-75.
FAN Wenjian，WEI Zhenhai，CAO Ruifeng.Research on the Layout of Modular Automobile Assembly Production Line［J］. Technology Innovation and Application，2021，11(25)：71-75.
［2］白磊，刘勇.3C自动化设备的模块化设计应用与研究［J］. 设备管理与维修，2021(11)：137-138.
BAI Lei，LIU Yong. Application and Research of Modular Design of 3C Automation Equipment［J］. Equipment Management and Maintenance，2021(11)：137-138.
［3］HARLEY O，LIU Ying，MANEESH K，et al. A Framework of Integrating Knowledge of Human Factors to Facilitate HMI and Collaboration in Intelligent Manufacturing［J］. Procedia CIRP，2018(72)：135-140.
［4］姚锡凡，马南峰，张存吉，等. 以人为本的智能制造：演进与展望［J］. 机械工程学报，2022，58(18)：2-15.
YAO Xifan，MA Nanfeng，ZHANG Cunji，et al. Human-centric Smart Manufacturing： Evolution and Outlook［J］. Journal of Mechanical Engineering，2022，58(18)：2-15.
［5］杨赓，周慧颖，王柏村. 数字孪生驱动的智能人机协作：理论、技术与应用［J］. 机械工程学报，2022，58(18)：279-291.
YANG Geng，ZHOU Huiying，WANG Baicun. Digital Twin-driven Smart Human-Machine Collaboration:Theory，Enabling Technologies and Applications［J］. Journal of Mechanical Engineering，2022，58(18)：279-291.
［6］ZHOU Ji，ZHOU Yanhong，WANG Baicun，et al. Human-Cyber-Physical Systems(HCPSs) in the Context of New Generation Intelligent Manufacturing［J］. Engineering，2019，5(4)：624-636.
［7］BOUSDEKIS A，APOSTOLOU D，MENTZAS G. A Human Cyber Physical System Framework for Operator 4.0-Artificial Intelligence Symbiosis［J］. Manufacturing Letters，2020，25：10-5.
［8］王柏村，臧冀原，屈贤明，等. 基于人-信息-物理系统(HCPS)的新一代智能制造研究［J］. 中国工程科学，2018，20(4)：29-34.
WANG Baicun，ZANG Jiyuan，QU Xianming，et al. Research on New-generation Intelligent Manufacturing Based on Human-Cyber-Physical Systems［J］. Engineering Sciences，2018，20(4)：29-34.
［9］KRITZINGER W，KARNER M，TRAAR G，et al. Digital Twin in Manufacturing:a Categorical Literature Review and Classification［J］. IFAC-PapersOnLine，2018，51(11)： 1016-1022.
［10］LIM K Y H，ZHENG P，CHEN C H，et al. A Digital Twin-enhanced System for Engineering Product Family Design and Optimization［J］. Journal of Manufacturing Systems，2020，57：82-93.
［11］WANG Tian，LI Jiakun，KONG Zhaoning，et al. Digital Twin Improved via Visual Question Answering for Vision-language Interactive Mode in Human-Machine Collaboration［J］. Journal of Manufacturing Systems，2020(58)：261-269.
［12］WANG Peng，LIU Hongyi，WANG Lihui，et al. Deep Learning-based Human Motion Recognition for Predictive Context-aware Human-Robot Collaboration［J］. CIRP Annals—Manufacturing Technology，2018, 67：17-20.
［13］LIU Zitong，LIU Quan，XU Wenjun，et al. Deep Learning-based Human Motion Prediction considering Context Awareness for Human-Robot Collaboration in Manufacturing［J］. Procedia CIRP，2019，83：272-278.
［14］李伯平.论工业机器人的技术应用趋势及维护［J］. 现代制造技术与装备，2020(1)：174-175.
LI Boping. Technical Application Trend and Maintenance of Industrial Robots［J］. Manufacturing Technology and Equipment，2020(1)：174-175.
［15］卢光明. 智能制造人机协同制造系统安全问题及其应对措施［J］. 网络空间安全，2018，9(2)：26-33.
LU Guangming. Security Problems and Its Countermeasures for Human-Machine Cooperative Intelligent Manufacturing System［J］. Cyberspace Security，2018，9(2)：26-33.
［16］LENNART B，TIMO B，ANDREAS K，et al. Human-Robot Collaboration in Decentralized Manufacturing Systems:an Approach for Simulation-based Evaluation of Future Intelligent Production［J］. Procedia CIRP，2017(62)：624-629.
［17］PREZ L，RODRGUEZ-JIMNEZ S，RODRGUEZ N，et al. Symbiotic Human-Robot Collaborative Approach for Increased Productivity and Enhanced Safety in the Aerospace Manufacturing Industry［J］. The International Journal of Advanced Manufacturing Technology，2020，106：851-863.
［18］YOUNG M S，BROOKHUIS K A，WICKENS C D，et al. State of Science：Mental Workload in Ergonomics［J］. Ergonomics，2015，58(1)：1-17.
［19］BIONDI N F，ANGELA C，CAITLYN D，et al. Overloaded and at Work：Investigating the Effect of Cognitive Workload on Assembly Task Performance［J］. Human Factors，2020，63(5):813-820.
［20］SHAW T H，SATTERFIELD K，RAMIREZ R，et al. Using Cerebral Hemovelocity to Measure Workload during a Spatialised Auditory Vigilance Task in Novice and Experienced Observers［J］. Ergonomics，2013，56(8)： 1251-1263.
［21］LIU Hao，TAO Jun，LYU Pan，et al. Human-Robot Cooperative Control Based on sEMG for the Upper Limb Exoskeleton Robot［J］. Robotics and Autonomous Systems，2020(125)：103-115.
［22］SARA S，JUNG M，ELIZABETH C. Cooperative Gestures for Industry:Exploring the Efficacy of Robot Hand Configurations in Expression of Instructional Gestures for Human-Robot Interaction［J］.The International Journal of Robotics Research，2017(36)：5-7.
［23］NORMAN D A，DRAPER S W. User Centered System Design:New Perspectives on Human-Computer Interaction［M］. Boca Raton:CRC Press, 1986.
［24］SCHIRNER G，ERDOGMUS D，CHOWDHURY K，et al. The Future of Human-in-the-loop Cyber-Physical Systems［J］. Computer，2013，1：36-45.
［25］王柏村，薛塬，延建林，等. 以人为本的智能制造：理念、技术与应用［J］.中国工程科学，2020，22(4)：139-146.
WANG Baicun，XUE Yuan，YAN Jianlin，et al. Human-centered Intelligent Manufacturing:Overview and Perspectives［J］. Engineering Sciences，2020，22(4)：139-146.
［26］李清，唐骞璘，陈耀棠，等. 智能制造体系架构、参考模型与标准化框架研究［J］. 计算机集成制造系统，2018，24(3)：539-549.
LI Qing，TANG Qianlin，CHEN Yaotang，et al. Smart Manufacturing Standardization:Reference Model and Standards Framework［J］. Computer Integrated Manufacturing Systems，2018，24(3)：539-549.
［27］齐二石，霍艳芳，刘洪伟. 面向智能制造的工业工程和精益管理［J］.中国机械工程，2022，33(21)：2521-2530.
QI Ershi，HUO Yanfang，LIU Hongwei. Industrial Engineering and Lean Management for Smart Manufacturing［J］. China Mechanical Engineering，2022，33(21)：2521-2530.
［28］刘强.智能制造理论体系架构研究［J］.中国机械工程，2020，31(1)：24-36.
LIU Qiang. Study on Architecture of Intelligent Manufacturing Theory［J］. China Mechanical Engineering，2020，31(1)：24-36.
［29］STADNICKA D，LITWIN P，ANTONELLI D. Human Factor in Intelligent Manufacturing Systems-knowledge Acquisition and Motivation［J］. Procedia CIRP，2019(79)：718-723.
［30］ZHUANG Yueting，WU Fei，CHEN Chun，et al. Challenges and Opportunities:from Big Data to Knowledge in AI 2.0［J］. Frontiers of Information Technology & Electronic Engineering，2017，18(1)： 3-14.
［31］ABHISHEK S，ABHINAV S，JITENDRA K，et al. Swarm Intelligence:Foundation，Principles，and Engineering Applications［M］. Boca Raton:CRC Press, 2021.
［32］王柏村，易兵，刘振宇，等. HCPS视角下智能制造的发展与研究［J］. 计算机集成制造系统，2020(10):2749-2761.
WANG Baicun，YI Bing，LIU Zhenyu， et al. Evolution and State-of-the-art of Intelligent Manufacturing from HCPS Perspective［J］. Computer Integrated Manufacturing Systems，2020(10):2749-2761.
［33］LI Shufei，ZHENG Pai，FAN Junming，et al. Toward Proactive Human-Robot Collaborative Assembly:a Multimodal Transfer-learning-enabled Action Prediction Approach［J］. IEEE Transactions on Industrial Electronics，2022，69(8)：8579-8588.
［34］NAHAVANDI S. Industry 5.0—a Human-centric Solution［J］. Sustainability，2019，11：4371.
［35］BORTOLINI M，FACCIO M，GAMBERI M，et al. Motion Analysis System(MAS) for Production and Ergonomics Assessment in the Manufacturing Processes［J］. Computers & Industrial Engineering，2020，139：105485.
［36］BATTINI D，BERTI N，FINCO S，et al. WEM-Platform:a Real-time Platform for Full-body Ergonomic Assessment and Feedback in Manufacturing and Logistics Systems［J］. Computers & Industrial Engineering，2021，164：107881.
［37］FAN J，ZHENG P，LI S. Vision-based Holistic Scene Understanding towards Proactive Human-Robot Collaboration［J］. Robotics and Computer-integrated Manufacturing，2022, 75:102304.
［38］LIN C，WANG K，ABIDE A，et al. Human-Robot Collaboration Empowered by Hidden Semi-Markov Model for Operator Behaviour Prediction in a Smart Assembly System［J］. Journal of Manufacturing Systems，2022，62：317-333.
［39］WANG Z，WANG Y，BAI X，et al. SHARIDEAS:a Smart Collaborative Assembly Platform Based on Augmented Reality Supporting Assembly Intention Recognition［J］. The International Journal of Advanced Manufacturing Technology，2021，115:475-486.
［40］CHADALAVADA R T，ANDREASSON H，SCHINDLER M，et al. Bi-directional Navigation Intent Communication Using Spatial Augmented Reality and Eye-tracking Glasses for Improved Safety in Human-Robot Interaction［J］. Robotics and Computer-integrated Manufacturing，2019，61:101830.
［41］LI S，ZHENG P，FAN J，et al. Toward Proactive Human-Robot Collaborative Assembly:a Multimodal Transfer-learning-enabled Action Prediction Approach［J］. IEEE Transactions on Industrial Electronics，2022，69：8579-8588.
［42］KUMAR N，KUMAR J. Measurement of Cognitive Load in HCI Systems Using EEG Power Spectrum:an Experimental Study［J］. Procedia Computer Science，2016，84：70-78.
［43］孙效华，张义文，秦觉晓，等. 人机智能协同研究综述［J］. 包装工程，2020，41(18)：1-11.
SUN Xiaohua，ZHANG Yiwen，QIN Juexiao，et al. Review on Human-intelligent System Collaboration［J］. Packaging Engineering，2020，41(18):1-11.
［44］刘承平，肖旭，赵竞全. 基于认知过程的飞行员脑力负荷动态预测［J/OL］.北京航空航天大学学报［2023-07-11］. https:∥kns.cnki.net/kcms/detail/11.2625.V.20220310.1456.003.html.
LIU Chengping，XIAO Xu，ZHAO Jingquan. Pilots Mental Workload Dynamic Prediction Based on Cognitive Process［J/OL］.Journal of Beijing University of Aeronautics and Astronautics［2023-07-11］. https:∥kns.cnki.net/kcms/detail/11.2625.V.20220310.1456.003.html.
［45］YANG X N，KIM J H. Assessing Situation Awareness in Multitasking Supervisory Control Using Success Rate of Self-terminating Search［J］. International Journal of Industrial Ergonomics，2019，72：354-362.
［46］王柏村，黄思翰，易兵，等. 面向智能制造的人因工程研究与发展［J］.机械工程学报，2020，56(16)：240-253.
WANG Baicun，HUANG Sihan，YI Bing，et al. State-of-art of Human Factors/Ergonomics in Intelligent Manufacturing［J］. Journal of Mechanical Engineering，2020，56(16)：240-253.
［47］PROCTOR R W，VAN ZANDT T. Human Factors in Simple and Complex Systems［M］. 3rd ed. London:Taylor and Francis, 2018.
［48］ZALLIO M，RAYMUNDO C ，HERNANDEZ J. Advances in Human Factors in Robots, Unmanned Systems and Cybersecurity［M］∥Lecture Notes in Networks and Systems. Berlin:Springer, 2021.
［49］MOHAMMAD B，RASHID H，FAKHRODIN G，et al. Ergonomics 4.0:a Bibliometric Review of Human Factors Research in Industrial Revolution 4.0(IR 4.0)［J］. Work，2021, 70(6)：1-14.
［50］PERUZZINI M，PELLICCIARI M. A Framework to Design a Human-centred Adaptive Manufacturing System for Aging Workers［J］. Advanced Engineering Informatics，2017，33：330-349.
［51］张长信.对现代传统制造系统中人因要素的探讨［J］. 科技创新与应用，2019(5)：47-48.
ZHANG Changxin. Exploration of Human Factors in Modern Traditional Manufacturing Systems［J］. Technological Innovation and Application，2019(5)：47-48.
［52］“新一代人工智能引领下的智能制造研究”课题组.中国智能制造发展战略研究［J］.中国工程科学， 2018，20(4)：9-16.
“Intelligent Manufacturing Investigation Driven by New Generation Artificial Intelligence” Group. Research on the Development Strategy of Intelligent Manufacturing in China［J］. Engineering Sciences，2018，20(4)：9-16.
［53］周济.智能制造是“中国制造2025”主攻方向［J］.企业观察家，2019(11)：54-55.
ZHOU Ji. Intelligent Manufacturing is the Main Direction of "Made in China 2025"［J］. Enterprise Observer，2019(11)：54-55.
［54］侯颖，许威威. 增强现实技术综述［J］. 计算机测量与控制，2017，25(2)：1-7.
HOU Ying，XU Weiwei. Overview of Augmented Reality Technology［J］. Computer Knowledge and Technology，2017，25(2)：1-7.
［55］WANG J，HU Y，YANG X. Multi-person Collaborative Augmented Reality Assembly Process Evaluation System Based on HoloLens［C］∥International Conference on Human-computer Interaction. Springer，2022：369-380.
［56］WANG P，HU Y，YANG X，et al. Augmented Reality-based Rapid Digital Verification of the Body-in-white for Intelligent Manufacturing［C］∥Proceedings of the 6th International Conference on Computer Science and Application Engineering. New York，2022：1-5.
［57］DUFFY V G. Digital Human Modeling and Applications in Health，Safety，Ergonomics and Risk Management［M］. Berlin:Springer，2019.
［58］MAO W，HU Y，YANG X，et al. ARE-platform:an Augmented Reality-based Ergonomic Evaluation Solution for Smart Manufacturing［J］. International Journal of Human-Computer Interaction，2023：10.1080/10447318.2023.2173894.
［59］LI C，YANG X，HU Y，et al. AR-based Accessibility Verification Method for Smart Manufacturing System with Human Motion Capture［C］∥Proceedings of the 6th International Conference on Computer Science and Application Engineering. New York，2022：1-6.
［60］CASSENTI D，SCATAGLINI S，RAJULU S L，et al. Advances in Simulation and Digital Human Modeling［M］. Berlin:Springer，2020.
［61］AHMED N，KHAROUB H，MEDJDEN S，et al. A Natural User Interface for 3D Animation Using Kinect［J］. International Journal of Technology and Human Interaction，2020，16(4)：35-54.
［62］GIBALDI A， VANEGAS M, BEX P J， et al. Evaluation of the Tobii EyeX Eye Tracking Controller and MATLAB Toolkit for Research［J］. Behavior Research Methods，2016，49(3)：923-946.
［63］ZENG Z，SIEBERT F W，VENJAKOB A C，et al. Calibration-free Gaze Interfaces Based on Linear Smooth Pursuit［J］. Journal of Eye Movement Research，2020，13(1):10.16910/jemr.13.1.3.
［64］SCHSSEL F，HONOLD F，WEBER M. Influencing Factors on Multimodal Interaction during Selection Tasks［J］. Journal on Multimodal User Interfaces，2013，7(4)：299-310.
［65］PARMINDER K，SINGH P，KAUR M. Comparative Analysis on Cross-modal Information Retrieval:a Review［J］.Computer Science Review，2021，39:100336.
［66］ROHACZ A，WEIENFELS S，STRASSBURGER S. Concept for the Comparison of Intralogistics Designs with Real Factory Layout Using Augmented Reality, SLAM and Marker-based Tracking［J］. Procedia CIRP，2020，93：341-346.
［67］FANG H，WEN J，YANG X，et al. Assisted Human-Robot Interaction for Industry Application Based Augmented Reality［C］∥24th HCI International Conference. Berlin:Springer，2022：291-301.
［68］徐小萍，吕健，金昱潼，等. 用户认知驱动的 VR 自然交互认知负荷研究［J］.计算机应用研究，2020，37(7)：1958-1963.
XU Xiaoping，LYU Jian，JIN Yutong，et al. Research on Cognitive Load of VR Natural Interaction Driven by User Cognition［J］. Computer Application Research，2020，37(7)：1958-1963.
［69］张奇良，杨坤华，曲行达，等.基于多模态生理信号的驾驶人脑力负荷评估［J］.深圳大学学报(理工版)，2022，39(3)：278-286.ZHANG Qiliang，YANG Kunhua，QU Xingda，et al. Evaluation of Drivers Mental Workload Based on Multi-modal Physiological Signals［J］. Journal of Shenzhen University(Science and Engineering Edition)，2022，39(3)：278-286.
［70］刘文，李琳琳，付光远. 面向指挥控制系统界面的认知负荷研究［J］. 中国电子科学研究院学报，2021，16(2)：203-208.
LIU Wen，LI Linlin，FU Guangyuan. Research on Cognitive Load of C2 System Interfaces［J］. Journal of China Academy of Electronic Science，2021，16(2)：203-208.
［71］缪相林，何绯娟，缪亚林，等.动态调整用户认知负荷的智能增强现实系统［J］. 计算机工程与应用，2019，55(7)：182-187.
MIAO Xianglin，HE Feijuan，MIAO Yalin，et al. Intelligent Augmented Reality System for Dynamically User Cognitive Load Adjustment［J］.Computer Engineering and Application，2019，55(7)：182-187.

[1]	WANG Baicun, ZHU Kailing, XUE Yuan, BAI Jie, ZANG Jiyuan, XIE Haibo, YANG Huayong, . Digital Transformation Mode and Strategy of SMEs in China [J]. China Mechanical Engineering, 2023, 34(14): 1756-1763.
[2]	LI Dongyang, YUAN Dongfeng, ZHANG Haixia, ZHENG Anzhu, DI Zijun, LIANG Daojun, . Research on Intelligent Tool Fault Diagnosis System of Machine Tools with Cloud-Edge-Device Collaboration [J]. China Mechanical Engineering, 2023, 34(05): 584-594.
[3]	WANG Liang1;AN Xingyun2;ZHANG Xueqin2;WANG Yilin1;ZHANG Yisheng1. Research and Application of Variable Speed Cooperative Transfer Control Strategy of Dual Manipulators [J]. China Mechanical Engineering, 2020, 31(16): 1985-1990.
[4]	SONG Tingxin, LI Ke. Data Communication Technology and Applicationins for Intelligent Manufacturing Workshops Based on OPC UA [J]. China Mechanical Engineering, 2020, 31(14): 1693-1699.
[5]	LI Xiaoying, ZHOU Datao. Product Design Requirement Information Visualization Approach for Intelligent Manufacturing Services [J]. China Mechanical Engineering, 2020, 31(07): 871-881.
[6]	GUO Jutao, HONG Haibo, ZHONG Keke, LIU Xiaojia, GUO Yu. Production Management and Control Method of Aerospace Manufacturing Workshops Based on Digital Twin [J]. China Mechanical Engineering, 2020, 31(07): 808-814.
[7]	CHEN Yong, CHEN Yi, PEI Zhi, WANG Cheng. Digital Twin: Recent Development and Future Trend from Bibliometrics Perspective [J]. China Mechanical Engineering, 2020, 31(07): 797-807.
[8]	LI Hao, WANG Haoqi, CHENG Ying, TAO Fei, HAO Bing, WANG Xinchang, JI Yangjian, SONG Wenyan, DU Wenliao, WEN Xiaoyu, GONG Xiaoyun, LI Ke, ZHANG Yingfeng, LUO Guofu, LI Qifeng. Technology and Application of Data-driven Intelligent Services for Complex Products [J]. China Mechanical Engineering, 2020, 31(07): 757-772.
[9]	ZHANG Jianmin, SHAN Xuyi. Application of Intelligent Manufacturing Technology in Hot Rolling Production Line——Baosteel 1580 Hot Rolling Demonstration Production Line [J]. China Mechanical Engineering, 2020, 31(02): 246-251.
[10]	LIU Qiang, . Study on Architecture of Intelligent Manufacturing Theory [J]. China Mechanical Engineering, 2020, 31(01): 24-36.
[11]	LIU Baicheng, . Innovation·Fundamentals·Intelligence—for Strong Manufacturing Industry [J]. China Mechanical Engineering, 2020, 31(01): 13-18.
[12]	LIU Minyang1,2;HUANG Yanqun1;ZHANG Dawei1. Innovative Design of Human Computer Interaction System for CNC Machine Tools Based on Performance Incentive Mechanism [J]. China Mechanical Engineering, 2019, 30(21): 2554-2559.
[13]	ZHANG Jie, WANG Junliang, LYU Youlong, BAO Jinsong. Big Data Driven Intelligent Manufacturing [J]. China Mechanical Engineering, 2019, 30(02): 127-133,158.
[14]	YANG Yixin1，2;YUAN Zhaocai2;PI Zhibo2;XIA Wei3;JIANG Chao1;GUO Konghui1. Construction and Implementation of Intelligent Transparent Automobile Factories [J]. China Mechanical Engineering, 2018, 29(23): 2867-2874.
[15]	ZHU Sheng, ZHOU Chaoji, ZHOU Kebing. Green Additive Remanufacturing Technology [J]. China Mechanical Engineering, 2018, 29(21): 2590-2593,2614.