基于复合规则和强化学习的混流装配线调度方法

doi:10.3969/j.issn.1004-132X.2023.21.009

中国机械工程 ›› 2023, Vol. 34 ›› Issue (21): 2600-2606,2614.DOI: 10.3969/j.issn.1004-132X.2023.21.009

基于复合规则和强化学习的混流装配线调度方法

郭具涛1,2;吕佑龙3;戴铮1;张洁3;郭宇2

1.上海航天精密机械研究所，上海，201600
2.南京航空航天大学机电学院，南京，210016
3.东华大学人工智能研究院，上海，201620

出版日期:2023-11-10 发布日期:2023-11-29
作者简介:郭具涛，男，1988年生，高级工程师、博士研究生。研究方向为数字化集成制造。E-mail:guojutao800@163.com。
基金资助:
装发快速支持项目（JZX7Y20220163200201）；科技创新行动计划启明星项目（ZZQB14042000）

Compound Rules and Reinforcement Learning Based Scheduling Method for Mixed Model Assembly Lines

GUO Jutao1,2;LYU Youlong3;DAI Zheng1;ZHANG Jie3;GUO Yu2

1.Shanghai Spaceflight Precision Machinery Institute,Shanghai，201600
2.School of Mechanical and Electrical Engineering,Nanjing University of Aeronautics and
Astronautics,Nanjing，210016
3.Institute of Artificial Intelligence,Donghua University，Shanghai，201620

Online:2023-11-10 Published:2023-11-29

摘要/Abstract

摘要： 针对混流装配线的平衡与排序问题，提出了一种基于复合规则和强化学习的智能调度方法。根据数学模型，设计了平衡规则库与排序规则库，提出了规则加权组合的近端策略优化（PPO）算法，并利用具有Actor-Critic训练流程和优先经验回放机制的强化学习过程，实现了复合规则权值参数的调控优化，生成了平衡与排序方案。所提方法与PPO+单一规则算法、复合规则和遗传算法的对比实验验证了所提方法的有效性。

关键词: 混流装配线, 平衡与排序, 深度强化学习, 复合规则, 集成优化

Abstract: A scheduling method was proposed based on compound rules and reinforcement learning for balancing and sequencing problems of mixed model assembly lines. A balancing rule set and a sequencing rule set were designed with the consideration of mathematical model, and a proximal policy optimization(PPO) algorithm featured with Actor-Critic training procedure and preferential experience learning mechanism was employed to regulate weighted parameters of these rules, in order to generate reasonable balancing and sequencing solutions. In comparative experiments, the proposed scheduling method demonstrates the effectiveness over other methods including PPO algorithm with single rule, compound rules, and a genetic algorithm.

Key words: mixed model assembly line, balancing and sequencing, deep reinforcement learning, compound rule, integrated optimization

中图分类号:

郭具涛, 吕佑龙, 戴铮, 张洁, 郭宇. 基于复合规则和强化学习的混流装配线调度方法[J]. 中国机械工程, 2023, 34(21): 2600-2606,2614.

GUO Jutao, LYU Youlong, DAI Zheng, ZHANG Jie, GUO Yu. Compound Rules and Reinforcement Learning Based Scheduling Method for Mixed Model Assembly Lines[J]. China Mechanical Engineering, 2023, 34(21): 2600-2606,2614.

参考文献

［1］肖静. 波音描绘航空航天业可持续发展未来［J］.中国民用航空，2021(10):58-59.
XIAO Jing. Boeing Depicts the Sustainable Development Future of the Aerospace Industry［J］. China Civil Aviation，2021(10)：58-59.
［2］BOCK S, BOYSEN N. Integrated Real-time Control of Mixed-model Assembly Lines and Their Part Feeding Processes［J］.Computers & Operations Research, 2021, 132：105344.
［3］李颖俐，李新宇，高亮. 混合流水车间调度问题研究综述［J］. 中国机械工程，2020，31(23):2798-2813.
LI Yingli, LI Xinyu, GAO Liang. Review on Hybrid Flow Shop Scheduling Problems［J］. China Mechanical Engineering, 2020，31(23)：2798-2813.
［4］GIORGIO A, MAFFEI A, ONORI M, et al. Towards Online Reinforced Learning of Assembly Sequence Planning with Interactive Guidance Systems for Industry 4.0 Adaptive Manufacturing［J］. Journal of Manufacturing Systems, 2021, 60：22-34.
［5］RAUF M, GUAN Z, SARFRAZ S, et al. A Smart Algorithm for Multi-criteria Optimization of Model Sequencing Problem in Assembly Lines［J］. Robotics and Computer-Integrated Manufacturing, 2020, 61：101844.
［6］AKYOL S D, BAYKASOGˇLU A. A Multiple-rule Based Constructive Randomized Search Algorithm for Solving Assembly Line Worker Assignment and Balancing Problem［J］. Journal of Intelligent Manufacturing, 2019, 30(2)：557-573.
［7］DEFERSHA F M, MOHEBALIZADEHGASHTI F. Simultaneous Balancing, Sequencing, and Workstation Planning for a Mixed Model Manual Assembly Line Using Hybrid Genetic Algorithm［J］. Computers & Industrial Engineering，2018，119：370-387.
［8］ZHANG B, XU L, ZHANG J. A Multi-objective Cellular Genetic Algorithm for Energy-oriented Balancing and Sequencing Problem of Mixed-model Assembly Line［J］. Journal of Cleaner Production, 2019, 244:118845.
［9］贺俊杰，张洁，张朋，等. 基于长短期记忆近端策略优化强化学习的等效并行机在线调度方法［J］. 中国机械工程，2022，33(3)：329-338.
HE Junjie, ZHANG Jie, ZHANG Peng, et al. Related Parallel Machine Online Scheduling Method Based on LSTM-PPO Reinforcement Learning［J］. China Mechanical Engineering, 2022, 33(3)：329-338.
［10］WANG H, SARKER B R, LI J, et al. Adaptive Scheduling for Assembly Job Shop with Uncertain Assembly Times Based on Dual Q-learning［J］. International Journal of Production Research, 2021, 59(19)：5867-5883.
［11］TORTORELLI A, IMRAN M, DELLI PRISCOLI F, et al. A Parallel Deep Reinforcement Learning Framework for Controlling Industrial Assembly Lines［J］. Electronics, 2022, 11(4)：539-548.
［12］OSTERMEIER F F. On the Trade-offs between Scheduling Objectives for Unpaced Mixed-model Assembly Lines［J］. International Journal of Production Research, 2022, 60(3)：866-893.
［13］LV Y, ZHANG J. A Genetic Regulatory Network Based Method for Multi-objective Sequencing Problem in Mixed-model Assembly Lines［J］. Mathematical Biosciences and Engineering, 2019, 16(3)：1228-1243.
［14］张凯，毕利，焦小刚. 集成强化学习算法的柔性作业车间调度问题研究［J］.中国机械工程，2023，34(2)：201-207.
ZHANG Kai，BI Li，JIAO Xiaogang. Research on Flexible Job-shop Scheduling Problems with Integrated Reinforcement Learning Algorithm［J］. China Mechanical Engineering，2023，34(2)：201-207.
［15］于鑫，吴文峻，罗杰，等. 面向群体共识机制的逆强化学习辨识方法［J］.中国科学:技术科学，2023，53(2)：258-267.
YU Xin, WU Wenjun, LUO Jie，et al. Identification Method for Collective Consensus Mechanism Based on Inverse Reinforcement Learning［J］. Sci. Sin. Tech., 2023, 53(2):258-267.
［16］PARK J, CHUN J, KIM S H, et al. Learning to Schedule Job-shop Problems:Representation and Policy Learning Using Graph Neural Network and Reinforcement Learning［J］. International Journal of Production Research，2021，59(11)：3360-3377.
［17］BAUTISTA J, CANO A. Solving Mixed Model Sequencing Problem in Assembly Lines with Serial Workstations with Work Overload Minimisation and Interruption Rules［J］. European Journal of Operational Research，2011，210(3)：495-513.
［18］周亚勤，杨长祺，吕佑龙，等. 双资源约束的航天结构件车间生产调度方法［J］. 机械工程学报，2018，54(9)：55-63.
ZHOU Yaqin, YANG Changqi, LYU Youlong, et al. Scheduling the Production of Aerospace Structural Parts with Dual Resource Constraints［J］. Journal of Mechanical Engineering, 2018, 54(9)：55-63.

[1]	何李, 陶翼飞, 罗俊斌, 荀洪凯. 基于两阶段狼群算法的自动化立体仓库作业集成优化[J]. 中国机械工程, 2022, 33(21): 2538-2546.
[2]	张恒;李爱平;傅翔;邵焕;刘雪梅. 面向混批加工的装夹方案选择与线平衡集成优化方法[J]. 中国机械工程, 2019, 30(20): 2497-2504.
[3]	王献红, 史国权. 一种基于单元化制造及时间任务总线的计划调度模式构建[J]. 中国机械工程, 2015, 26(5): 637-640.
[4]	李苏珂, 刘盛强, 喻道远, 张三强, 史登松, 何世林. 面向整车混流装配线客户订单的优化分配排序 [J]. 中国机械工程, 2010, 21(20): 2459-2462.
[5]	蒋艳, 黎向锋, 左敦稳, 焦光明, 薛善良. 基于改进遗传算法的混流装配线的优化设计 [J]. 中国机械工程, 2010, 21(19): 2322-2328.
[6]	苏平;于兆勤;. 混流装配线平衡问题的多目标优化方法研究[J]. J4, 2009, 20(19): 0-2274.

基于复合规则和强化学习的混流装配线调度方法

Compound Rules and Reinforcement Learning Based Scheduling Method for Mixed Model Assembly Lines

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics