考虑峰值功率受限约束的柔性作业车间调度研究

doi:10.3969/j.issn.1004-132X.2025.02.011

摘要/Abstract

摘要： 针对车间峰值功率受限约束下的柔性作业车间调度面临的作业周期增加、机器负荷增大的问题，建立以最小化最大完工时间和最小化机器最大负载为优化目标、考虑车间峰值功率约束的柔性作业车间调度问题（PPCFJSP）模型。为更好地调度决策，首先将该问题转化为马尔可夫决策过程，基于此设计了一个结合离线训练与在线调度的用于求解PPCFJSP的调度框架。然后设计了一种基于优先级经验重放的双重决斗深度Q网络(D3QNPER）算法，并设计了一种引入噪声的ε-贪婪递减策略，提高了算法收敛速度，进一步提高了求解能力和求解结果的稳定性。最后开展实验与算法对比研究，验证了模型和算法的有效性。

关键词: 柔性作业车间调度, 马尔可夫决策过程, 深度强化学习, 峰值功率受限

Abstract: Peak power constrained flexible job shop scheduling problem(PPCFJSP) model was established to address the challenges of increased work cycles and increased machine load in flexible job shop scheduling under the constraints of peak power in the workshops. The optimization objectives were to minimize the maximum completion time and the maximum machine loads, taking into account the constraints of peak power in the workshops. For better scheduling decisions, firstly, the problem was transformed into a Markov decision process, then, a scheduling framework combining offline training and online scheduling was designed for solving PPCFJSP. Secondly, a double dueling deep q-network based on priority experience replay(D3QNPER) algorithm was designed based on priority experience replay, and a ε- greedy descent strategy introducing noise was designed to improve the convergence speed of the algorithm, further enhance the solving ability and stability of the solution results. Finally, experimental and algorithmic comparative studies were conducted to verify the effectiveness of the model and algorithm.

Key words: flexible job shop scheduling, Markov decision process, deep reinforcement learning, peak power constrained

中图分类号:

TH165
TP18

李益兵1, 2, 曹岩1, 郭钧1, 2, 王磊1, 2, 李西兴3, 孙利波4. 考虑峰值功率受限约束的柔性作业车间调度研究[J]. 中国机械工程, 2025, 36(02): 280-293.

LI Yibing1, 2, CAO Yan1, GUO Jun1, 2, WANG Lei1, 2, LI Xixing3, SUN Libo4. Research on Flexible Job-shop Scheduling Considering Constraints of Peak Power Constrained[J]. China Mechanical Engineering, 2025, 36(02): 280-293.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.cmemo.org.cn/CN/10.3969/j.issn.1004-132X.2025.02.011

https://www.cmemo.org.cn/CN/Y2025/V36/I02/280

参考文献

［1］李凯文, 张涛, 王锐, 等. 基于深度强化学习的组合优化研究进展［J］. 自动化学报, 2021, 47(11):2521-2537.
LI Kaiwen, ZHANG Tao, WANG Rui, et al. Research Reviews of Combinatorial Optimization Methods Based on Deep Reinforcement Learning［J］. Acta Automatica Sinica, 2021, 47(11):2521-2537.
［2］李颖俐, 李新宇, 高亮. 混合流水车间调度问题研究综述［J］. 中国机械工程, 2020, 31(23):2798-2813.
LI Yingli, LI Xinyu, GAO Liang. Review on Hybrid Flow Shop Scheduling Problems［J］. China Mechanical Engineering, 2020, 31(23):2798-2813.
［3］黎声益, 马玉敏, 刘鹃. 基于双深度Q学习网络的面向设备负荷稳定的智能车间调度方法［J］. 计算机集成制造系统, 2023, 29(1):91-99.
LI Shengyi, MA Yumin, LIU Juan. Smart Shop Floor Scheduling Method for Equipment Load Stabilization Based on Double Deep Q-learning Network［J］. Computer Integrated Manufacturing Systems, 2023, 29(1):91-99.
［4］贺俊杰, 张洁, 张朋, 等. 基于长短期记忆近端策略优化强化学习的等效并行机在线调度方法［J］. 中国机械工程, 2022, 33(3):329-338.
HE Junjie, ZHANG Jie, ZHANG Peng, et al. Related Parallel Machine Online Scheduling Method Based on LSTM-PPO Reinforcement Learning［J］. China Mechanical Engineering, 2022, 33(3):329-338.
［5］LIU Renke, PIPLANI R, TORO C. Deep Reinforcement Learning for Dynamic Scheduling of a Flexible Job Shop［J］. International Journal of Production Research, 2022, 60(13):4049-4069.
［6］LI Yuxin, GU Wenbin, YUAN Minghai, et al. Real-time Data-driven Dynamic Scheduling for Flexible Job Shop with Insufficient Transportation Resources Using Hybrid Deep Q Network［J］. Robotics and Computer-Integrated Manufacturing, 2022, 74:102283.
［7］WU Wenbo, HUANG Zhengdong, ZENG Jiani, et al. A Fast Decision-making Method for Process Planning with Dynamic Machining Resources via Deep Reinforcement Learning［J］. Journal of Manufacturing Systems, 2021, 58:392-411.
［8］LEE Y H, LEE S. Deep Reinforcement Learning Based Scheduling within Production Plan in Semiconductor Fabrication［J］. Expert Systems with Applications, 2022, 191:116222.
［9］HE Zhenglei, TRAN K P, THOMASSEY S, et al. Multi-objective Optimization of the Textile Manufacturing Process Using Deep-Q-network Based Multi-agent Reinforcement Learning［J］. Journal of Manufacturing Systems, 2022, 62:939-949.
［10］郭具涛, 吕佑龙, 戴铮, 等. 基于复合规则和强化学习的混流装配线调度方法［J］. 中国机械工程, 2023, 34(21):2600-2606.
GUO Jutao, LYU Youlong, DAI Zheng, et al. Compound Rules and Reinforcement Learning Based Scheduling Method for Mixed Model Assembly Lines［J］. China Mechanical Engineering, 2023, 34(21):2600-2606.
［11］刘亚辉, 申兴旺, 顾星海, 等. 面向柔性作业车间动态调度的双系统强化学习方法［J］. 上海交通大学学报, 2022, 56(9):1262-1275.
LIU Yahui, SHEN Xingwang, GU Xinghai, et al. A Dual-system Reinforcement Learning Method for Flexible Job Shop Dynamic Scheduling［J］. Journal of Shanghai Jiao Tong University, 2022, 56(9):1262-1275.
［12］ZHANG Jiadong, HE Zhixiang, CHAN W H, et al. DeepMAG:Deep Reinforcement Learning with Multi-agent Graphs for Flexible Job Shop Scheduling［J］. Knowledge-Based Systems, 2023, 259:110083.
［13］GUI Yong, TANG Dunbing, ZHU Haihua, et al. Dynamic Scheduling for Flexible Job Shop Using a Deep Reinforcement Learning Approach［J］. Computers & Industrial Engineering, 2023, 180:109255.
［14］ZHANG Lu, FENG Yi, XIAO Qinge, et al. Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling Problem Considering Variable Processing Times［J］. Journal of Manufacturing Systems, 2023, 71:257-273.
［15］何彦, 王乐祥, 李育锋, 等. 一种面向机械车间柔性工艺路线的加工任务节能调度方法［J］. 机械工程学报, 2016, 52(19):168-179.
HE Yan, WANG Lexiang, LI Yufeng, et al. A Scheduling Method for Reducing Energy Consumption of Machining Job Shops Considering the Flexible Process Plan［J］. Journal of Mechanical Engineering, 2016, 52(19):168-179.
［16］DU Yu, LI Junqing, LI Chengdong, et al. A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem with Crane Transportation and Setup Times［J］. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(4):5695-5709.
［17］NAIMI R, NOUIRI M, CARDIN O. A Q-learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives［J］. Sustainability, 2021, 13(23):13016.
［18］LI Rui, GONG Wenyin, LU Chao, et al. A Learning-based Memetic Algorithm for Energy-efficient Flexible Job-shop Scheduling with Type-2 Fuzzy Processing Time［J］. IEEE Transactions on Evolutionary Computation, 2023, 27(3):610-620.
［19］张凯, 毕利, 焦小刚. 集成强化学习算法的柔性作业车间调度问题研究［J］. 中国机械工程, 2023, 34(2):201-207.
ZHANG Kai, BI Li, JIAO Xiaogang. Research on Flexible Job-shop Scheduling Problems with Integrated Reinforcement Learning Algorithm［J］. China Mechanical Engineering, 2023, 34(2):201-207.
［20］陈睿奇, 黎雯馨, 王传洋, 等. 基于深度强化学习的工序交互式智能体Job shop调度方法［J］. 机械工程学报, 2023, 59(12):78-88.
CHEN Ruiqi, LI Wenxin, WANG Chuanyang, et al. Interactive Operation Agent Scheduling Method for Job Shop Based on Deep Reinforcement Learning［J］. Journal of Mechanical Engineering, 2023, 59(12):78-88.

[1]	胡楷雄1, 宋远航1, 周勇1, 李卫东2. 基于深度强化学习的混杂场景目标物体推抓协同策略[J]. 中国机械工程, 2025, 36(01): 133-140.
[2]	祝正宇1, 郭具涛2, 吕佑龙3, 左丽玲1, 张洁3. 面向柔性作业车间生产调度的深度强化学习方法[J]. 中国机械工程, 2024, 35(11): 2007-2014，2034.
[3]	魏书鹏, 唐红涛, 李西兴, 杨冠宇, 张健. 考虑双资源约束的柔性机械加工车间逆调度问题研究[J]. 中国机械工程, 2024, 35(03): 457-471.
[4]	郭具涛, 吕佑龙, 戴铮, 张洁, 郭宇. 基于复合规则和强化学习的混流装配线调度方法[J]. 中国机械工程, 2023, 34(21): 2600-2606,2614.
[5]	张凯, 毕利, 焦小刚. 集成强化学习算法的柔性作业车间调度问题研究[J]. 中国机械工程, 2023, 34(02): 201-207.
[6]	孙爱红, 宋豫川, 杨云帆, 雷琦. 考虑关键件加工质量的双资源约束车间调度算法[J]. 中国机械工程, 2022, 33(21): 2590-2600.
[7]	李西兴, 杨道明, 李鑫, 吴锐, . 基于混合遗传鲸鱼优化算法的柔性作业车间自动导引车融合调度方法[J]. 中国机械工程, 2021, 32(08): 938-950，986.
[8]	李益兵1,2;黄炜星1;吴锐1. 基于改进人工蜂群算法的多目标绿色柔性作业车间调度研究[J]. 中国机械工程, 2020, 31(11): 1344-1350,1385.
[9]	杨宇;黄敏;王震宇;朱启兵. 利用有限制稳定配对策略求解双目标柔性作业车间调度问题[J]. 中国机械工程, 2018, 29(14): 1743-1750.
[10]	朱传军 , 宋文家, 张超勇, 曹静 , 朱孟周. 基于维修时间窗的柔性作业车间调度优化研究[J]. 中国机械工程, 2016, 27(10): 1337-1343.
[11]	赵敏, 殷欢, 孙棣华, 郑林江, 何伟, 袁川. 基于改进人工鱼群算法的柔性作业车间调度[J]. 中国机械工程, 2016, 27(08): 1059-1065.
[12]	彭建刚, 刘明周, 张玺, 张铭鑫, 葛茂根. 基于Pareto优化的离散自由搜索算法求解多目标柔性作业车间调度问题[J]. 中国机械工程, 2015, 26(5): 620-626.
[13]	巴黎, 李言, 曹源, 杨明顺, 刘永. 考虑批量装配的柔性作业车间调度问题研究[J]. 中国机械工程, 2015, 26(23): 3200-3207.
[14]	刘琼, 田有全, John W. Sutherland, 周迎冬. 产品制造过程碳足迹核算及其优化问题[J]. 中国机械工程, 2015, 26(17): 2336-2343.
[15]	宋文家, 张超勇, 尹勇, 邵新宇. 基于多目标混合殖民竞争算法的设备维护与车间调度集成优化[J]. 中国机械工程, 2015, 26(11): 1478-1487.