Abstract:

To address the issues of complex physical modeling， poor adaptability， and insufficient algorithm generalization in the current selective disassembly sequence planning problem， a structured heterogeneous graph modeling method was proposed， which combined with an adaptive proximal policy optimization algorithm to achieve efficient disassembly sequence optimization. Through the structured heterogeneous graph modeling， the multi-constraint relationships of the product components were unified， providing a more expressive state representation for subsequent optimization. Additionally， in the optimization algorithm， advantage function normalization and entropy regularization mechanism were introduced to standardize the data distribution inconsistency caused by dimensional differences across different training stages， while adaptively adjusting the exploration intensity during the training processes to enhance the model's training stability and generalization ability. Experimental results show that the introduction to advantage function normalization significantly improves the algorithm's convergence speed and training stability， while the entropy regularization mechanism enhances the algorithm's exploration ability. Compared with traditional deep reinforcement learning algorithms， the proposed method performes better in terms of convergence and the quality of the optimal policy.

Key words: retired product, disassembly planning, deep reinforcement learning, heterogeneous graph

摘要：

针对当前选择性拆卸序列规划问题中存在物理建模复杂、适应性差以及算法泛化能力不足等问题，结合结构化异构图建模方法和自适应近端策略优化算法，提出了一种高效的拆卸序列优化方法。通过结构化异构图建模，统一表示产品中零部件的多约束关系，为后续优化提供更具表达力的状态表示；在优化算法中引入优势函数标准化与熵正则化机制，对不同训练阶段数据因量纲差异所带来的分布不一致进行规范化调整，同时自适应调节训练过程中的探索强度，以提高模型的训练稳定性和泛化能力。实验结果表明，引入优势函数标准化显著提高了算法的收敛速度和训练稳定性，而熵正则化机制则增强了算法的探索能力。与传统深度强化学习算法相比，自适应近端策略优化算法在收敛性和最优策略质量方面均表现更好。

关键词: 退役产品, 拆卸规划, 深度强化学习, 异构图