Joint Optimization of Inspection， Maintenance and Spare Parts Inventory Based on POMDP Model

doi:10.3969/j.issn.1004-132X.2025.08.020

Abstract

Abstract:

In the research direction of joint optimization of maintenance and spare parts inventory， most of the existing research assumed that the system condition monitoring was perfect and ignored the detection errors in practical applications. In order to solve the above problems， a single-component system containing imperfect condition monitoring and fixed inspection and maintenance intervals was studied， and the condition-based maintenance（CBM） and spare parts inventory management problems of the systems were considered. POMDP were used to model the system and derive the system state transfer probabilities. In order to deal with the complex belief state spaces and improve the algorithm solution efficiency， an improved Perseus algorithm was used. In the numerical case section， the effectiveness of the algorithm was verified， and then the optimal policy structure was analyzed to show that the belief states may represent the state information relatively reasonably compared with the observations， and the effectiveness of the spare parts inventory is also demonstrated by comparing the cases without spare parts inventory.

Key words: imperfect condition monitoring, partially observable Markov decision processes（POMDP）, belief state, joint optimization of maintenance and spare parts inventory

摘要：

在维修与备件库存联合优化时，已有的研究大多假设系统状态监测是完美的，忽略了实际应用中的误差。为了解决该问题，以包含不完美状态监测和固定检修周期的单部件系统为研究对象，考虑该系统的视情维修（CBM）与备件库存管理问题，采用部分可观测的马尔可夫决策过程（POMDP）对系统进行建模，并推导系统状态转移概率。为了处理复杂的信念状态空间，提高算法求解效率，采用了一种改进的Perseus算法。在数值案例部分验证了该算法的有效性，并对最优策略结构进行分析，结果表明：信念状态相比于观测值能相对合理地表示状态信息，同时也对比了有无备件库存的情况，证明了备件库存的有效性。

关键词: 不完美状态监测, 部分可观测的马尔可夫决策过程, 信念状态, 维修与备件库存联合优化

CLC Number:

TH17

Kai WANG, Liudong GU, Yifan ZHOU. Joint Optimization of Inspection， Maintenance and Spare Parts Inventory Based on POMDP Model[J]. China Mechanical Engineering, 2025, 36(8): 1853-1863.

汪凯, 顾刘栋, 周一帆. 基于POMDP模型的检修与备件库存联合优化[J]. 中国机械工程, 2025, 36(8): 1853-1863.

Figures/Tables 13

Fig.1 The relationship between inspection and monitoring points

Fig.2 Sequence of events

Fig.3 Schematic diagram of the interaction between agent and environment

Fig.4 Diagram of component failure curves

Fig.5 Algorithm 1——explore to collect belief points

Fig.6 Algorithm 2——compute the set of value functions

Tab.1 System and cost parameters

$α$	$β$	$σ$	$γ$	$F$	$N$	$C O$	$C H$
0.4	1.6	0.3	0.99	5	10	25	5
$C C$	$C E$	$C P$	$C P'$	$T I$	$T M$	$T L$
90	300	30	50	6	1	2

Tab.1 System and cost parameters

$α$	$β$	$σ$	$γ$	$F$	$N$	$C O$	$C H$
0.4	1.6	0.3	0.99	5	10	25	5
$C C$	$C E$	$C P$	$C P'$	$T I$	$T M$	$T L$
90	300	30	50	6	1	2

Tab.2 Comparison of algorithm performance at different scales

状态数	算法	时间/s	$\| V \|$	期望成本
5	原始Perseus算法	65	18	6.18
5	改进的Perseus算法	41	12	6.18
10	原始Perseus算法	2492	38	6.15
10	改进的Perseus算法	856	24	6.15

Tab.2 Comparison of algorithm performance at different scales

状态数	算法	时间/s	$\| V \|$	期望成本
5	原始Perseus算法	65	18	6.18
5	改进的Perseus算法	41	12	6.18
10	原始Perseus算法	2492	38	6.15
10	改进的Perseus算法	856	24	6.15

Tab.3 Costs and strategies under different experiments

实验	长期期望成本	参数化策略
本文实验	6.18	/
对比实验1	6.81	/
对比实验2	8.31	$[1,4, 3,4]$

Tab.3 Costs and strategies under different experiments

实验	长期期望成本	参数化策略
本文实验	6.18	/
对比实验1	6.81	/
对比实验2	8.31	$[1,4, 3,4]$

Fig.7 Ordering and maintenance strategy when spare parts are in stock

Fig.8 Ordering and maintenance strategy when no spare parts are in stock

Fig.9 Analyzing states and actions during simulation

Tab.4 Expected costs under different parameters

参数	$C P'$			$C H$			$T I$			$T L$			$N$
参数	30	40	50	0	5	10	4	6	8	1	2	3	3	5	10
成本	5.92	6.05	6.18	4.23	6.18	6.95	6.05	6.18	6.31	5.57	6.18	6.71	6.95	6.18	6.15

Tab.4 Expected costs under different parameters

参数	$C P'$			$C H$			$T I$			$T L$			$N$
参数	30	40	50	0	5	10	4	6	8	1	2	3	3	5	10
成本	5.92	6.05	6.18	4.23	6.18	6.95	6.05	6.18	6.31	5.57	6.18	6.71	6.95	6.18	6.15

References 23

[1]	徐兆平，郭波. 复杂装备故障预测方法研究综述［J］. 长沙理工大学学报（自然科学版）， 2023， 20（2）： 10-26.
	XU Zhaoping， GUO Bo. A Research Review on Fault Prognostic Techniques for Complex Equipments［J］. Journal of Changsha University of Science & Technology （Natural Science）， 2023， 20（2）： 10-26.
[2]	LI H， ZHOU Y. Joint Optimization of Condition-based Maintenance and Spare Parts Orders for a Two-unit System［C］∥2022 Global Reliability and Prognostics and Health Management. Yantai， 2022： 1-6.
[3]	WANG Jingjing， QIU Qingan， WANG Huanhuan. Joint Optimization of Condition-based and Age-based Replacement Policy and Inventory Policy for a Two-unit Series System［J］. Reliability Engineering & System Safety， 2021， 205： 107251.
[4]	ZHENG Meimei， LIN Jie， XIA Tangbin， et al. Joint Condition-based Maintenance and Spare Provisioning Policy for a K-out-of-N System with Failures during Inspection Intervals［J］. European Journal of Operational Research， 2023， 308（3）： 1220-1232.
[5]	TCHAKOUA P， WAMKEUE R， TAMEGHE T A， et al. A Review of Concepts and Methods for Wind Turbines Condition Monitoring［C］∥2013 World Congress on Computer and Information Technology （WCCIT）. Sousse， Tunisia， 2013： 1-9.
[6]	LI Yanrong， PENG Shizhe， LI Yanting， et al. A Review of Condition-based Maintenance： Its Prognostic and Operational Aspects［J］. Frontiers of Engineering Management， 2020， 7（3）： 323-334.
[7]	张红旗，邵晓东，胡祥涛. 基于部分可观察马尔可夫决策过程的机电装备动态可靠性评价方法［J］. 中国机械工程， 2016， 27（18）： 2482-2486.
	ZHANG Hongqi， SHAO Xiaodong， HU Xiangtao. Dynamic Reliability Assessment Method Based on POMDP for Electromechanical Equipment［J］. China Mechanical Engineering， 2016， 27（18）： 2482-2486.
[8]	刘大玲，黄小钢. 高速铁路无砟轨道系统状态监测及预防性维修［J］. 中国机械工程， 2019， 30（3）： 349-353.
	LIU Daling， HUANG Xiaogang. Condition Monitoring and Preventive Maintenance of Ballastless Track Systems for High-speed Railways［J］. China Mechanical Engineering， 2019， 30（3）： 349-353.
[9]	ROUX M， FANG Y P， BARROS A. Maintenance Planning under Imperfect Monitoring： an Efficient POMDP Model Using Interpolated Value Function［J］. IFAC-PapersOnLine， 2022， 55（16）： 128-135.
[10]	MORATO P G， NIELSEN J S， MAI A Q， et al. POMDP Based Maintenance Optimization of Offshore Wind Substructures Including Monitoring［C］∥13th International Conference on Applications of Statistics and Probability in Civil Engineering. Seoul， 2019：1-8.
[11]	NGUYEN K T， DO P， HUYNH K T， et al. Joint Optimization of Monitoring Quality and Replacement Decisions in Condition-based Maintenance［J］. Reliability Engineering & System Safety， 2019， 189： 177-195.
[12]	ZHAO Fei， LIU Xuejuan， PENG Rui， et al. Joint Optimization of Inspection and Spare Ordering Policy with Multi-level Defect Information［J］. Computers & Industrial Engineering， 2020， 139： 106205.
[13]	张奥. 考虑状态演变过程的高速铁路牵引供电设备维修策略［D］. 成都：西南交通大学， 2017.
	ZHANG Ao. High Speed Railway Traction Power Supply Equipment Maintenance Strategy Considering State Evolution Process［D］. Chengdu： Southwest Jiaotong University， 2017.
[14]	张龙，黄文艺，熊国良，等. 基于多域特征与高斯混合模型的滚动轴承性能退化评估［J］. 中国机械工程， 2014， 25（22）： 3066-3072.
	ZHANG Long， HUANG Wenyi， XIONG Guoliang， et al. Assessment of Rolling Bearing Performance Degradation Using Gauss Mixture Model and Multi-domain Features［J］. China Mechanical Engineering， 2014， 25（22）： 3066-3072.
[15]	van NOORTWIJK J M. A Survey of the Application of Gamma Processes in Maintenance［J］. Reliability Engineering & System Safety， 2009， 94（1）： 2-21.
[16]	NEWBY M J， BARKER C T. A Bivariate Process Model for Maintenance and Inspection Planning［J］. International Journal of Pressure Vessels and Piping， 2006， 83（4）： 270-275.
[17]	SUTTON R S， BARTO A G. Reinforcement Learning： an Introduction［M］. Cambridge： The MIT Press， 2018.
[18]	BRODY D C， HUGHSTON L P， MACRINA A. Dam Rain and Cumulative Gain［J］. Proceedings of the Royal Society A： Mathematical， Physical and Engineering Sciences， 2008， 464（2095）： 1801-1822.
[19]	PARK C， PADGETT W J. Accelerated Degradation Models for Failure Based on Geometric Brownian Motion and Gamma Processes［J］. Lifetime Data Analysis， 2005， 11（4）： 511-527.
[20]	SHANI G， PINEAU J， KAPLOW R. A Survey of Point-based POMDP Solvers［J］. Autonomous Agents and Multi-Agent Systems， 2013， 27（1）： 1-51.
[21]	VIRIN Y， SHANI G， SHIMONY S E， et al. Scaling Up： Solving POMDPs through Value Based Clustering［C］∥AAAI Conference on Artificial Intelligence. Vancouver， 2007：1290-1295.
[22]	YANG Li， YE Zhisheng， LEE C G， et al. A Two-phase Preventive Maintenance Policy Considering Imperfect Repair and Postponed Replacement［J］. European Journal of Operational Research， 2019， 274（3）： 966-977.
[23]	周一帆，郭凯，李帮诚. 基于多智能体强化学习的多部件系统维修优化［J］. 长沙理工大学学报（自然科学版）， 2023， 20（2）： 27-34.
	ZHOU Yifan， GUO Kai， LI Bangcheng. Maintenance Optimization of Multi-component System Based on Multi-agent Reinforcement Learning［J］. Journal of Changsha University of Science & Technology （Natural Science）， 2023， 20（2）： 27-34.