交通信息与安全2023，Vol.41Issue(6)：132-141,10.DOI:10.3963/j.jssn.1674-4861.2023.06.015
基于DQN算法的支线集装箱船航线规划与配载协同优化方法
Coordinated Optimization Method for Feeder Container Ship Route Planning and Stowage Based on DQN Algorithm
摘要
Abstract
Given the unique features of feeder container shipping,including varying feeder port numbers and incon-sistent berthing conditions,as well as the divers'types of container fleets,this research investigates the coordinated optimization for route planning and stowage in feeder container shipping considering their close connection in the actual transportation process.A two-stage hierarchical method is employed to study the route planning and container stowage problems.Multiple ports,different ship types with their respective bays and stack combinations,and con-tainers of various sizes are included in the study.The fundamental relationships among these elements are estab-lished to achieve integrity and continuity of the two-stage optimization process.The first stage involves establishing a ship route planning model with the objective of minimizing the total operational cost.The second stage focuses on optimizing the stowage from the perspective of primary bay planning.The correspondence between containers and stacks is determined,and a ship stowage model is developed with the objective of minimizing the number of mixed container stacks.The stowage model ensures that the ship's stability meets the requirements throughout the route,while reducing the number of mixed stacks to improve port operation efficiency.To efficiently solve the proposed models,a Markov process corresponding to route planning and stowage decision-making is designed based on the Deep Q-learning Network(DQN)algorithm from deep reinforcement learning.The intelligent agent's state space,action space,and reward function are designed based on the problem's characteristics to construct the two-stage hi-erarchical DQN algorithm.Experimental results demonstrate that as the number of ships and the ship loading rate in-crease,the time required for accurate model solution significantly rises.Some cases cannot be solved within 600 sec-onds,while the DQN algorithm achieves rapid solutions in all examples.Compared with traditional models and the Particle Swarm Optimization(PSO)algorithm,the DQN algorithm efficiently solves cases of different scales.The maximum solving time for large-scale cases is 31.40 s,with an average time of less than 30 s,indicating good solu-tion efficiency.Further calculations indicate that under different feeder port numbers,the average standard deviation of solving time for the DQN algorithm is only 1.74,showing better robustness compared to the average standard de-viation of 11.20 for the PSO algorithm.Overall,the DQN algorithm exhibits less fluctuation in solving time with changing problem scales,showcasing stable solving performance and efficient optimization capabilities.关键词
支线集装箱船运输/航线规划/集装箱配载/深度强化学习/DQN算法Key words
feeder container shipping/route planning/container stowage/deep reinforcement learning/DQN algo-rithm分类
交通工程引用本文复制引用
李俊,肖笛,温想,赵雅洁..基于DQN算法的支线集装箱船航线规划与配载协同优化方法[J].交通信息与安全,2023,41(6):132-141,10.基金项目
湖北省自然科学基金项目(2023AFB071)资助 （2023AFB071）
