计算机工程2025，Vol.51Issue(6)：136-145,10.DOI:10.19678/j.issn.1000-3428.0069109
基于分层强化学习的在线三维装箱模型
Online 3D Bin Packing Model Based on Hierarchical Reinforcement Learning
摘要
Abstract
Previous studies have shown increasing interest in understanding how artificial intelligence represents perception and action planning in a hierarchical manner across multiple abstraction levels and timescales.Owing to technological constraints,most studies have been limited to the artificial decomposition of tasks,such as the 3D Bin Packing Problem(3DBPP).In this scenario,heuristic rules guide neural networks in the analysis of the packing points during the task decomposition stage,thus helping the agent decompose the state space.This transforms the originally vast and complex space into individual subspaces,thereby providing the neural network with better alternative solutions.However,these rules cause performance limitations.If the rules cannot perfectly decompose the problem,fixed-rule assistance may restrict the performance of the neural network by overlooking better solutions that the rules may ignore.To address this problem,a heuristic rule fusion strategy is used in this study to improve the original Packing Configuration Tree(PCT)model.This strategy is based on the concept of hierarchical reinforcement learning to layer the problem,in which a graph attention classification model is introduced to determine the optimal spatial point expansion scheme for the current situation.This approach ensures more possibilities for the combination and arrangement of dismantling internal space points and exploring feasible positions.The results of experiments show that the improved model based on heuristic fusion strategy for layered problems performs better than the original model on multiple datasets.In datasets containing additional density information,the average packing utilization rate reaches 77.2％,which is a 1.7 percentage point improvement over the original model.The proposed model provides more optimal solutions within a reasonable amount of time.关键词
分层强化学习/三维装箱/图注意力网络/启发式空间拓展/深度强化学习Key words
hierarchical reinforcement learning/3D bin packing/Graph Attention Network(GAT)/heuristic space expansion/deep reinforcement learning分类
信息技术与安全科学引用本文复制引用
亓明凯,王迪,张立晔..基于分层强化学习的在线三维装箱模型[J].计算机工程,2025,51(6):136-145,10.基金项目
山东省自然科学基金(ZR2023MF015). （ZR2023MF015）
