Transit Time-Based Graph Convolutional Network Integrated with a Q-Learning Approach for Solving the Vehicle Routing Problem to Minimize Carbon Dioxide Emissions in the Transportation of Building Components: Transit Time-Based Graph Convolutional Network Integrated with a Q-Learning Approach

Abstract

During the construction process, the timely transportation and efficient allocation of prefabricated components are critical factors influencing project progress, construction quality, and cost management. This study aims to optimize the logistics transit of components in building construction by employing reinforcement learning algorithms, with the objective of improving transportation efficiency and minimizing transportation costs. We address the existing limitations of reinforcement learning in solving vehicle routing problems (VRP), specifically suboptimal solution quality and limited generalization capabilities. To overcome these challenges, we propose an optimization approach based on transit time, which integrates Graph Convolutional Networks (GCN) with Q-learning and incorporates a road traffic congestion index within the three-dimensional coordinate solution space. By combining graph neural networks with reinforcement learning, the method enhances the decision-making effectiveness and adaptability of the reinforcement learning model within complex graph environments, enhances the convergence efficiency and solution quality of Q-learning during path search, and ultimately achieves routes with lower transportation costs. The integration of Graph Convolutional Networks (GCN) with Q-learning facilitates a more rapid convergence during the learning phase. This improvement is attributed to the generation of more rational state vectors and more precise Q-value updates, enabling the agent to more effectively differentiate and select optimal actions, thereby enhancing the global optimization capability of path selection. Notably, under conditions of road congestion, the proposed Transit Time-based GCN integrated Q-learning approach significantly outperforms the original GCN+Q-learning approach, achieving reductions in fuel consumption and gas emissions by 21.7%, 22.06%, and 12.78% in scenarios involving 11, 21, and 31 stops, respectively.