Crane scheduling method in steelmaking workshop based on deep reinforcement learning
LIN Shi-jing1, XU An-jun1, LIU Cheng2, FENG Kai1, LI Ji1
1. School of Metallurgical and Ecological Engineering,University of Science and Technology Beijing, Beijing 100083, China; 2. Science and Technology Information Research Institute, Masteel Technology Center, Ma′anshan 243000, Anhui, China
Abstract:Aiming at the dynamic uncertainty caused by overhead crane operation in steelmaking workshop, a crane scheduling method in steelmaking workshop based on deep reinforcement learning algorithm is proposed.Firstly, based on reinforcement learning, the crane scheduling problem is transformed into solving the sequence of crane operation, and DQN (Deep Q-network) algorithm is used to build the action value network model for solving.Then, taking a steel plant as the research object, taking the shortest time to complete the task as the goal, the specific design of the crane scheduling method based on deep reinforcement learning is introduced.Finally, the actual data is used to train the crane action value network model, and the method proposed in this paper is compared with the current widely used crane scheduling method based on fixed partition by simulation experiments. The results show that the cranescheduling method based on deep reinforcement learning reduces the total task completion time by 11.52%, improves the completion efficiency of the crane task, and proves the feasibility of the method. It provides a new idea for the research of the crane scheduling.
林时敬, 徐安军, 刘成, 冯凯, 李稷. 基于深度强化学习的炼钢车间天车调度方法[J]. 中国冶金, 2021, 31(3): 37-43.
LIN Shi-jing, XU An-jun, LIU Cheng, FENG Kai, LI Ji. Crane scheduling method in steelmaking workshop based on deep reinforcement learning[J]. China Metallurgy, 2021, 31(3): 37-43.
LI J,XU A J,ZANG X S.Simulation-based solution for a dynamic multi-crane-scheduling problem in a steelmaking shop[J].International Journal of Production Research,2019(9):1.
Mnih V,Kavukcuoglu K,Silver D,et al.Playing atari with deep reinforcement learning[DB/OL].(2013-12-19) [2020-02-13].https://arxiv.org/pdf/1312.5602.pdf.
[14]
MENG H,CHAO D,GUO Q,et al.Delay-sensitive task scheduling with deep reinforcement learning in mobile-edge computing systems[J].Journal of Physics: Conference Series,2019,1229(1):484.
[15]
WANG H J,YANG Z,ZHOU W G,et al.Online scheduling of image satellites based on neural networks and deep reinforcement learning[J].Chinese Journal of Aeronautics,2019,32(4):1011.
[16]
Sutton R S,Barto A G.Reinforcement learning: An introduction[J].Machine Learn,1992,8(3):279.
[17]
Watkins C J,Dayan P.Q-learning[J].Machine Learn,1992,8(3):225.
[18]
Mnih V,Kavukcuoglu K,Silver D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540): 529.