عنوان انگلیسی مقاله:
Towards safe reinforcement-learning in industrial grid-warehousing
ترجمه فارسی عنوان مقاله:
به سمت تقویت و یادگیری ایمن در شبکه انبارهای صنعتی
Sciencedirect - Elsevier - Information Sciences, 537 (2020) 467-484. doi:10.1016/j.ins.2020.06.010
Per-Arne Andersen ⇑, Morten Goodwin, Ole-Christoffer Granmo
Reinforcement learning has shown to be profoundly successful at learning optimal policies
for simulated environments using distributed training with extensive compute capacity.
Model-free reinforcement learning uses the notion of trial and error, where the error is a
vital part of learning the agent to behave optimally. In mission-critical, real-world environments,
there is little tolerance for failure and can cause damaging effects on humans and
equipment. In these environments, current state-of-the-art reinforcement learning
approaches are not sufficient to learn optimal control policies safely.
On the other hand, model-based reinforcement learning tries to encode environment
transition dynamics into a predictive model. The transition dynamics describes the mapping
from one state to another, conditioned on an action. If this model is accurate enough,
the predictive model is sufficient to train agents for optimal behavior in real environments.
This paper presents the Dreaming Variational Autoencoder (DVAE) for safely learning good
policies with a significantly lower risk of catastrophes occurring during training. The algorithm
combines variational autoencoders, risk-directed exploration, and curiosity to train
deep-q networks inside ”dream” states. We introduce a novel environment, ASRS-Lab,
for research in the safe learning of autonomous vehicles in grid-based warehousing. The
work shows that the proposed algorithm has better sample efficiency with similar performance
to novel model-free deep reinforcement learning algorithms while maintaining
safety during training.
Keywords: Model-based reinforcement learning | Neural networks | Variational autoencoder | Markov decision processes | Exploration | Safe reinforcement learning