عنوان انگلیسی مقاله:
Adaptive cache pre-forwarding policy for distributed deep learning
ترجمه فارسی عنوان مقاله:
سیاست پیش هدایت حافظه پنهان تطبیقی برای یادگیری عمیق توزیع شده
Sciencedirect - Elsevier - Computers and Electrical Engineering 82 (2020) 106558
Sheng-Tzong Cheng a , Chih-Wei Hsu a , Gwo-Jiun Horng b , ∗, Che-Hsuan Lin a
With the rapid growth of deep learning algorithms, several high-accuracy models have been developed and applied to many real-world domains. Deep learning is parallel and suitable for distributed computing, which can significantly improve the system through- put. However, there is a bottleneck for cross-machine training, that is, network latency. Nodes frequently need to wait for synchronization, and the content of each synchroniza- tion may range from several megabytes to hundred megabytes. Thus, network communi- cation takes considerable time in the training process, which reduces system performance. Therefore, many computing architectures have been proposed. This paper proposes a type of distributed computing system for deep learning. Our design aims to reduce synchro- nization times and network blocking times by using a new cache mechanism, called cache pre-forwarding. The design concept of cache pre-forwarding aims to exploit reinforcement learning to train a pre-forwarding policy to increase the cache hit rate. Because of the features of reinforcement learning, our policy is adaptive and applicable to different com- puting environments. Finally, we experimentally demonstrate that our system is feasible.
Keywords: Deep learning | Distributed computing | Cache | Reinforcement learning