Fixed-Wing UAVs flocking in continuous spaces: A deep reinforcement learning approach
پهپادهای ثابت بال در فضاهای مداوم هجوم می آورند: یک رویکرد یادگیری تقویتی عمیق-2020
Fixed-Wing UAVs (Unmanned Aerial Vehicles) flocking is still a challenging problem due to the kinematics complexity and environmental dynamics. In this paper, we solve the leader–followers flocking problem using a novel deep reinforcement learning algorithm that can generate roll angle and velocity commands by training an end-to-end controller in continuous state and action spaces. Specifically, we choose CACLA (Continuous Actor–Critic Learning Automation) as the base algorithm and we use the multi-layer perceptron to represent both the actor and the critic. Besides, we further improve the learning efficiency by using the experience replay technique that stores the training data in the experience memory and samples from the memory as needed. We have compared the performance of the proposed CACER (Continuous Actor–Critic with Experience Replay) algorithm with benchmark algorithms such as DDPG and double DQN in numerical simulation, and we have demonstrated the performance of the learned optimal policy in semi-physical simulation without any parameter tuning.
Keywords: Fixed-wing UAV | Flocking | Reinforcement learning | Actor–critic
Coactive design of explainable agent-based task planning and deep reinforcement learning for human-UAVs teamwork
طراحی همکاری برنامه ریزی وظیفه مبتنی بر عامل و یادگیری تقویتی عمیق برای کار گروهی انسان-پهپاد-2020
Unmanned Aerial Vehicles (UAVs) are useful in dangerous and dynamic tasks such as search-and-rescue, forest surveillance, and anti-terrorist operations. These tasks can be solved better through the collaboration of multiple UAVs under human supervision. However, it is still difficult for human to monitor, understand, predict and control the behaviors of the UAVs due to the task complexity as well as the black-box machine learning and planning algorithms being used. In this paper, the coactive design method is adopted to analyze the cognitive capabilities required for the tasks and design the interdependencies among the heterogeneous teammates of UAVs or human for coherent collaboration. Then, an agent-based task planner is proposed to automatically decompose a complex task into a sequence of explainable subtasks under constrains of resources, execution time, social rules and costs. Besides, a deep reinforcement learning approach is designed for the UAVs to learn optimal policies of a flocking behavior and a path planner that are easy for the human operator to understand and control. Finally, a mixed-initiative action selection mechanism is used to evaluate the learned policies as well as the human’s decisions. Experimental results demonstrate the effectiveness of the proposed methods
KEYWORDS : Coactive design | Deep reinforcement learning | Human-robot teamwork | Mixed-initiative | Multi-agent system | Task planning | UAV
Flocking based evolutionary computation strategy for measuring centrality of online social networks
استراتژی محاسبات تکاملی مبتنی بر Flocking برای اندازه گیری مرکزیت شبکه های اجتماعی آنلاین-2017
Centrality in social network is one of the major research topics in social network analysis. Even though there are more than half a dozen methods to find centrality of a node, each of these methods has some drawbacks in one aspect or the other. This paper analyses different centrality calculation methods and proposes a new swarm based method named Flocking Based Centrality for Social network (FBCS). This new computation technique makes use of parameters that are more realistic and practical in online social networks. The interactions between nodes play a significant role in determining the centrality of node. The new method has been calculated both empirically as well as experimentally. The new method is tested, verified and validated for different sets of random networks and benchmark datasets. The method has been correlated with other state of the art centrality measures. The new centrality measure is found to be realistic and suits well with online social networks. The proposed method can be used in applications such as finding the most prestigious node and for discovering the node which can influence maximum number of users in an online social network. FBCS centrality has higher Kendall’s tau correlation when compared with other state of the art centrality methods. The robustness of the FBCS centrality is found to be better than other centrality measures.
Key Terms: Centrality in social network | Degree of nodes | Online social network analysis | Boid’s algorithm | Flocking of birds