Automated vehicles (AVs) are deemed to be the key element for the intelligent transportation system in the future. Many studies have been made to improve AVs’ ability of environment recognition and vehicle control, while the attention paid to decision making is not enough and the existing decision algorithms are very preliminary. Therefore, a framework of the decisionmaking training and learning is put forward in this paper. It consists of two parts: the deep reinforcement learning (DRL) training program and the high-fidelity virtual simulation environment. Then the basic microscopic behavior, car-following (CF), is trained within this framework. In addition, theoretical analysis and experiments were conducted to evaluate the proposed reward functions for accelerating training using DRL. The results show that on the premise of driving comfort, the efficiency of the trained AV increases 7.9% and 3.8% respectively compared to the classical adaptive cruise control models, intelligent driver model and constant-time headway policy. Moreover, on a more complex three-lane section, we trained an integrated model combining both CF and lane-changing behavior, with the average speed further growing 2.4%. It indicates that our framework is effective for AV’s decision-making learning.
Keywords: Automated vehicle | Decision making | Deep reinforcement learning | Reward function

Fault diagnosis methods for rotating machinery have always been a hot research topic, and artificial intelligencebased approaches have attracted increasing attention from both researchers and engineers. Among those related studies and methods, artificial neural networks, especially deep learning-based methods, are widely used to extract fault features or classify fault features obtained by other signal processing techniques. Although such methods could solve the fault diagnosis problems of rotating machinery, there are still two deficiencies. (1) Unable to establish direct linear or non-linear mapping between raw data and the corresponding fault modes, the performance of such fault diagnosis methods highly depends on the quality of the extracted features. (2) The optimization of neural network architecture and parameters, especially for deep neural networks, requires considerable manual modification and expert experience, which limits the applicability and generalization of such methods. As a remarkable breakthrough in artificial intelligence, AlphaGo, a representative achievement of deep reinforcement learning, provides inspiration and direction for the aforementioned shortcomings. Combining the advantages of deep learning and reinforcement learning, deep reinforcement learning is able to build an end-to-end fault diagnosis architecture that can directly map raw fault data to the corresponding fault modes. Thus, based on deep reinforcement learning, a novel intelligent diagnosis method is proposed that is able to overcome the shortcomings of the aforementioned diagnosis methods. Validation tests of the proposed method are carried out using datasets of two types of rotating machinery, rolling bearings and hydraulic pumps, which contain a large number of measured raw vibration signals under different health states and working conditions. The diagnosis results show that the proposed method is able to obtain intelligent fault diagnosis agents that can mine the relationships between the raw vibration signals and fault modes autonomously and effectively. Considering that the learning process of the proposed method depends only on the replayed memories of the agent and the overall rewards, which represent much weaker feedback than that obtained by the supervised learning-based method, the proposed method is promising in establishing a general fault diagnosis architecture for rotating machinery.
Keywords: Fault diagnosis | Rotating machinery | Deep reinforcement learning | Deep Q-network

This study aims to provide a systematic review of the applications of machine learning methods in addiction research. In this study, multiple searches on MEDLINE, Embase and the Cochrane Database of Systematic Reviews were performed. 23 full-text articles were assessed and 17 articles met the inclusion criteria for the final review. The selected studies covered mainly substance addiction (N=14, 82.4%), including smoking (N=4), alcohol drinking (N=3), as well as uses of cocaine (N=4), opioids (N=1), and multiple substances (N=2). Other studies were non-substance addiction (N=3, 17.6%), including gambling (N=2) and internet gaming (N=1). There were eight cross-sectional, seven cohort, one non-randomized controlled, and one crossover trial studies. Majority of the studies employed supervised learning (N=13), and others employed unsupervised learning (N=2) and reinforcement learning (N=2). Among the supervised learning studies, five studies used ensemble learning methods or multiple algorithm comparisons, six used regression, and two used classification. The two included reinforcement learning studies used the direct methods. These results suggest that machine learning methods, particularly supervised learning are increasingly used in addiction psychiatry for informing medical decisions.
Keywords: Machine learning | Supervised learning | Unsupervised learning | Reinforcement learning | Addiction

Because of the increasing application of reinforcement learning (RL), particularly deep Q- learning algorithm, research organizations utilize it with increasing frequency. The predic- tion of cyber vulnerability and development of efficient real-time online network intrusion detection (NID) systems are progressions toward becoming RL-powered. An open issues in NID is the model design and prediction of real-time online data composed of a series of time-related feature patterns. There have been concerns regarding the operation of the developed systems because cyber-attack scenarios vary continuously to circumvent NID. These issues have been related to the human interaction significance and the decrease in accuracy verification. Therefore, we employ an RL that permits a deep auto-encoder in the Q-network (DAEQ-N). The proposed DAEQ-N attempts to achieve the maximum prediction accuracy in online learning systems into which continuous behavior patterns are fed and which are trained with more significant weights by classifying it as either “normal”or “anomalous.”
Keywords: Network anomalies | Online learning systems | Network intrusion detection (NID) | Deep Q-Network (DQN) | Reinforcement learning (RL)

Due to environment-friendliness, renewable energy like solar power and wind power is more and more introduced to energy systems all over the world. Simultaneously, high penetrations of wind and solar generation also have brought severe curtailment of wind and solar. How to alleviate curtailment of wind and solar is a crucial problem in evaluating accommodation capability of renewable energy, which reflects the extent of utilization of renewable energy and economic benefits. The uncertainty of renewable energy brings challenges to precisely describe renewable generation, which leads to difficulty in designing effective mechanisms for accommodation capability of renewable energy. Existing work suffers from high computation overhead from frequently updated data, and low precision of describing renewable energy, which leads to less effective policies for renewable energy accommodation and underestimated accommodation capability. To make the most of renewable energy, an algorithm AccCap-DRL based on deep reinforcement learning is proposed. AccCap-DRL partitions a distribution into segments by time intervals, employs WGAN to describe distributions of renewable energy data, and employs DDPG to obtain approximate policies for renewable energy accommodation in different scenarios. Simulation results from real power generation and users’ demand data show high effectiveness of the proposed algorithm, and high efficiency of evaluating accommodation capability
Keywords: Accommodation capability | Deep reinforcement learning | Uncertain renewable energy description