MemoryPath: A Deep Reinforcement Learning Framework for Incorporating Memory Component into Knowledge Graph Reasoning
MemoryPath: یک چارچوب یادگیری تقویتی عمیق برای ترکیب مولفه حافظه در استدلال نمودار دانش-2020
Knowledge Graph (KG) is identified as a major area in artificial intelligence, which is used for many real-world applications. The task of knowledge graph reasoning has been widely used and proven to be effective, which aims to find these reasonable paths for various relations to solve the issue of incompleteness in KGs. However, many previous works on KG reasoning, such as path-based or rein- forcement learning-based methods, are too reliant on the pre-training, where the paths from the head entity and the target entity must be given to pre-train the model, which would easily lead the model to overfit on the given paths seen in the pre-training. To address this issue, we propose a novel reasoning model named MemoryPath with a deep reinforcement learning framework, which incorporates Long Short Term Memory(LSTM) and graph attention mechanism to form the memory component. The well-designed memory component can get rid of the pre-training so that the model doesn’t depend on the given target entity for training. A tailored mechanism of reinforcement learning is presented in this proposed deep reinforcement framework to optimize the training procedure, where two metrics, Mean Selection Rate (MSR) and Mean Alternative Rate (MAR), are defined to quantitatively mea- sure the complexities of the query relations. Meanwhile, three different training mechanisms, Action Dropout, Reward Shaping and Force Forward, are proposed to optimize the training process of the proposed MemoryPath. The proposed MemoryPath is validated on two datasets from FB15K-237 and NELL-995 on different tasks including fact prediction, link prediction and success rate in finding paths. The experimental results demonstrate that the tailored mechanism of reinforcement learning make the MemoryPath achieves state-of-the-art performance comparing with the other models. Also, the qualitative analysis indicates that the MemoryPath can store the learning process and automati- cally find the promising paths for a reasoning task during the training, and shows the effectiveness of the memory component.
Keywords: Knowledge Graph Reasoning | Memory Component | Deep Reinforcement Learning | Link Prediction | Attentional Path | Force Forward
ADRL: An attention-based deep reinforcement learning framework for knowledge graph reasoning
ADRL: یک چارچوب یادگیری تقویتی عمیق مبتنی بر توجه برای استدلال نمودار دانش-2020
Knowledge graph reasoning is one of the key technologies for knowledge graph construction, which plays an important part in application scenarios such as vertical search and intelligent question answering. It is intended to infer the desired entity from the entities and relations that already exist in the knowledge graph. Most current methods for reasoning, such as embedding-based methods, globally embed all entities and relations, and then use the similarity of vectors to infer relations between entities or whether given triples are true. However, in real application scenarios, we require a clear and interpretable target entity as the output answer. In this paper, we propose a novel attention-based deep reinforcement learning framework (ADRL) for learning multi-hop relational paths, which improves the efficiency, generalization capacity, and interpretability of conventional approaches through the structured perception of deep learning and relational reasoning of reinforcement learning. We define the entire process of reasoning as a Markov decision process. First, we employ CNN to map the knowledge graph to a low-dimensional space, and a message-passing mechanism to sense neighbor entities at each level, and then employ LSTM to memorize and generate a sequence of historical trajectories to form a policy and value functions. We design a relational module that includes a selfattention mechanism that can infer and share the weights of neighborhood entity vectors and relation vectors. Finally, we employ the actor–critic algorithm to optimize the entire framework. Experiments confirm the effectiveness and efficiency of our method on several benchmark data sets.
Keywords: Knowledge graph | Knowledge reasoning | Reinforcement learning | Deep learning | Attention