Graph Deconvolutional Networks
شبکه Deconvolutional گراف-2020
Graphs and networks are very common data structure for modelling complex systems that are composed of a number of nodes and topologies, such as social networks, citation networks, biological protein-protein interactions networks, etc. In recent years, machine learning has become an efficient technique to obtain representation of graph for downstream graph analysis tasks, including node classification, link prediction, and community detection. Different with traditional graph analytical models, the representation learning on graph tries to learn low dimensional embeddings by means of machine learning models that could be trained in supervised, unsupervised or semi-supervised manners. Compared with traditional approaches that directly use input node attributes, these embeddings are much more informative and helpful for graph analysis. There are a number of developed models in this respect, that are different in the ways of measuring similarity of vertexes in both original space and feature space. In order to learn more efficient node representation with better generalization property, we propose a task-independent graph representation model, called as graph deconvolutional network (GDN), and corresponding unsupervised learning algorithm in this paper. Different with graph convolution network (GCN) from the scratch, which produces embeddings by convolving input attribute vec- tors with learned filters, the embeddings of the proposed GDN model are desired to be convolved with filters so that reconstruct the input node attribute vectors as far as possible. The embeddings and filters are alternatively optimized in the learning procedure. The correctness of the proposed GDN model is verified by multiple tasks over several datasets. The experimental results show that the GDN model outperforms existing alternatives with a big margin
Keywords: graph representation | representation learning | unsupervised learning |node embedding | machine learning
Data analysis of multi-dimensional thermophysical properties of liquid substances based on clustering approach of machine learning
تجزیه و تحلیل داده ها از خصوصیات حرارتی فیزیکی چند بعدی مواد مایع بر اساس روش خوشه بندی یادگیری ماشین-2019
In order to develop an efficient framework for global screening in the material exploration, we performed a clustering analysis of machine learning on the multi-dimensional thermophysical properties of the liquid substances. Data mining using a self-organizing map (SOM) based on the unsupervised learning was employed to project high-dimensional thermophysical data onto a low-dimensional space. Here we adopted 98 liquid substances with eight thermo-physical properties for the SOM training in order to group the liquid substances. The present SOM-clustering approach properly categorized liquid substances according to the chemical species characterized by the functional groups.
Keywords: Self-organizing map | Clustering analysis | Machine learning | Thermophysical properties | Heat medium
Forming a new small sample deep learning model to predict total organic carbon content by combining unsupervised learning with semi supervised learning
تشکیل یک نمونه جدید کوچک از یادگیری عمیق برای پیش بینی مقدار کل کربن آلی با ترکیب یادگیری بدون نظارت با یادگیری نیمه نظارت-2019
The total organic carbon (TOC) content is a parameter that is directly used to evaluate the hydrocarbon generation capacity of a reservoir. For a reservoir, accurately calculating TOC using well logging curves is a problem that needs to be solved. Machine learning models usually yield the most accurate results. Problems of existing machine learning models that are applied to well logging interpretations include poor feature extraction methods and limited ability to learn complex functions. However, logging interpretation is a small sample problem, and traditional deep learning with strong feature extraction ability cannot be directly used; thus, a deep learning model suitable for logging small sample features, namely, a combination of unsupervised learning and semisupervised learning in an integrated DLM (IDLM), is proposed in this paper and is applied to the TOC prediction problem. This study is also the first systematic application of a deep learning model in a well logging interpretation. First, the model uses a stacked extreme learning machine sparse autoencoder (SELM-SAE) unsupervised learning method to perform coarse feature extraction for a large number of unlabeled samples, and a feature extraction layer consisting of multiple hidden layers is established. Then, the model uses the deep Boltzmann machine (DBM) semisupervised learning method to learn a large number of unlabeled samples and a small number of labeled samples (the input is extracted from logging curve values into SELM-SAE extracted features), and the SELM-SAE and DBM are integrated to form a deep learning model (DLM). Finally, multiple DLMs are combined to form an IDLM algorithm through an improved weighted bagging algorithm. A total of 2381 samples with an unlabeled logging response from 4 wells in 2 shale gas areas and 326 samples with determined TOC values are used to train the model. The model is compared with 11 other machine learning models, and the IDLM achieves the highest precision. Moreover, the simulation shows that for the TOC prediction problem, when the number of labeled samples included in the training is greater than 20, even if this number of samples is used to train 10 hidden layer IDLMs, the trained model has a very low overfitting probability and exhibits the potential to exceed the accuracies of other models. Relative to the existing mainstream shallow model, the IDLM based on a DLM provides the most advanced performance and is more effective. This method implements a small sample deep learning algorithm for TOC prediction and can feasibly use deep learning to solve logging interpretation problems and other small sample set problems for the first time. The IDLM achieves high precision and provides novel insights that can aid in oil and gas exploration and development.
Keywords: Small sample | Deep learning | Integrated deep learning model | Coarse-detailed feature extraction | Total organic carbon content
Deep learning only by normal brain PET identify unheralded brain anomalies
یادگیری عمیق فقط با PET مغز نرمال ناهنجاریهای مغزی هدایت نشده را شناسایی می کند-2019
Background: Recent deep learning models have shown remarkable accuracy for the diagnostic classification. However, they have limitations in clinical application due to the gap between the training cohorts and realworld data. We aimed to develop a model trained only by normal brain PET data with an unsupervised manner to identify an abnormality in various disorders as imaging data of the clinical routine. Methods: Using variational autoencoder, a type of unsupervised learning, Abnormality Scorewas defined as how far a given brain image is from the normal data. The model was applied to FDG PET data of Alzheimers disease (AD) andmild cognitive impairment (MCI) and clinical routine FDG PET data for assessing behavioral abnormality and seizures. Accuracy was measured by the area under curve (AUC) of receiver-operating-characteristic (ROC) curve.We investigated whether deep learning has additional benefits with experts visual interpretation to identify abnormal patterns. Findings: The AUC of the ROC curve for differentiating AD was 0.90. The changes in cognitive scores frombaseline to 2-year follow-up were significantly correlated with Abnormality Score at baseline. The AUC of the ROC curve for discriminating patients with various disorders from controls was 0.74. Experts visual interpretation was helped by the deep learning model to identify abnormal patterns in 60% of cases initially not identified without the model. Interpretation:We suggest that deep learning model trained only by normal data was applicable for identifying wide-range of abnormalities in brain diseases, even uncommon ones, proposing its possible use for interpreting real-world clinical data.
Keywords: PET | Deep learning | Variational autoencoder | Alzheimer | Anomaly detection
Applications of machine learning in addiction studies: A systematic review
کاربردهای یادگیری ماشین در مطالعات اعتیاد: یک مرور سیستماتیک-2019
This study aims to provide a systematic review of the applications of machine learning methods in addiction research. In this study, multiple searches on MEDLINE, Embase and the Cochrane Database of Systematic Reviews were performed. 23 full-text articles were assessed and 17 articles met the inclusion criteria for the final review. The selected studies covered mainly substance addiction (N=14, 82.4%), including smoking (N=4), alcohol drinking (N=3), as well as uses of cocaine (N=4), opioids (N=1), and multiple substances (N=2). Other studies were non-substance addiction (N=3, 17.6%), including gambling (N=2) and internet gaming (N=1). There were eight cross-sectional, seven cohort, one non-randomized controlled, and one crossover trial studies. Majority of the studies employed supervised learning (N=13), and others employed unsupervised learning (N=2) and reinforcement learning (N=2). Among the supervised learning studies, five studies used ensemble learning methods or multiple algorithm comparisons, six used regression, and two used classification. The two included reinforcement learning studies used the direct methods. These results suggest that machine learning methods, particularly supervised learning are increasingly used in addiction psychiatry for informing medical decisions.
Keywords: Machine learning | Supervised learning | Unsupervised learning | Reinforcement learning | Addiction
Unsupervised deep learning and semi-automatic data labeling in weed discrimination
یادگیری عمیق نظارت نشده و برچسب زدن داده های نیمه اتوماتیک در تبعیض علف هرز-2019
In recent years, supervised Deep Neural Networks have achieved the state-of-the-art in image recognition and this success has spread in many areas. In agricultural field, several researches have been conducted using architectures such as Convolutional Neural Networks. Despite this success, these works are still highly dependent on very time–costly manual data labeling. In contrast to this scenario, Unsupervised Deep Learning has no dependency on data labeling and is targeted as the future of the area, but after a promising start has been obfuscated by the success of supervised networks. Meanwhile, the low-cost of acquisition of field crop imagery using Unnamed Aerial Vehicles could be largely boosted in real-world applications if these images could be annotated without the need for a human specialist. In this work, we tested two recent unsupervised deep clustering algorithms, Joint Unsupervised Learning of Deep Representations and Image Clusters (JULE) and Deep Clustering for Unsupervised Learning of Visual Features (DeepCluster), using two public weed datasets. The first dataset was captured in a soybean plantation in Brazil and discriminates weeds between grass and broadleaf. The second dataset consists of 17,509 labeled images of eight nationally significant weed species native to Australia. We evaluated the purely unsupervised clustering performance using the NMI and Unsupervised Clustering Accuracy metrics and analysed the effects of techniques like data augmentation and transfer learning to improve clustering quality in a broad discussion that can be useful for unsupervised deep clustering in general. We also propose the usage of semi-automatic data labeling which greatly reduces the cost of manual data labeling and can be easily replicated to different datasets. This approach achieved 97% accuracy in discrimination of grass and broadleaf while reducing the number of manual annotations by 100 times, using a custom set of training images, without images labeled using inaccurate clusters.
Keywords: Deep learning | Unsupervised clustering | Weed discrimination | Semi-automatic labeling
Machine learning in resting-state fMRI analysis
یادگیری ماشین در تجزیه و تحلیل fMRI وضعیت استراحت-2019
Machine learning techniques have gained prominence for the analysis of resting-state functional Magnetic Resonance Imaging (rs-fMRI) data. Here, we present an overview of various unsupervised and supervised machine learning applications to rs-fMRI. We offer a methodical taxonomy of machine learning methods in restingstate fMRI. We identify three major divisions of unsupervised learning methods with regard to their applications to rs-fMRI, based on whether they discover principal modes of variation across space, time or population. Next, we survey the algorithms and rs-fMRI feature representations that have driven the success of supervised subjectlevel predictions. The goal is to provide a high-level overview of the burgeoning field of rs-fMRI from the perspective of machine learning applications.
Keywords: Machine learning | Resting-state | Functional MRI | Intrinsic networks | Brain connectivity
Queuing theory guided intelligent traffic scheduling through video analysis using Dirichlet process mixture model
زمانبندی ترافیک هوشمند هدایت شده تئوری صف از طریق تجزیه و تحلیل ویدئو با استفاده از مدل فرایند Dirichlet مخلوط-2019
Intelligent traffic signaling is an important part of city road traffic management systems. In many coun- tries, it is done through supervised/semi-supervised ways. With the advances in computer vision and machine learning, it is now possible to develop expert systems guided intelligent traffic signaling sys- tems that are unsupervised in nature. In order to schedule traffic signals, it is essential to learn the traf- fic characterization parameters such as the number of vehicles, their arrival and departure rates, etc. In this work, we use unsupervised machine learning with the help of a modified Dirichlet Process Mixture Model (DPMM) to measure the aforementioned traffic parameters. This has been done using a new fea- ture, named temporal clusters or tracklets extracted using DPMM. Detailed analysis on tracklet behavior during signal on/offperiod has been carried out to derive a queuing theory-based method for signal du- ration prediction. The queuing behavior at a junction is analyzed using tracklets for understanding their applicability. Queue clearance time at the junction has been used for predicting the signal duration with the help of Gaussian regression of historical data. Two publicly available video datasets, namely QMUL and MIT have been used for verification of the hypothesis. The mean absolute error (MAE) of the proposed method using tracklets has been reduced by a factor of 2.4 and 6.3 when compared with the tracks generated using Kernel Correlation Filters (KCF) and Kanade–Lucas–Tomasi (KLT), respectively. Through experiments, we are also able to establish that KCF and KLT tracks do not consider spatial occupancy of the vehicles on roads, leading to error in the estimation. The results reveal that the proposed queuing theory-based approach predicts the signal duration for the next cycle more accurately as compared to the ground truths. The method can be used for building intelligent traffic control systems for roadway junctions in cities and highways.
Keywords: Traffic intersection management | Signal duration | prediction Dirichlet process | Queuing theory | Unsupervised learning | Visual surveillance
Biologically plausible deep learning—But how far can we go with shallow networks?
یادگیری عمیق زیست شناختی قابل قبول: اما تا چه حد می توانیم با شبکه های کم عمق برویم؟-2019
Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data set. Here, we investigate how far we can go on digit (MNIST) and object (CIFAR10) classification with biologically plausible, local learning rules in a network with one hidden layer and a single readout layer. The hidden layer weights are either fixed (random or random Gabor filters) or trained with unsupervised methods (Principal/Independent Component Analysis or Sparse Coding) that can be implemented by local learning rules. The readout layer is trained with a supervised, local learning rule. We first implement these models with rate neurons. This comparison reveals, first, that unsupervised learning does not lead to better performance than fixed random projections or Gabor filters for large hidden layers. Second, networks with localized receptive fields perform significantly better than networks with all-to-all connectivity and can reach backpropagation performance on MNIST. We then implement two of the networks – fixed, localized, random & random Gabor filters in the hidden layer – with spiking leaky integrate-and-fire neurons and spike timing dependent plasticity to train the readout layer. These spiking models achieve >98.2% test accuracy on MNIST, which is close to the performance of rate networks with one hidden layer trained with backpropagation. The performance of our shallow network models is comparable to most current biologically plausible models of deep learning. Furthermore, our results with a shallow spiking network provide an important reference and suggest the use of data sets other than MNIST for testing the performance of future models of biologically plausible deep learning.
Keywords: Deep learning | Local learning rules | Random projections | Unsupervised feature learning | Spiking networks | MNIST
Computer aided Alzheimers disease diagnosis by an unsupervised deep learning technology
تشخیص بیماری آلزایمر به کمک کامپیوتر توسط یک تکنولوژی یادگیری عمیق-2019
Deep learning technologies have played more and more important roles in Computer Aided Diagnosis (CAD) in medicine. In this paper, we tackled the problem of automatic prediction of Alzheimer’s Disease (AD) based on Magnetic Resonance Imaging (MRI) images, and propose a fully unsupervised deep learn- ing technology for AD diagnosis. We first implement the unsupervised Convolutional Neural Networks (CNNs) for feature extraction, and then utilize the unsupervised predictor to achieve the final diagnosis. In the proposed method, two kinds of data forms, one slice and three orthogonal panels (TOP) of MRI image, are employed as the input data respectively. Experimental results run on all the 1075 subjects in database of the Alzheimer’s Disease Neuroimaging Initiative (ADNI 1 1.5T) show that the proposed method with one slice data yields the promising prediction results for AD vs. MCI (accuracy 95.52%) and MCI vs. NC (accuracy 90.63%), and the proposed methods with TOP data yields the best overall prediction results for AD vs. MCI (accuracy 97.01%) and MCI vs. NC (accuracy 92.6%).
Keywords: Deep learning | Unsupervised learning | Convolutional neural network | Alzheimer’s disease prediction | Magnetic Resonance Imaging data | Computer aided diagnosis