با سلام خدمت کاربران عزیز، به اطلاع می رساند ترجمه مقالاتی که سال انتشار آن ها زیر 2008 می باشد رایگان بوده و میتوانید با وارد شدن در صفحه جزییات مقاله به رایگان ترجمه را دانلود نمایید.
Extract interpretability-accuracy balanced rules from artificial neural networks: A review
استخراج قوانین متعادل با دقت تفسیر از شبکه های عصبی مصنوعی: بررسی-2020
Artificial neural networks (ANN) have been widely used and have achieved remarkable achievements. However, neural networks with high accuracy and good performance often have extremely complex in- ternal structures such as deep neural networks (DNN). This shortcoming makes the neural networks as incomprehensible as a black box, which is unacceptable in some practical applications. But pursuing ex- cessive interpretation of the neural networks will make the performance of the model worse. Based on this contradictory issue, we first summarize the mainstream methods about quantitatively evaluating the accuracy and interpretability of rule set. And then review existing methods on extracting rules from Mul- tilayer Perceptron (MLP) and DNN in three categories: Decomposition Approach (Extract rules in neuron level such as visualizing the structure of network), Pedagogical Approach (By studying the correspon- dence between input and output such as by computing gradient) and Eclectics Approach (Combine the above two ideas). Some potential research directions about extracting rules from DNN are discussed in the last.
Keywords: Rule extraction | Accuracy | Interpretability | Multilayer Perceptron | Deep neural network
Sparse low rank factorization for deep neural network compression
فاکتورسازی رتبه پراکنده برای فشرده سازی شبکه عصبی عمیق-2020
Storing and processing millions of parameters in deep neural networks is highly challenging during the deployment of model in real-time application on resource constrained devices. Popular low-rank approx- imation approach singular value decomposition (SVD) is generally applied to the weights of fully con- nected layers where compact storage is achieved by keeping only the most prominent components of the decomposed matrices. Years of research on pruning-based neural network model compression re- vealed that the relative importance or contribution of each neuron in a layer highly vary among each other. Recently, synapses pruning has also demonstrated that having sparse matrices in network archi- tecture achieve lower space and faster computation during inference time. We extend these arguments by proposing that the low-rank decomposition of weight matrices should also consider significance of both input as well as output neurons of a layer. Combining the ideas of sparsity and existence of un- equal contributions of neurons towards achieving the target, we propose sparse low rank (SLR) method which sparsifies SVD matrices to achieve better compression rate by keeping lower rank for unimportant neurons. We demonstrate the effectiveness of our method in compressing famous convolutional neural networks based image recognition frameworks which are trained on popular datasets. Experimental re- sults show that the proposed approach SLR outperforms vanilla truncated SVD and a pruning baseline, achieving better compression rates with minimal or no loss in the accuracy. Code of the proposed ap- proach is avaialble at https://github.com/sridarah/slr .
Keywords: Low-rank approximation | Singular value decomposition | Sparse matrix | Deep neural networks | Convolutional neural networks
Log-sum enhanced sparse deep neural network
شبکه عصبی پراکنده عمیق با افزایش log-sum-2020
How to design deep neural networks (DNNs) for the representation and analysis of high dimensional but small sample size data is still a big challenge. One solution is to construct a sparse network. At present, there exist many approaches to achieve sparsity for DNNs by regularization, but most of them are carried out only in the pre-training process due to the difficulty in the derivation of explicit formulae in the finetuning process. In this paper, a log-sum function is used as the regularization terms for both the responses of hidden neurons and the network connections in the loss function of the fine-tuning process. It provides a better approximation to the L0-norm than several often used norms. Based on the gradient formula of the loss function, the fine-tuning process can be executed more efficiently. Specifically, the commonly used gradient calculation in many deep learning research platforms, such as PyTorch or TensorFlow, can be accelerated. Given the analytic formula for calculating gradients used in any layer of DNN, the error accumulated from successive numerical approximations in the differentiation process can be avoided. With the proposed log-sum enhanced sparse deep neural network (LSES-DNN), the sparsity of the responses and the connections can be well controlled to improve the adaptivity of DNNs. The proposed model is applied to MRI data for both the diagnosis of schizophrenia and the study of brain developments. Numerical experiments demonstrate its superior performance among several classical classifiers tested.
Keywords: Deep neural network | Log-sum enhanced sparsity | Back propagation algorithm | Concise gradient formula | Magnetic resonance imaging
Detection of flood disaster system based on IoT, big data and convolutional deep neural network
تشخیص سیستم بحرانی سیل بر اساس اینترنت اشیا، داده های بزرگ و شبکه عصبی عمیق پیچشی-2020
Natural disasters could be defined as a blend of natural risks and vulnerabilities. Each year, natural as well as human-instigated disasters, bring about infrastructural damages, distresses, revenue losses, injuries in addition to huge death roll. Researchers around the globe are trying to find a unique solution to gather, store and analyse Big Data (BD) in order to predict results related to flood based prediction system. This paper has proposed the ideas and methods for the detection of flood disaster based on IoT, BD, and convolutional deep neural network (CDNN) to overcome such difficulties. First, the input data is taken from the flood BD. Next, the repeated data are reduced by using HDFS map-reduce (). After removal of repeated data, the data are pre-processed using missing value imputation and normalization function. Then, centred on the pre-processed data, the rule is generated by using a combination of attributes method. At the last stage, the generated rules are provided as the input to the CDNN classifier which classifies them as a) chances for the occurrence of flood and b) no chances for the occurrence of a flood. The outcomes obtained from the proposed CDNN method is compared parameters like Sensitivity, Specificity, Accuracy, Precision, Recall and F-score. Moreover, when the outcomes is compared other existing algorithms like Artificial Neural Network (ANN) & Deep Learning Neural Network (DNN), the proposed system gives is very accurate result than other methods.
Keywords: Hadoop distributed file system (HDFS) | Convolutional deep neural network (CDNN) | Normalization | Rule generation | Missing value imputation
کمترین از دست دادن حاشیه برای تشخیص چهره عمیق
سال انتشار: 2020 - تعداد صفحات فایل pdf انگلیسی: 9 - تعداد صفحات فایل doc فارسی: 24
تشخیص چهره موفقیت بزرگی به دست آورده است که دلیل اصلی آن توسعه سریع شبکه های عصبی عمیق (DNN) در سال های اخیر است. کارکردهای مختلف ازدست دادن (اتلاف) در یک شبکه عصبی عمیق قابل استفاده است که منجر به عملکرد متفاوتی می شود. اخیراً برخی از کارکردهای تلفات پیشنهاد داده شده است. با این حال، آن ها نمی توانند مساله جهت گیری حاشیه ای را که در مجموعه داده های غیر متعادل وجود دارد حل کنند. در این مقاله حل مساله تمایل حاشیه ای را با تعیین یک حاشیه حداقلی برای تمامی زوج کلاس ها پیشنهاد می دهیم. ما تابع اتلاف جدیدی به نام حداقل اتلاف حاشیه ای (MML) پیشنهاد می دهیم که هدف آن گسترش محدوده آن هایی است که به زوج های مرکزی دسته بیش از حد نزدیک می شوند تا قابلیت متمایز کننده ویژگی های عمیق را ارتقاء دهد. تابع MML همراه با توابع Softmax Loss و Centre Loss بر فرآیند آموزش نظارت می کنند تا حاشیه های تمامی دسته ها را صرف نظر از توزیع دسته آن ها مورد نظارت قرار دهند. ما تابع MML را در پلتفورم Inception-ResNet-v1 پیاده سازی می کنیم و آزمایش های گسترده ای را بر روی هفت مجموعه داده تشخیص چهره انجام می دهیم که شامل MegaFace، FaceScrub، LFW، SLLFW، YTF، IJB-B و IJB-C است. نتایج تجربی نشان می دهد که تابع از دست دادن MML پیشنهادی منجر به حالت جدیدی در تشخیص چهره می شود و اثر منفی جهت گیری حاشیه ای را کاهش می دهد.
کلید واژه ها :یادگیری عمیق | شبکه های عصبی باز رخدادگر (CNN) | تشخیص چهره| کمترین از دست دادن حاشیه ای (MML)
|مقاله ترجمه شده|
Radiological images and machine learning: Trends, perspectives, and prospects
تصاویر رادیولوژی و یادگیری ماشین: روند، دیدگاه ها، و چشم انداز-2019
The application of machine learning to radiological images is an increasingly active research area that is expected to grow in the next five to ten years. Recent advances in machine learning have the potential to recognize and classify complex patterns from different radiological imaging modalities such as x-rays, computed tomography, magnetic resonance imaging and positron emission tomography imaging. In many applications, machine learning based systems have shown comparable performance to human decision-making. The applications of machine learning are the key ingredients of future clinical decision making and monitoring systems. This review covers the fundamental concepts behind various machine learning techniques and their applications in several radiological imaging areas, such as medical image segmentation, brain function studies and neurological disease diagnosis, as well as computer-aided systems, image registration, and content-based image retrieval systems. Synchronistically, we will briefly discuss current challenges and future directions regarding the application of machine learning in radiological imaging. By giving insight on how take advantage of machine learning powered applications, we expect that clinicians can prevent and diagnose diseases more accurately and efficiently.
Keywords: Deep learning | Machine learning | Imaging modalities | Deep neural networ
Transfer learning of deep neural network representations for fMRI decoding
انتقال یادگیری بازنمایی های شبکه عصبی عمیق برای رمزگشایی fMRI-2019
Background: Deep neural networks have revolutionised machine learning, with unparalleled performance in object classification. However, in brain imaging (e.g., fMRI), the direct application of Convolutional Neural Networks (CNN) to decoding subject states or perception from imaging data seems impractical given the scarcity of available data. New method: In this work we propose a robust method to transfer information from deep learning (DL) features to brain fMRI data with the goal of decoding. By adopting Reduced Rank Regression with Ridge Regularisation we establish a multivariate link between imaging data and the fully connected layer (fc7) of a CNN. We exploit the reconstructed fc7 features by performing an object image classification task on two datasets: one of the largest fMRI databases, taken from different scanners from more than two hundred subjects watching different movie clips, and another with fMRI data taken while watching static images. Results: The fc7 features could be significantly reconstructed from the imaging data, and led to significant decoding performance. Comparison with existing methods: The decoding based on reconstructed fc7 outperformed the decoding based on imaging data alone. Conclusion: In this work we show how to improve fMRI-based decoding benefiting from the mapping between functional data and CNN features. The potential advantage of the proposed method is twofold: the extraction of stimuli representations by means of an automatic procedure (unsupervised) and the embedding of high-dimensional neuroimaging data onto a space designed for visual object discrimination, leading to a more manageable space from dimensionality point of view.
Keywords: Deep learning | Convolutional Neural Network | Transfer learning | Brain decoding | fMRI | MultiVoxel Pattern Analysis
A novel deep multi-criteria collaborative filtering model for recommendation system
یک مدل جدید عمیق مشترک چند منظوره برای سیستم توصیه-2019
Recommender systems have been in existence everywhere with most of them using single ratings in prediction. However, multi-criteria predictions have been proved to be more accurate. Recommender systems have many techniques; collaborative filtering is one of the most commonly used. Deep learning has achieved impressive results in many domains such as text, voice, and computer vision. Lately, deep learning for recommender systems began to gain massive interest, and many recommendation models based on deep learning have been proposed. However, as far as we know, there is not yet any study which gathers multi-criteria recommendation and collaborative filtering with deep learning. In this work, we propose a novel multi-criteria collaborative filtering model based on deep learning. Our model contains two parts: in the first part, the model obtains the users and items’ features and uses them as an input to the criteria ratings deep neural network, which predicts the criteria ratings. Those criteria ratings constitute the input to the second part, which is the overall rating deep neural network and is used to predict the overall rating. Experiments on a realworld dataset demonstrate that our proposed model outperformed the other state-of-the-art methods, and this provides evidence pointing to the success of employing deep learning and multi-criteria in recommendation systems
Keywords: Collaborative filtering | Deep learning | Deep neural network | Multi-criteria | Recommender system
A novel deep learning based method for the computational material design of flexoelectric nanostructures with topology optimization
یک روش مبتنی بر یادگیری عمیق برای طراحی مواد محاسباتی نانوساختارهای فلکسوالکتریک با بهینه سازی توپولوژی-2019
We present a deep learning method to investigate the effect of flexoelectricity in nanostructures. For this purpose, deep neural network (DNN) algorithm is employed to map the relation between the inputs and the material response of interest. The DNN model is trained and tested making use of database that has been established by solving the governing equations of flexoelectricity using a NURBS-based IGA formulation at design points in the full probability space of the input parameters. Firstly, pure flexoelectric cantilever nanobeam is investigated under mechanical and electrical loading conditions. Then, structures of composite system constituted by two nonpiezoelectric material phases are addressed in order to find the optimized topology with respect to the energy conversion factor. The results show promising capabilities of the proposed method, in terms of accuracy and computational efficiency. The deep learning method we used have produced superior optimal designs compared to the numerical methods. The findings of this study will be of profound interest to researcher involved further in the optimization and design of flexoelectric structures.
Keywords: Flexoelectricity | Piezoelectricity | Isogeometric analysis (IGA) | Machine learning | Deep neural network | Topology optimization
DNNRec: A novel deep learning based hybrid recommender system
DNNRec: یک سیستم توصیه گر ترکیبی مبتنی بر یادگیری عمیق-2019
We propose a novel deep learning hybrid recommender system to address the gaps in Collaborative Fil- tering systems and achieve the state-of-the-art predictive accuracy using deep learning. While collabo- rative filtering systems are popular with many state-of-the-art achievements in recommender systems, they suffer from the cold start problem, when there is no history about the users and items. Further, the latent factors learned by these methods are linear in nature. To address these gaps, we describe a novel hybrid recommender system using deep learning. The solution uses embeddings for representing users and items to learn non-linear latent factors. The solution alleviates the cold start problem by inte- grating side information about users and items into a very deep neural network. The proposed solution uses a decreasing learning rate in conjunction with increasing weight decay, the values cyclically varied across epochs to further improve accuracy. The proposed solution is benchmarked against existing meth- ods on both predictive accuracy and running time. Predictive Accuracy is measured by Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and R-squared. Running time is measured by the mean and standard deviation across seven runs. Comprehensive experiments are con- ducted on several datasets such as the MovieLens 100 K, FilmTrust, Book-Crossing and MovieLens 1 M. The results show that the proposed technique outperforms existing methods in both non-cold start and cold start cases. The proposed solution framework is generic from the outperformance on four different datasets and can be leveraged for other ratings prediction datasets in recommender systems.
Keywords: Deep learning | Recommender systems | Embeddings | Side information | Cyclical learning rates | Deep neural network | Cold start problem