با سلام خدمت کاربران عزیز، به اطلاع می رساند ترجمه مقالاتی که سال انتشار آن ها زیر 2008 می باشد رایگان بوده و میتوانید با وارد شدن در صفحه جزییات مقاله به رایگان ترجمه را دانلود نمایید.
A mesh-free method for interface problems using the deep learning approach
روشی بدون مش برای مشکلات رابط با استفاده از روش یادگیری عمیق-2020
In this paper, we propose a mesh-free method to solve interface problems using the deep learning approach. Two types of PDEs are considered. The first one is an elliptic PDE with a discontinuous and high-contrast coefficient. While the second one is a linear elasticity equation with discontinuous stress tensor. In both cases, we represent the solutions of the PDEs using the deep neural networks (DNNs) and formulate the PDEs into variational problems, which can be solved via the deep learning approach. To deal with inhomogeneous boundary conditions, we use a shallow neural network to approximate the boundary conditions. Instead of using an adaptive mesh refinement method or specially designed basis functions or numerical schemes to compute the PDE solutions, the proposed method has the advantages that it is easy to implement and is mesh-free. Finally, we present numerical results to demonstrate the accuracy and efficiency of the proposed method for interface problems.
Keywords: Deep learning | Variational problems | Mesh-free method | Linear elasticity | High-contrast | Interface problems
Goal driven network pruning for object recognition
هرس شبکه هدف محور برای شناسایی هدف -2020
Pruning studies up to date focused on uncovering a smaller network by removing redundant units, and fine-tuning to compensate accuracy drop as a result. In this study, unlike the others, we propose an approach to uncover a smaller network that is competent only in a specific task, similar to top-down attention mechanism in human visual system. This approach doesn’t require fine-tuning and is proposed as a fast and effective alternative of training from scratch when the network focuses on a specific task in the dataset. Pruning starts from the output and proceeds towards the input by computing neuron importance scores in each layer and propagating them to the preceding layer. In the meantime, neurons determined as worthless are pruned. We applied our approach on three benchmark datasets: MNIST, CIFAR-10 and ImageNet. The results demonstrate that the proposed pruning method typically reduces computational units and storage without harming accuracy significantly.
Keywords: Deep learning | Computer vision | Network pruning | Network compressing | Top-down attention | Perceptual visioning
From chemical structure to quantitative polymer properties prediction through convolutional neural networks
از ساختار شیمیایی گرفته تا پیش بینی کمی از خواص پلیمر از طریق شبکه های عصبی در هم تنیده -2020
In this work convolutional-fully connected neural networks were designed and trained to predict the glass transition temperature of polymers based only on their chemical structure. This approach has shown to successfully predict the Tg of unknown polymers with average relative errors as low as 6%. Several networks with different architecture or hiperparameters were successfully trained using a previously studied glass transition temperatures dataset for validation, and then the same method was employed for an extended dataset, with larger Tg dispersion and polymer’s structure variability. This approach has shown to be accurate and reliable, and does not require any time consuming or expensive measurements and calculations as inputs. Furthermore, it is expected that this method can be easily extended to predict other properties. The possibility of predicting the properties of polymers not even synthesized will save time and resources for industrial development as well as accelerate the scientific understanding of structure-properties relationships in polymer science.
Keywords: QSPR | Properties prediction | Deep learning | Neural network | Smart design
Predicting academic performance of students from VLE big data using deep learning models
پیش بینی عملکرد علمی دانش آموزان از داده های بزرگ VLE با استفاده از مدل های یادگیری عمیق-2020
The abundance of accessible educational data, supported by the technology-enhanced learning platforms, provides opportunities to mine learning behavior of students, addressing their issues, optimizing the educational environment, and enabling data-driven decision making. Virtual learning environments complement the learning analytics paradigm by effectively providing datasets for analysing and reporting the learning process of students and its reflection and contribution in their respective performances. This study deploys a deep artificial neural network on a set of unique handcrafted features, extracted from the virtual learning environments clickstream data, to predict at-risk students providing measures for early intervention of such cases. The results show the proposed model to achieve a classification accuracy of 84%–93%. We show that a deep artificial neural network outperforms the baseline logistic regression and support vector machine models. While logistic regression achieves an accuracy of 79.82%–85.60%, the support vector machine achieves 79.95%–89.14%. Aligned with the existing studies - our findings demonstrate the inclusion of legacy data and assessment-related data to impact the model significantly. Students interested in accessing the content of the previous lectures are observed to demonstrate better performance. The study intends to assist institutes in formulating a necessary framework for pedagogical support, facilitating higher education decision-making process towards sustainable education.
Keywords: Learning analytics | Predicting success | Educational data | Machine learning | Deep learning | Virtual learning environments (VLE)
Ensemble sparse supervised model for bearing fault diagnosis in smart manufacturing
گروه نظارت پراکنده برای تشخیص خطا در تولید هوشمند ، مدل نظارت پراکنده-2020
Machinery fault diagnosis is of great significance to improve the reliability of smart manufacturing. Deep learning based fault diagnosis methods have achieved great success. However, the features extracted by different models may vary resulting in ambiguous representation of the data, and even wasted time with manually selecting the optimal hyperparameters. To solve the problems, this paper proposes a new framework named Ensemble Sparse Supervised Model (ESSM), in which a typical deep learning model is treated as two phases of feature learning and model learning. In the feature learning phase, the original data is represented to be a feature matrix as non-redundant as possible by applying sparse filtering. Then, the feature matrix is fed into the model learning phase. Regularization, dropout and rectified linear unit (ReLU) are used in the models neurons and layers to build a sparse deep neural network. Finally, the output of the sparse deep neural network provides feedback to the first phase to obtain better sparse features. In the proposed method, hyperparameters need to be pre-specified and a python library of talos is employed to finish the process automatically. The proposed method is verified using the bearing data provided by Case Western Reserve University. The result demonstrates that the proposed method can capture the effective pattern of data with the help of sparse constraints and simultaneously provide convenience for the operators with assuring performance.
Keywords: Sparse representation | Deep learning | Fault diagnosis
Detection and classification of bruises of pears based on thermal images
تشخیص و طبقه بندی کبودی گلابی بر اساس تصاویر حرارتی-2020
The detection and classification of bruises of pears based on thermal images have been investigated. A simple thermal imaging system in the long-wavelength ranges (8–14 μm) assembledμwith hot air equipment was constructed to capture cleaner images. Higher velocity and temperature of the air reduced the time required to obtain a clean image, but the images were not sufficient able to discriminate the slight and invisible variation of bruises over consecutive days. The grey-level co-occurrence matrix of the thermal images were analysed, and the slight differences in the pears over consecutive days were presented in the form of a line chart. A traditional deep learning algorithm commonly used in classification of big data sets was modified to one suitable for classification of a small sample data set of thermasl images (3246 samples were used as the training data set and 1125 were used as a test data set) collected from 300 pears over 10 days. The best test prediction accuracy obtained was 99.25%.
Keywords: Detection and classification | Thermal images | Grey-level co-occurrence matrix | Deep learning
Universal approximation with quadratic deep networks
تخمین جهانی با شبکه های عمیق درجه دوم-2020
Recently, deep learning has achieved huge successes in many important applications. In our previous studies, we proposed quadratic/second-order neurons and deep quadratic neural networks. In a quadratic neuron, the inner product of a vector of data and the corresponding weights in a conventional neuron is replaced with a quadratic function. The resultant quadratic neuron enjoys an enhanced expressive capability over the conventional neuron. However, how quadratic neurons improve the expressing capability of a deep quadratic network has not been studied up to now, preferably in relation to that of a conventional neural network. Specifically, we ask four basic questions in this paper: (1) for the one-hidden-layer network structure, is there any function that a quadratic network can approximate much more efficiently than a conventional network? (2) for the same multi-layer network structure, is there any function that can be expressed by a quadratic network but cannot be expressed with conventional neurons in the same structure? (3) Does a quadratic network give a new insight into universal approximation? (4) To approximate the same class of functions with the same error bound, could a quantized quadratic network have a lower number of weights than a quantized conventional network? Our main contributions are the four interconnected theorems shedding light upon these four questions and demonstrating the merits of a quadratic network in terms of expressive efficiency, unique capability, compact architecture and computational capacity respectively.
Keywords: Deep learning | Quadratic networks | Approximation theory
Learning in the machine: To share or not to share?
یادگیری در دستگاه: برای به اشتراک گذاشتن یا عدم اشتراک گذاری؟-2020
Weight-sharing is one of the pillars behind Convolutional Neural Networks and their successes. However, in physical neural systems such as the brain, weight-sharing is implausible. This discrepancy raises the fundamental question of whether weight-sharing is necessary. If so, to which degree of precision? If not, what are the alternatives? The goal of this study is to investigate these questions, primarily through simulations where the weight-sharing assumption is relaxed. Taking inspiration from neural circuitry, we explore the use of Free Convolutional Networks and neurons with variable connection patterns. Using Free Convolutional Networks, we show that while weight-sharing is a pragmatic optimization approach, it is not a necessity in computer vision applications. Furthermore, Free Convolutional Networks match the performance observed in standard architectures when trained using properly translated data (akin to video). Under the assumption of translationally augmented data, Free Convolutional Networks learn translationally invariant representations that yield an approximate form of weight-sharing.
Keywords: Deep learning | Convolutional neural networks | Weight-sharing | Biologically plausible architectures
Noise can speed backpropagation learning and deep bidirectional pretraining
سر و صدا می تواند یادگیری پشت پرده و پیشبرد عمیق دو طرفه را سرعت بخشد-2020
We show that the backpropagation algorithm is a special case of the generalized Expectation-Maximization (EM) algorithm for iterative maximum likelihood estimation. We then apply the recent result that carefully chosen noise can speed the average convergence of the EM algorithm as it climbs a hill of probability. Then injecting such noise can speed the average convergence of the backpropagation algorithm for both the training and pretraining of multilayer neural networks. The beneficial noise adds to the hidden and visible neurons and related parameters. The noise also applies to regularized regression networks. This beneficial noise is precisely the noise that makes the current signal more probable. We show that such noise also tends to improve classification accuracy. The geometry of the noise-benefit region depends on the probability structure of the neurons in a given layer. The noise-benefit region in noise space lies above the noisy-EM (NEM) hyperplane for classification and involves a hypersphere for regression. Simulations demonstrate these noise benefits using MNIST digit classification. The NEM noise benefits substantially exceed those of simply adding blind noise to the neural network. We further prove that the noise speed-up applies to the deep bidirectional pretraining of neural-network bidirectional associative memories (BAMs) or their functionally equivalent restricted Boltzmann machines. We then show that learning with basic contrastive divergence also reduces to generalized EM for an energy-based network probability. The optimal noise adds to the input visible neurons of a BAM in stacked layers of trained BAMs. Global stability of generalized BAMs guarantees rapid convergence in pretraining where neural signals feed back between contiguous layers. Bipolar coding of inputs further improves pretraining performance.
Keywords: Backpropagation | neural networks | noise benefit | stochastic resonance | Expectation-Maximization algorithm | bidirectional associative memory | deep learning | regularization | pretraining | contrastive divergence
Deep learning model for end-to-end approximation of COSMIC functional size based on use-case names
مدل یادگیری عمیق برای تخمین پایان به پایان اندازه کاربردی COSMIC بر اساس نامهای مورد استفاده-2020
Context: COSMIC is a widely used functional size measurement (FSM) method that supports software development effort estimation. The FSM methods measure functional product size based on functional requirements. Unfortu- nately, when the description of the product’s functionality is often abstract or incomplete, the size of the product can only be approximated since the object to be measured is not yet fully described. Also, the measurement performed by human-experts can be time-consuming, therefore, it is worth considering automating it. Objective: Our objective is to design a new prediction model capable of approximating COSMIC-size of use cases based only on their names that is easier to train and more accurate than existing techniques. Method: Several neural-network architectures are investigated to build a COSMIC size approximation model. The accuracy of models is evaluated in a simulation study on the dataset of 437 use cases from 27 software develop- ment projects in the Management Information Systems (MIS) domain. The accuracy of the models is compared with the Average Use-Case approximation (AUC), and two recently proposed two-step models —Average Use-Case Goal-aware Approximation (AUCG) and Bayesian Network Use-Case Goal AproxImatioN (BN-UCGAIN). Results: The best prediction accuracy was obtained for a convolutional neural network using a word-embedding model trained on Wikipedia + Gigaworld. The accuracy of the model outperformed the baseline AUC model by ca. 20%, and the two-step models by ca. 5–7%. In the worst case, the improvement in the prediction accuracy is visible after estimating 10 use cases. Conclusions: The proposed deep learning model can be used to automatically approximate COSMIC size of software applications for which the requirements are documented in the form of use cases (or at least in the form of use- case names). The advantage of the model is that it does not require collecting historical data other than COSMIC size and names of use cases.
Keywords: Functional size approximation | Approximate software sizing methods | COSMIC | Deep learning | Word embeddings | Use cases