با سلام خدمت کاربران در صورتی که با خطای سیستم پرداخت بانکی مواجه شدید از طریق کارت به کارت (6037997535328901 بانک ملی ناصر خنجری ) مقاله خود را دریافت کنید (تا مشکل رفع گردد).
ردیف | عنوان | نوع |
---|---|---|
1 |
Structured pruning of recurrent neural networks through neuron selection
هرس ساختاری شبکه های عصبی مکرر از طریق انتخاب نورون-2020 Recurrent neural networks (RNNs) have recently achieved remarkable successes in a number of
applications. However, the huge sizes and computational burden of these models make it difficult for
their deployment on edge devices. A practically effective approach is to reduce the overall storage and
computation costs of RNNs by network pruning techniques. Despite their successful applications, those
pruning methods based on Lasso either produce irregular sparse patterns in weight matrices, which
is not helpful in practical speedup. To address these issues, we propose a structured pruning method
through neuron selection which can remove the independent neuron of RNNs. More specifically, we
introduce two sets of binary random variables, which can be interpreted as gates or switches to
the input neurons and the hidden neurons, respectively. We demonstrate that the corresponding
optimization problem can be addressed by minimizing the L0 norm of the weight matrix. Finally,
experimental results on language modeling and machine reading comprehension tasks have indicated
the advantages of the proposed method in comparison with state-of-the-art pruning competitors. In
particular, nearly 20× practical speedup during inference was achieved without losing performance
for the language model on the Penn TreeBank dataset, indicating the promising performance of the
proposed method. Keywords: Feature selection | Recurrent neural networks | Learning sparse models | Model compression |
مقاله انگلیسی |
2 |
Sparse low rank factorization for deep neural network compression
فاکتورسازی رتبه پراکنده برای فشرده سازی شبکه عصبی عمیق-2020 Storing and processing millions of parameters in deep neural networks is highly challenging during the deployment of model in real-time application on resource constrained devices. Popular low-rank approx- imation approach singular value decomposition (SVD) is generally applied to the weights of fully con- nected layers where compact storage is achieved by keeping only the most prominent components of the decomposed matrices. Years of research on pruning-based neural network model compression re- vealed that the relative importance or contribution of each neuron in a layer highly vary among each other. Recently, synapses pruning has also demonstrated that having sparse matrices in network archi- tecture achieve lower space and faster computation during inference time. We extend these arguments by proposing that the low-rank decomposition of weight matrices should also consider significance of both input as well as output neurons of a layer. Combining the ideas of sparsity and existence of un- equal contributions of neurons towards achieving the target, we propose sparse low rank (SLR) method which sparsifies SVD matrices to achieve better compression rate by keeping lower rank for unimportant neurons. We demonstrate the effectiveness of our method in compressing famous convolutional neural networks based image recognition frameworks which are trained on popular datasets. Experimental re- sults show that the proposed approach SLR outperforms vanilla truncated SVD and a pruning baseline, achieving better compression rates with minimal or no loss in the accuracy. Code of the proposed ap- proach is avaialble at https://github.com/sridarah/slr . Keywords: Low-rank approximation | Singular value decomposition | Sparse matrix | Deep neural networks | Convolutional neural networks |
مقاله انگلیسی |